Bridging urban theory and artificial intelligence: a multi-agent recommendation system for sustainable city development

Abstract

As cities increasingly rely on AI for sustainability challenges, a critical gap emerges: AI applications in urban planning and safety predominantly proceed without explicit guidance from established urban theories that have guided sustainable development for decades. Our analysis reveals that technology-driven research dominates the field, while problem-driven approaches addressing genuine urban needs remain minimal. To bridge this theory-practice disconnect, we develop a Large Language Model (LLM)-based multi-agent recommendation system [publicly available online] that realigns AI development with sustainable city principles. The system employs specialized agents to recommend appropriate theoretical frameworks, AI methods, and data sources for urban challenges, drawing from classical urban theories. Through diverse case studies, we demonstrate how our approach transforms technology-focused solutions into theory-grounded interventions that address sustainability’s interconnected dimensions. Our framework fundamentally shifts the question from “what can algorithms do?” to “what does this urban challenge require for sustainable outcomes?”—ensuring AI amplifies rather than replaces the theoretical wisdom essential for creating resilient, equitable, and livable cities that contribute to global sustainability targets.

Introduction

Artificial intelligence (AI) offers unprecedented opportunities for optimizing urban systems as cities worldwide race to meet sustainability targets. Yet, our large-scale empirical analysis of 1123 AI applications in urban planning and safety across global contexts reveals a critical misalignment: current AI applications in urban safety and planning areas proliferate in a theoretical vacuum, prioritizing technical metrics over the social, environmental, and spatial dynamics essential for sustainable cities^1,2. Without grounding in established urban wisdom, even the most sophisticated AI system may inadvertently compromise the resilience, equity, and livability that future generations depend upon^3,4. Recent frameworks further demonstrate AI’s potential to operationalize sustainability targets through systemic approaches addressing emissions, inequality, and exclusion simultaneously⁵, yet such integration requires explicit theoretical grounding to avoid techno-solutionist pitfalls⁶.

This misalignment stems from AI development processes that bypass decades of urban theoretical knowledge. Established urban theories—from Jacobs’ social surveillance principles⁷ to Newman’s spatial security frameworks—demonstrate that sustainable cities emerge from integrated social-spatial-economic systems. But as AI proliferates in urban sustainability efforts—from carbon reduction to equitable resource distribution—few applications engage with this legacy. Comprehensive reviews of AI, IoT, and big data convergence in sustainable smart cities⁸ confirm this pattern: despite impressive technological sophistication, theoretical integration remains minimal. Consequently, these applications risk replicating a pattern that scholars have critiqued: optimising for efficiency while neglecting the holistic principles of sustainable urbanism^6,9,10.

Quantitative analysis confirms limited explicit theoretical integration. Among 1123 studies representing 151.6-fold growth since 2008, only 1.16% explicitly cite established urban theories by name. Nearly half (47.3%) are technology-driven, focused on algorithmic capabilities rather than urban crises (8.6%)¹¹. This imbalance demonstrates that urban AI research predominantly asks “what can AI do?” rather than “what do cities need for sustainable development?”—signaling a fundamental misalignment between how AI is being developed and how sustainable cities actually function^12,13. This misalignment has intensified dramatically over time, with particularly concerning implications for urban sustainability. Since 2020 alone, over 60% of all urban AI applications have emerged, coinciding with the proliferation of pre-trained models and accessible AI frameworks. The acceleration is striking: newer algorithms achieve near-instantaneous adoption. Contrastive Language-Image Pre-training (CLIP) and Generative Pre-trained Transformer 3 (GPT-3) were applied to urban problems within the same year of their release, reflecting intense pressure to demonstrate innovation regardless of sustainability outcomes¹⁴. Consequently, the field increasingly prioritizes algorithmic novelty over addressing genuine urban sustainability challenges, further widening the gap between technological capabilities and urban needs^15,16.

We address this gap through a theory-first AI recommendation framework that prioritizes urban sustainability requirements over algorithmic capabilities. This approach advances recent proposals for human-AI symbiosis in urban informatics¹⁷, positioning AI not as an autonomous solution but as a collaborative partner that amplifies human expertise. Specifically, our system employs five specialized agents working sequentially—from problem structuring to theory retrieval, algorithm matching, data selection, and integration validation. Through this process, the framework transforms the core question from “what can this algorithm do?” to “what does this urban challenge require?”¹⁸. To achieve this, the system maps alignments between 46 classical urban theories and contemporary AI methods, thereby generating recommendations that integrate sustainability’s social, environmental, and economic dimensions¹⁹. We validate this approach through three case studies spanning food security, urban heat mitigation, and disaster resilience. The results demonstrate how theory-grounded AI generates solutions aligned with Sustainable Development Goal (SDG) 11 targets, ultimately enabling cities to transcend technical performance metrics and achieve genuine sustainability outcomes^20,21.

Results

The technology-first paradigm

Our empirical investigation examines four key dimensions of AI adoption in urban planning and safety domains: algorithmic preferences, data source utilization, theoretical integration, and research motivations. Our systematic analysis reveals the mechanisms driving the technology-first paradigm. Such evidence establishes the foundation for understanding the theory-practice disconnect in urban AI applications.

We retrieved an initial corpus of 2797 papers from the Web of Science (WoS)²² and Scopus²³ databases using the targeted search query ("urban planning” OR “urban safety”) AND “AI”, with no time restriction, and performed initial data cleaning. After further data cleaning and duplicate removal, we conducted systematic screening to ensure quality. A BERT-based classifier performed domain-specific classification (see Supplementary Material 8, Table S10). We then filtered the results to peer-reviewed research articles²⁴, ultimately identifying 1123 genuine AI applications in the urban planning and safety domains. This BERT-based approach follows established practices in natural language processing²⁵, with similar methods recently applied to the classification of urban sustainability literature, achieving comparable accuracy levels²⁶. These 1123 validated papers form the foundation for examining the current state of AI integration in urban planning and safety management.

Four distinct algorithm groups and six data source categories emerged from clustering analysis. Dimensionality reduction using t-distributed Stochastic Neighbour Embedding (t-SNE) reveals distinct clustering patterns in both algorithmic approaches and data sources utilised in urban planning and safety AI research (Table 1, Fig. 1). The t-SNE method has proven effective for visualizing high-dimensional urban data structures^27,28, with recent theoretical advances supporting its reliability for cluster detection in complex datasets²⁹.

Fig. 1: Algorithm and data source grouping visualisation using t-SNE dimensionality reduction.

a Four algorithm groups identified through clustering, with Machine Learning & Deep Learning (Group 2) showing highest density (42.7% of papers). b Six data source groups revealing clear separation between traditional (Groups 0–1, 68.3% combined) and emerging data types (Groups 3–5, 19.2% combined).

Table 1 Algorithm and data source grouping summary

Machine Learning (ML) and Deep Learning methods (Group 2) dominate the algorithmic landscape, comprising 42.7% of all methods employed and forming the densest cluster (Fig. 1a)—a finding consistent with recent systematic reviews documenting the proliferation of ML approaches in urban applications^30,31. Traditional optimization and statistical approaches (Groups 0–1: Optimization & Decision Methods, and Statistical & Spatial Analysis) demonstrate established methodological foundations, while the emergence of Generative & Pre-trained Models (Group 3) reflects recent advances in AI capabilities for urban applications.

Environmental & Climate Data and Sensor & Internet of Things (IoT) Data (Groups 0–1) exhibit the highest utilization among data sources, accounting for 68.3% of all data usage (Table 1, Fig. 1b). This pattern reflects the maturity and accessibility of environmental monitoring systems and sensor networks in urban contexts^32,33. Composite Data, Regional & Experimental Data, and Image & Remote Sensing Data (Groups 3–5) account for only 19.2% of combined usage (Fig. 1b), despite all 46 theories requiring at least one of these categories at the recommended relevance threshold (≥0.6), with 69.6% requiring two or more. Meanwhile, Environmental & Sensor data (Groups 0–1) is the only grouping where observed usage exceeds theory-derived demand (68.3% vs. 58.7%), consistent with technology-driven selection favoring readily available automated sources³⁴. Geographic Information & Social Survey Data (Group 2) occupies an intermediate position, bridging traditional geospatial approaches with contemporary data collection methods. The difference in utilization between established environmental/sensor data sources and emerging composite data approaches was statistically significant (χ² = 156.4, p < 0.001).

Only 13 of 1123 papers (1.16%) demonstrated explicit theoretical integration by citing established urban planning or safety theories by name. This analysis reveals limited visible theoretical communication between AI applications and established urban frameworks—a pattern that recent critiques have identified as problematic for sustainable urban development³⁵. None of the 63 Random Forest applications analyzing urban spatial patterns reference spatial interaction theories. Similarly, Convolutional Neural Network (CNN) - based urban image analyses (n = 47) contain no citations to established visual assessment frameworks from urban design literature³⁶. Among the 13 papers that did engage with theory, 10 were authored by interdisciplinary teams including at least one urban planning or safety expert, suggesting the importance of cross-disciplinary collaboration³⁷. This theoretical vacuum is particularly concerning given the diversity of algorithmic approaches identified (Table 1), each of which requires domain-specific adaptation.

The distribution of research motivations (see Supplementary Material 3) (reveals a technology-first paradigm dominating urban planning and safety AI studies (Fig. 2a). Technology-driven papers constitute 47.3% of the corpus (531 papers), focusing on exploring new technological applications, algorithm development, and method transfer—a trend multiple reviews have identified as misaligned with urban planning needs^38,39. Method-driven research accounts for 14.0% (157 papers) and addresses the limitations of existing solutions through performance improvements or efficiency enhancements. In stark contrast, problem-driven research initiated by specific urban crises or real-world events comprises only 8.6% (97 papers)—examples include Beijing’s traffic congestion incidents, New York City’s flooding events, and Tokyo’s earthquake evacuation crises. General needs-driven studies account for 17.0% (191 papers) and address broad urban requirements without specific triggering events. The remaining 147 papers (13.1%) exhibit unclear or mixed motivations (Table 2).

Fig. 2: Analysis of research paradigms in urban planning and safety AI studies.

a Distribution of research motivations among 1,123 papers, showing the dominance of technology-driven approaches (47.3%) over problem-driven research (8.6%). b Algorithm selection criteria priorities, revealing emphasis on technical novelty (59.0%) and computational efficiency (51.3%) over domain-specific applicability (12.8%) and geographical generalizability (5.1%).

Table 2 Research motivation distribution in urban planning and safety AI studies (N = 1123)

Algorithm selection criteria analysis reveals a distinct hierarchy in researchers’ priorities (Fig. 2b): technical novelty (59.0%), algorithmic performance (48.7%), computational efficiency (51.3%), ease of implementation (10.3%), domain-specific applicability (12.8%), and geographical generalizability (5.1%). This distribution demonstrates a clear preference for technical metrics over practical relevance in urban planning applications. Rigorous domain validation analysis reveals significant gaps in verification practices. Only 12.8% of studies conduct meaningful domain-specific verification. More critically, comprehensive multi-method validation is completely absent (0.0%). Geographical generalizability represents another critical gap. This factor is essential for urban planning algorithms intended for diverse global contexts, yet remains minimally addressed. Only 5.1% of studies conduct rigorous cross-city validation, while 97.4% fail to address cultural adaptability.

Validation of theoretical integration measurement

Our 1.16% finding captures explicit theoretical citations, but may not reflect the full extent of theoretical engagement. To evaluate this measurement’s validity and explore implicit theoretical foundations, we conducted a three-method validation protocol on a stratified subsample (n = 200, see Section “Validation of Theoretical Integration Measurement” and Supplementary Material 10). Automated classification using GPT-4-turbo to detect theoretical engagement from introduction sections achieved only 43.0% accuracy (86/200 correct; Cohen’s κ = 0.35) against expert coding. Among 114 misclassifications, primary errors included over-interpretation of technical descriptions (36 cases, 31.6%), missing implicit theoretical cues (32 cases, 28.1%), and surface citation misjudgment (29 cases, 25.4%). For example, the model classified Xiang et al.’s water resource management study⁴⁰ as “theory-driven” by misinterpreting sustainability terminology, while missing implicit travel behavior theory in Xin et al.’s mobility analysis⁴¹. This poor performance demonstrates why explicit citation criteria became necessary—automated semantic analysis cannot reliably detect domain-specific theoretical foundations at scale.

Team composition analysis revealed paradoxical patterns. Contrary to expectations, disciplinary diversity showed no significant correlation with theory-driven intensity (Pearson’s r = 0.08, p = 0.267; Spearman’s ρ = 0.09, p = 0.221). However, pure urban planning teams scored highest (0.65 ± 0.24), followed by interdisciplinary teams (0.39 ± 0.23) and pure computer science teams (0.24 ± 0.18). To further validate these team-level patterns and explore implicit theoretical engagement, semantic theoretical density (STD) analysis using an 847-term theoretical lexicon revealed moderate correlation with expert assessment (r = 0.62), slightly higher than explicit citation (r = 0.58), though not significantly different (Steiger’s Z = 0.54, p = 0.59). Critically, all three measurement approaches demonstrated identical rank ordering across team types (Table 3), with semantic density showing Pure Planning teams scoring 65.7% higher than Pure CS teams (0.58 vs 0.35; t = 6.83, p < 0.001, Cohen’s d = 1.34). Among papers lacking explicit citations (n = 165, 82.5%), semantic density varied substantially (range: 0.18–0.74, SD = 0.19). Manual review of top-quartile cases (n = 31) revealed that 87.1% (27/31) demonstrated sophisticated domain knowledge through problem framing and methodological choices.

Table 3 Three-method validation results: measurement convergence analysis (N = 200)

Temporal evolution: from problem-pull to technology-push

To understand the drivers of AI adoption in urban planning and safety, we analyze the temporal gap between algorithm development and domain application across all 1123 papers. Our corpus captures self-labeled AI research–papers explicitly using “AI" terminology–rather than the complete history of computational methods in urban domains. The timeline visualization (Fig. 3) reveals an important distinction about algorithmic adoption. Rather than showing when methods first entered urban research, it captures when they became reframed under the AI label. Such reframing documents shifts in disciplinary discourse, consistent with Rogers’ diffusion of innovation theory⁴².

Fig. 3: Temporal evolution of AI algorithm adoption in urban planning and safety research (2008–2025).

a Group-level timelines from development (left genesis points) to AI-labeled application (right scatter points). Genesis circles mark the earliest algorithms; c–f reveal within-group heterogeneity (Table 4). The “AI Terminology Adoption Surge” (2008) marks when methods became systematically framed as AI, not when developed. Inset boxplot shows the latency distribution; for Groups 0--1, the means reflect all algorithms, not early outliers. b Application share 2020--2025 by group: G0 (13.8%), G1 (21.7%), G2 (47.1%), G3 (17.4%), showing G2 dominance. Algorithm-level heterogeneity: (c) G0, CV = 237%; (d) G1, CV = 38%; (e) G2, CV = 32%; (f) G3, CV = 16%. Color intensity represents application frequency. Dates reflect the first AI-labeled appearance; many methods existed in urban research decades earlier.

Our analysis reveals a dramatic transformation in AI adoption patterns. Prior to 2008, merely 14 papers (0.7%) employed AI methods in urban planning and safety contexts. Following 2008, applications surge to 1109 papers (99.3%)–a 151.6-fold increase visible in Fig. 3a. This transformation coincides with the broader AI renaissance, driven by computational advances and data availability⁴³, rather than with emerging urban challenges requiring AI solutions. Critically, this growth reflects the proliferation of AI terminology and framing: many algorithms shown in Fig. 3 (particularly Groups 0–1) were established in planning practice decades earlier^44,45, their appearance here marking terminological repackaging rather than initial methodological introduction.

Temporal evolution unfolds in three distinct phases clearly delineated in Fig. 3. Critically, the 2008 inflection point labeled “AI Terminology Adoption Surge” in panel (a) marks systematic reframing under AI terminology following deep learning breakthroughs, not algorithm development or latency occurrence–this initiates Phase 1 (2008–2012), characterized by sparse, experimental adoption of existing algorithms. Subsequently, Phase 2 (2012–2017) witnesses a machine learning revolution penetrating urban domains following AlexNet’s breakthrough demonstration⁴⁶, exemplifying technology-push wherein innovations diffuse based on capability rather than domain need². Finally, Phase 3 (2018–2025) represents an unprecedented explosion visible in panel (b): 2020–2025 alone contributes over 60% of all applications, driven by pre-trained models and accessible AI frameworks.

The latency between algorithm development and urban application reveals fundamental patterns in technology transfer consistent with diffusion theory⁴². Rather than absolute latency values, what matters most is their dramatic compression across time: Group 3 algorithms demonstrate near-instantaneous adoption—CLIP achieves same-year development and application (2021), while GPT-3 enters urban planning within months of release⁴⁷. Such compression from decades (Groups 0–2) to essentially zero (Group 3), illustrated by converging lines in Fig. 3a, reflects intensifying pressure to demonstrate AI innovation regardless of domain relevance. Within-group heterogeneity (CV) decreases from 237% (G0) to 16% (G3) (Table 4), demonstrating convergence toward uniform, immediate adoption. Group-specific patterns shown in Fig. 3c–f further reveal mounting innovation pressure across the field.

Table 4 Algorithm adoption patterns in urban AI research

Building on the latency patterns identified above, the 2020–2025 period represents an inflection point in algorithm density and diversity (Fig. 3b). The AlexNet moment of 2012^46,48—when deep learning achieves a 15.3% top-5 error rate compared to 26.2% for traditional methods—catalyzes widespread AI adoption across domains. This “winner-take-all” performance gap creates what innovation theorists call a “technology imperative”⁴², in which adoption is driven by the fear of being left behind rather than genuine utility.

Group 2 algorithms dominate the landscape with Random Forest (RF)⁴⁹, Support Vector Machines (SVM)⁵⁰, eXtreme Gradient Boosting (XGBoost)⁵¹, and Convolutional Neural Networks (CNNs)⁴³ (28) applied across disparate urban challenges, as detailed in Fig. 3e. The algorithm distributions reveal concerning patterns of methodological misalignment: CNNs designed for image recognition⁴³ are repurposed for non-visual planning problems, while natural language models process numerical safety metrics.

Group 3’s emergence, visualized in Fig. 3f, exemplifies the innovation imperative driving the field. Generative Adversarial Networks produce urban designs (17 applications) without engaging design theory, BERT analyzes planning documents while ignoring planning discourse analysis frameworks²⁵, and GPT models automate report generation absent planning communication principles. From 2021 to 2025, virtually every newly released AI algorithm finds immediate application in urban contexts, irrespective of theoretical fit or practical necessity—a pattern Rogers⁴² identifies as characteristic of hype-driven diffusion.

Simultaneously, Group 1 shows distinct patterns (Fig. 3d) with statistical and spatial analysis methods. While Group 0 maintains a steady presence (Fig. 3c), primarily in optimization tasks where mathematical foundations naturally align with planning objectives. These patterns, consistent with the latency characteristics shown in Table 4, demonstrate differential adoption dynamics across algorithm groups.

Bridging theory and practice: a computational mapping framework

To address the theory-practice disconnect identified in our analysis, we develop a computational framework that systematically maps urban theories to appropriate AI methods and data sources. Our approach transforms the question from “what can this algorithm do?” to “what theoretical principles should guide this urban intervention?” Through systematic analysis of 46 classical urban theories, we establish explicit connections between theoretical frameworks and computational approaches, providing a foundation for theory-driven AI development in urban contexts.

We employ NLP (Natural Language Processing) techniques to systematically identify and categorize 46 classical theories in urban planning and safety (see Supplementary Material 5, Table S6). Table 5 presents the most influential theories based on citation frequency and temporal persistence in the literature. Our computational linguistics framework extracts 127 distinct computational principles from the 46 urban theories (see Supplementary Material 5), revealing how classical theories inherently establish connections to modern AI methods. We select 11 representative theories spanning all four clusters for detailed mapping analysis against algorithms and data sources identified in the 1123 AI application papers (Fig. 4).

Fig. 4: Theory-algorithm-data mapping framework (instance-level).

Based on 1123 papers (2008–2025) yielding 2847 algorithm instances. Left: Algorithm groups (G0: Optimization; G1: Statistical/Spatial; G2: ML/DL; G3: Generative) with instance frequencies. Center: Eleven representative theories from 46-theory knowledge base. Right: Data source categories with usage distribution. Flow interpretation: Thickness represents proportion of algorithm instances, not papers. Example: CPTED's G2 flow = 36% of instances (32/89), while 72% of papers use G2 (Table S11). Multi-method papers contribute to multiple flows. See Table S11 for paper-level statistics.

Table 5 Most influential theories in urban planning and safety fields

The mapping demonstrates how urban theories provide natural computational frameworks (Fig. 4). Taking CPTED as an illustrative example^52,53, its “natural surveillance” principle translates to G1 spatial analysis for viewshed calculations, G2 machine learning for activity pattern recognition, and G0 optimization for sight-line configurations, with alignments formalized through logical consistency constraints (Supplementary Material 4, Table S5). CPTED’s “territorial reinforcement” principle further requires diverse data sources—environmental data for lighting, geographic data for layouts, image data for surveillance, and experimental data from field observations. Yet current implementations reveal systematic simplification^54,55: at the instance level, G2 methods comprise only 36% of 89 algorithm applications from 47 CPTED papers, though paper-level analysis shows 72% employ at least one G2 method (Table S11)—indicating widespread but shallow adoption. More critically, only 18% of papers integrate three or more data types, substantially below the theory-derived demand of 78.3% for multi-modal integration. Neighborhood Unit theory^56,57 reveals complementary patterns: while Perry’s framework emphasizes service accessibility requiring socio-demographic data, current applications predominantly employ G1 statistical analysis (42% of papers) with environmental data (35%), reflecting research priorities focused on environmental assessment rather than population-centered optimization.

Case a: problem-driven research—urban food waste crisis

As illustrated in Fig. 5, the multi-agent system processes research inputs through three steps: theory identification, algorithm matching, and data source selection. We evaluate recommendations using five metrics (Table 6) that assess scenario complexity, theory-practice alignment, implementation feasibility, and solution robustness. Three representative cases demonstrate the system’s application across different research motivations: Case A (problem-driven) exemplifies the complete transformation process, while Cases B (method-driven) and C (technology-driven) highlight key variations. Importantly, these cases represent scenarios where preliminary assessment suggested AI exploration was warranted. The system does not claim universal AI applicability; many urban problems—particularly those centered on trust-building, political negotiation, or community empowerment—may require non-technical interventions outside our system’s scope.

Fig. 5: Multi-agent recommendation system architecture for urban scenario analysis (implemented as publicly accessible web application at [https://llm-based-multi-agent-recommendation-system-eljs4zovvgefco8gzj.streamlit.app/]).

The system comprises five specialized agents: a Scenario Analyzer Agent transforms unstructured urban challenge descriptions into structured representations; b Theory Retriever Agent uses BERT-based semantic matching to identify relevant urban planning theories from a pre-loaded repository; c Algorithm Matcher Agent matches AI/ML algorithms to theoretical requirements from an algorithm database; d Data Source Selector Agent evaluates and selects appropriate data sources; and e Integration Validator Agent validates the complete recommendation package.

Table 6 Key metrics for theory-driven recommendations

A major US city faces the paradox of discarding 10,000 tons of edible food annually⁵⁸ while significant populations experience food insecurity. Consequently, the original approach employs continuous approximation methods to optimize vehicle routing, yet focuses solely on minimizing transportation costs. Initially, the Scenario Analyzer Agent decomposes the food waste crisis into seven structured dimensions. This decomposition reveals multiple stakeholders (donors, recovery organizations, vulnerable populations), temporal dynamics, and spatial considerations. As a result, the complexity score ξ(S) = 0.82 indicates a multi-faceted challenge requiring integrated solutions. Subsequently, the Theory Retriever Agent identifies Urban Metabolism Theory^59,60 (σ = 0.91), which conceptualizes food waste as systemic flow disruption, alongside Environmental Justice Theory⁶¹ (σ = 0.87), which reveals inequitable access patterns. Together, these complementary frameworks guide the design of equitable resource circulation.

Building on these theoretical foundations, the Algorithm Matcher Agent maps requirements to computational methods: specifically, Multi-objective Optimization with Decomposition (MOEA/D)⁶² for efficiently balancing multiple competing objectives through problem decomposition, Graph Neural Networks (GNN)⁶³ for spatial equity analysis, and Temporal LSTM⁶⁴ for demand prediction. Consequently, the integrated framework achieves a capability score cap(A, r) = 0.89. In parallel, the Data Source Selector Agent prioritizes social vulnerability indices⁶⁵ (Q = 0.92), followed by food supply data (Q = 0.85), environmental data (Q = 0.81), and transportation networks (Q = 0.78). This expansion beyond routing data therefore encompasses equity-relevant sources critical to addressing food insecurity. Finally, the Integration Validator Agent achieves robustness score $R=0.91 > {R}_{\min }=0.85$ through Monte Carlo simulations. Moreover, expert review confirms effective integration of efficiency and equity objectives, particularly highlighting the vulnerability-weighted distribution mechanism that ensures food reaches high-need populations. Table 7 summarizes this transformation from technical optimization to systemic solution.

Table 7 From technical optimization to systemic solutions in urban food waste management

Case B: method-driven research—urban heat island prediction

Researchers addressing limitations in Urban Heat Island (UHI) prediction initially focus on improving accuracy from R² < 0.8 to R² = 0.95 using new stereoscopic urban morphology metrics. Recent studies have demonstrated the effectiveness of 3D urban morphological indicators⁶⁶ and building volume information⁶⁷ combined with XGBoost models, showing superior performance in predicting land surface temperature⁶⁸ and analyzing urban heat island drivers⁶⁹. Our system identifies two relevant theoretical frameworks to expand this approach. Urban Climate Theory⁷⁰ (σ = 0.93) provides a foundational understanding of heat island formation⁷¹, while Compact City Theory⁷² (σ = 0.82) reveals that compact cities, despite their sustainability goals, face environmental quality and heat stress challenges. This theoretical grounding leads to algorithmic integration. Physics-Informed Neural Networks (PINN) embed thermodynamic constraints by incorporating physical laws into neural network training⁷³, making them suitable for heat transfer problems⁷⁴ and scientific machine learning applications⁷⁵. Spatial-GCN captures neighborhood heat interactions through graph-based spatial-temporal modeling^76,77, while SHAP⁷⁸ provides interpretability through game-theoretic explanations.

Data sources expand significantly beyond basic morphology and meteorology to include social vulnerability data, recognizing that heat exposure disproportionately affects vulnerable populations^79,80, as well as dynamic urban factors. This transformation elevates a narrow focus on accuracy into a comprehensive planning tool that generates vulnerability maps and actionable design guidelines, shifting the focus from technical performance to practical urban interventions (see Supplementary Material 6, Table S7).

Case C: technology-driven research—AI applications in disaster management

A GRU-CNN architecture for urban applications is reframed by our system, which identifies Urban Resilience Theory (σ = 0.94) as the primary framework^81,82. This theory emphasizes systemic reconfiguration and collective agency–principles that guide our multi-method integration. Based on this framework, the system recommends augmenting GRU-CNN with three complementary approaches. Agent-based modeling (ABM) enables population response simulation⁸³, Network Analysis captures infrastructure interdependencies⁸⁴, and Reinforcement Learning supports adaptive strategies⁸⁵. To support these integrated methods, data requirements expand from basic time-series and hazard maps to include infrastructure networks, social communication patterns, and historical event data. The integration of GRU-CNN architecture has shown promising results in urban environmental monitoring⁸⁶.

Beyond technical enhancements, this transformation fundamentally redefines the technology’s role in urban contexts. The system shifts from a capability demonstration to a community-centered resilience platform. More importantly, residents become active participants in resilience-building rather than passive recipients of alerts. Accordingly, system functions extend to cascading failure prediction⁸⁷, evacuation modeling, and resource optimization. Table 8 illustrates this comprehensive transformation (see Supplementary Material 7, Table S8 and Supplementary Material 6, Table S7).

Table 8 From technology showcase to community resilience platform

Discussion

Only 1.16% of papers in urban planning and safety AI applications demonstrate explicit theoretical integration by citing established urban theories (see Fig. 4). This striking gap is not merely a citation oversight but reflects three deeply interconnected systemic barriers that perpetuate the theory-practice disconnect. First, cross-domain expertise remains critically scarce. Among 948 papers with identifiable author backgrounds, merely 11.7% of first authors possess knowledge spanning both AI/computing and urban domains. Furthermore, only 33.3% of research teams include members from multiple disciplines³⁵. This expertise gap creates research environments where algorithmic capabilities dominate problem framing, often at the expense of urban theory considerations. Second, and closely related, data availability increasingly dictates research priorities rather than theoretical importance.

Traditional Urban and Geospatial Data (Groups 0–1) dominate at 68.3%, while emerging data sources remain largely untapped at 19.2%—substantially below the theory-derived demand, where all 46 theories require at least one such source. This imbalance creates what we term “data opportunism”-a phenomenon where research questions arise from measurability rather than theoretical significance⁸⁸. In other words, readily accessible datasets determine which urban problems researchers address, rather than theories driving data collection. For instance, abundant sensor data shapes traffic optimisation studies, while harder-to-measure social equity dimensions receive less attention despite their theoretical significance. Third, specialised research communities develop implicit domain expertise that remains invisible to cross-disciplinary evaluation. Domain expertise becomes highly context-dependent, acquired through sustained professional practice rather than formal articulation^89,90. Travel behaviour researchers, for example, may apply decades of accumulated knowledge about built environment-transport relationships without explicitly citing spatial interaction theories. Such sophisticated theoretical understanding remains invisible to automated detection or evaluation by researchers from other disciplines⁹¹.

These three barriers do not operate in isolation; rather, they collectively create a self-reinforcing cycle that significantly impedes interdisciplinary communication. Knowledge structures across disciplines exhibit substantial heterogeneity, leading to ambiguity and misunderstandings when computer scientists collaborate with urban planners^92,93. When domain expertise operates implicitly, the theoretical foundations underlying apparently technical work remain obscured⁹⁴. Indeed, our team composition analysis reveals a counterintuitive pattern that quantifies this challenge. Pure urban planning teams score highest on theoretical engagement (0.65), while interdisciplinary teams achieve only moderate scores (0.39). This paradox demonstrates that theoretical knowledge does not automatically transfer across disciplinary boundaries–diverse teams require intentional strategies to bridge epistemological gaps. Combined, these factors produce a self-reinforcing cycle: scarce interdisciplinary expertise, data-driven research design, and invisible theoretical communication collectively ensure that technical accessibility determines research trajectories rather than theoretical importance⁹⁵. As a result, the field consequently exhibits technical sophistication while remaining theoretically impoverished, failing to engage with complex socio-spatial realities that urban theories have long addressed and ultimately limiting both real-world impact and broader relevance⁹⁶.

Building on these structural barriers, we now examine how they manifest in researchers’ motivations and methodological choices. Two interconnected patterns explain why 67.8% of articles pursue technology-driven research while only 1.3% address concrete urban problems (Table 2). First, data accessibility creates a self-reinforcing cycle that privileges certain research directions. Traditional and Geospatial Data dominate at 68.3% (exceeding theory-derived demand of 58.7%), while Urban-generated Data accounts for only 19.2% (Fig. 1b), as readily available datasets enable shorter research cycles^97,98. In contrast, emerging sources–social media, mobile trajectories, crowdsourced platforms–require institutional partnerships, ethical approvals, and extended timelines that exceed standard project cycles^99,100. Consequently, analytical frameworks favor static administrative data while dynamic user-generated data are less frequently incorporated.

Second, the 2020–2025 period intensified this technology-first orientation through unprecedented adoption speed. During these years, 60% of all applications emerged (Fig. 3b), with algorithms like CLIP and GPT-3 achieving the same-year development and application. Notably, algorithm selection criteria reveal misaligned priorities: performance (37.4%) and novelty (26.0%) drastically outweigh domain applicability (8.2%). This misalignment leads to methodological mismatches–CNNs designed for image recognition process non-visual planning data, while NLP models analyze numerical safety metrics. Moreover, Group 3 algorithms demonstrate mean latency of merely 13.2 years compared to 48.1 years for traditional methods (Table 4), reflecting a rush to adopt cutting-edge techniques regardless of their appropriateness. These combined patterns create problematic selection effects favoring researchers skilled in rapid implementation over those focused on urban problem-solving¹⁰¹. In practice, this means researchers optimize for publishability rather than urban impact. They pursue datasets already available rather than those theoretically necessary. They adopt algorithms based on recency rather than appropriateness. This self-reinforcing cycle ultimately produces a field rich in technical demonstrations but poor in genuine urban problem-solving, where technical accessibility determines research trajectories rather than theoretical importance.

Having diagnosed the problem and its causes, we now turn to evidence that theoretical integration can fundamentally transform research outcomes. Our case studies reveal three fundamental transformations when theoretical grounding guides AI research, demonstrating practical pathways to break the technology-push cycle. First, theory expands the scope of solutions from narrow optimization to systemic intervention. Case A’s food waste routing initially minimized costs ($\min \sum {c}_{ij}{x}_{ij}$), but it ignored food insecurity–a critical urban challenge. When Urban Metabolism and Environmental Justice theories were integrated, this narrow objective was reframed into a comprehensive food security framework. Specifically, data sources expanded from routes to vulnerability indices (2 → 4 categories), algorithms evolved from single-method to integrated frameworks (MOGA + GNN + LSTM), and metrics shifted from efficiency-only to multi-dimensional measures including equity and emissions^102,103. Second, theory transforms technical gains into actionable planning tools. Case B initially pursued accuracy enhancement (R²: 0.8 → 0.95) until Urban Climate Theory introduced physical constraints via PINN. When combined with Compact City and Sustainable Design theories, outcomes evolved from temperature predictions to vulnerability-aware planning guidelines with interpretable interventions–theory added not accuracy but actionability^104,105. Third, the theory reverses the innovation direction from technology-push to need-pull. Case C’s GRU-CNN seeking applications became a community resilience platform through Urban Resilience Theory. This transformation involved architecture expansion (adding ABM, Network Analysis, RL), data diversification (2 → 4 types), and critically, residents shifting from alert recipients to resilience co-creators^106,107. These transformations share a common pattern: theoretical grounding fundamentally redefines problems rather than merely improving solutions. Across all cases, robustness scores exceeded 0.85, confirming that theoretical frameworks enhance rather than constrain performance. More importantly, theory-driven approaches generated value beyond technical metrics–Case A addressed food equity alongside efficiency, Case B produced planning prescriptions rather than predictions, and Case C enabled community participation instead of passive monitoring. This paradigmatic shift demonstrates that integrating urban theories into AI development produces outcomes simultaneously more comprehensive, actionable, and aligned with genuine urban needs than purely technical approaches^88,108.

While our study provides comprehensive evidence of the theory-practice disconnect, several limitations warrant acknowledgment. First, our 1.16% finding captures only explicit theory citations; semantic density analysis reveals that 87.1% of high-density papers demonstrate domain knowledge invisible to citation-based detection, suggesting our measurement prioritizes transparency over capturing tacit expertise. Second, our coverage has geographical boundaries—the literature search may miss non-English publications, and the 46-theory knowledge base underrepresents Global South perspectives¹⁰⁹. Third, the recommendation system operates conditionally, assuming users have determined AI approaches warrant exploration. Many urban problems involving community trust or political negotiation may be ill-suited to technical optimization; built-in safeguards (σ < 0.70, R < 0.85) can signal such cases, but recommendations indicate how AI could be applied, not that it should be. Fourth, technical constraints remain: the system’s keyword-based classification of research objectives may inadequately capture nuanced analytical goals, BERT classifier accuracy (80.85%) suggests potential misclassifications, and case studies represent conceptual demonstrations rather than real-world implementations¹¹⁰.

Despite these limitations, our research makes three significant contributions that advance both theoretical understanding and practical implementation. Empirically, we quantify barriers to theoretical integration (1.16% explicit rate, 47.3% technology-driven research) across 1123 papers, establishing measurable baselines for tracking progress^3,111. Methodologically, our multi-agent system demonstrates how LLMs can facilitate interdisciplinary knowledge integration, addressing expertise gaps affecting 88.3% of research teams^112,113. Theoretically, we extract 127 computational principles from urban theories, revealing their inherent algorithmic compatibility and challenging assumptions about incompatibility between traditional wisdom and computational methods. Looking forward, future research should pursue three interconnected directions to build on these foundations. First, expand theoretical coverage by incorporating Global South planning theories, indigenous urban knowledge, and emerging post-pandemic frameworks^8,17 while integrating diverse research databases beyond traditional academic sources. Concretely, this could involve partnering with institutions like UN-Habitat to establish Global South theory databases and conducting systematic reviews in non-English journals. Second, validate real-world impact through comparative studies of theory-grounded versus purely technical AI applications, tracking implementation success, sustainability outcomes, and community acceptance¹¹⁴. Such validation studies could follow pilot projects over multi-year periods to assess long-term adoption and adaptation patterns. Third, foster institutional change by working with funding agencies, journals, and educational institutions to develop evaluation criteria and training programs that value theory-practice integration^110,115. This might include establishing new review standards that explicitly assess theoretical grounding, creating interdisciplinary PhD programs, and incentivizing cross-sector collaborations. As cities confront unprecedented sustainability challenges, these efforts can ensure AI amplifies rather than replaces the accumulated wisdom of how cities actually work-and how they might work better for all their inhabitants.

Methods

Our methodological framework integrates diagnostic analysis with solution development. We first construct dual knowledge bases of urban theories and AI research papers, then measure theoretical integration and classify research motivations to quantify the theory-practice disconnect. We then develop a theory-algorithm-data mapping framework and a multi-agent recommendation system that bridges theoretical principles with AI implementations. Three case studies demonstrate practical applications across distinct research motivation patterns. We detail each component below.

Data on traditional urban theories

We construct a comprehensive knowledge base of urban planning and safety theories through a streamlined extraction process (Fig. 6), building on recent advances in theory formalization^35,116. Theory selection integrates three complementary criteria (see Supplementary Material 5): foundational theories with over 500 citations, emerging theories with over 50 citations annually, and expert-identified essential theories, regardless of metrics¹¹⁷-ensuring comprehensive coverage across established and culturally diverse perspectives. Guided by these criteria, our corpus includes: (1) highly-cited papers from Web of Science (WoS) and Scopus and Google Scholar using queries ("urban planning” OR “urban safety”) AND ("theory” OR “framework”), (2) works by seminal authors (Jacobs, Lynch, Newman, etc.) from 1960 to 2024, (3) 15 standard urban planning textbooks, and (4) professional guidelines from organizations like the American Planning Association (see Supplementary Table 6).

Fig. 6: Workflow for constructing the urban theory knowledge base.

a Data collection from four primary sources spanning 1960–2024. b Theory selection using composite criteria. c Automated extraction using NLP methods. d Three-expert validation with inter-rater reliability assessment. e PostgreSQL implementation with flexible JSON storage. f Final knowledge base supporting semantic similarity matching for theory-to-practice recommendations.

NLP implementation employs spaCy’s en_core_web_lg model (v3.4) for Named Entity Recognition to identify theory names through capitalized noun phrases preceded by theory-indicating terms and author-attributed concepts. Core principle extraction combines BERT embeddings (bert-base-uncased) with TF-IDF scoring, ranking sentences by combined metrics, and applying dependency parsing for grammatical completeness. A multi-label tagging system^118,119 preserves theoretical complexity by maintaining associations across spatial, social, safety, and economic dimensions with relevance weights (0–1 scale). Complete taxonomy with semantic keyword clusters for each dimension is detailed in Supplementary Material 1, Table S1. The PostgreSQL-based knowledge base with flexible JSON storage enables semantic similarity matching between urban challenges and relevant theories^120,121.

Data on urban research literature

To capture the current landscape of AI applications in urban planning and safety domains, we conduct a comprehensive literature search through the WoS and Scopus databases. Our search strategy employs two primary queries—“Urban Safety” AND “AI” and “Urban Planning” AND “AI”—without temporal restrictions to ensure comprehensive coverage of the field’s evolution. This unrestricted temporal coverage aligns with recent systematic reviews of urban AI applications^16,122,123, enabling us to trace the full trajectory of AI adoption in urban planning and safety contexts. The initial search retrieved 2797 papers, which underwent systematic screening to establish the final corpus of 1123 validated AI applications (Fig. 7).

Fig. 7: Literature Screening Process.

Literature screening workflow illustrating the progression from initial data collection through expert assessment and machine learning classification to final corpus establishment.

This keyword-based approach introduces systematic temporal bias that warrants acknowledgment. The term “AI” gained widespread use primarily after 2010, particularly following deep learning breakthroughs that catalyzed the adoption of AI terminology across disciplines^43,46. Consequently, our corpus excludes earlier algorithmic applications in urban research–including regression models, optimization techniques, spatial statistics, and early neural networks–that were not explicitly labeled as “AI”^44,45. Our analysis, therefore, captures the phenomenon of AI adoption as disciplinary framing rather than a comprehensive computational history, documenting when established methods became repackaged under AI terminology. This intentional scope limitation enables our core contribution: revealing how contemporary AI-labeled research exhibits a technology-first orientation regardless of algorithmic age.

A rigorous two-stage screening process combines human expertise with machine learning capabilities to evaluate the collected literature (Fig. 7). Domain experts first assess each paper’s relevance to genuine AI urban applications, after which a BERT-based classifier performs automated screening with NLP pipeline configurations detailed in Supplementary Material 2, Table S3. The BERT-large-uncased model was fine-tuned with learning rate 2e-5, batch size 16, training epochs 5, using AdamW optimizer with binary cross-entropy loss, achieving 80.85% accuracy (see Supplementary Material 8, Table S10) across 5-fold cross-validation (mean accuracy: 0.8085 ± 0.0467, precision: 0.7825 ± 0.0841, recall: 0.7188 ± 0.0470). Building on established methods for automated article classification in systematic reviews²⁴, our approach incorporates domain-specific adaptations^124,125. Recent advances in machine learning for systematic reviews have demonstrated that such hybrid approaches can significantly reduce workload while maintaining high accuracy^126,127.

Following the screening process, a novel dual LLM expert framework systematically analyzes the refined corpus (Fig. 8). Two independent LLMs work in parallel to extract AI algorithms and data sources from each paper, with human experts subsequently validating the outputs to ensure accuracy. Drawing inspiration from recent advances in generative information extraction¹²⁸ and LLM applications in computational social science¹²⁹, this dual-model architecture addresses single-model biases and enhances extraction reliability-critical considerations highlighted in recent LLM security and privacy literature¹³⁰. Semi-automated validation processes balance automation efficiency with accuracy requirements, creating a robust extraction pipeline that ensures replicability (see Supplementary Material 8, Table S9).

Fig. 8: Dual LLM expert analysis framework for urban safety AI research.

The three-stage process encompasses: a paper preparation and prompt engineering, b parallel extraction of algorithms and data sources by two independent LLM experts, and c human expert validation and reconciliation of results.

Algorithm and data source grouping

To organize the extracted algorithms and data sources into interpretable categories, we implement a four-stage transformation pipeline (Table 9, Fig. 1). First, we aggregate contextual information by concatenating descriptive text from all papers mentioning each item-for instance, “Random Forest” contexts from 84 papers produced approximately 500 tokens describing ensemble learning principles and urban applications. Building on this textual foundation, we employ sentence-transformers (all-mpnet-base-v2)¹³¹ to encode all contexts into 768-dimensional embeddings. Critically, BERT¹³² operates on aggregated text rather than directly on LLM JSON outputs-LLMs extract structured metadata while BERT encodes semantic meanings for clustering. To enable visualization and clustering, we then reduce embeddings to 2D coordinates via t-distributed Stochastic Neighbor Embedding²⁷ using standard parameters (perplexity = 30, learning rate = 200, iterations = 1000, random state = 42). Finally, DBSCAN¹³³ with ε = 0.5 and min_samples = 5 identify initial clusters, which three domain experts-representing urban planning, computer science, and data science perspectives-iteratively refine through four consensus rounds based on methodological coherence and domain interpretability. This hybrid process produce four algorithm groups (G0–G3) and six data source groups (G0–G5, Table 1). External validation involve two independent researchers classifying stratified samples, with detailed agreement metrics and cluster quality measures documented in Supplementary Material 9 alongside complete technical specifications.

Table 9 Seven-stage pipeline: from LLM extraction to validated grouping

Validation of theoretical integration measurement

Our explicit citation-based measurement prioritizes transparency and replicability, yet may not capture domain knowledge operating without explicit citations⁸⁹. To evaluate whether this approach underestimates implicit theoretical engagement, we implement a three-method validation protocol on a stratified subsample (n = 200). The sample oversamples papers with explicit citations (35 papers, 17.5% vs. 1.16% in the full corpus) to enable robust statistical comparison while maintaining diversity across algorithm types, publication years, and team compositions.

We apply three complementary approaches. First, LLM-based classification using GPT-4-turbo (temperature = 0.2) analyzes introduction sections to test whether automated semantic analysis can detect theoretical engagement beyond explicit citations, with accuracy assessed against expert coding using Cohen’s kappa. Second, team composition analysis quantifies disciplinary diversity using Shannon’s entropy ${H}_{{\text{norm}}}=-{\sum }_{i=1}^{k}{p}_{i}{\text{ln}}{(}{p}_{i}{)}/{\text{ln}}{(}k{)}$ and tests its correlation with theory-driven intensity scores (0–1 scale) assigned by an interdisciplinary expert panel (n = 3). Third, semantic theoretical density (STD) measures implicit engagement through proximity to an 847-term theoretical lexicon derived from our 46-theory knowledge base. Using sentence-transformers (all-mpnet-base-v2)¹³¹, we compute:

$${{\rm{STD}}}_{i}=\frac{1}{N}\mathop{\sum }\limits_{j=1}^{N}\mathop{\max }\limits_{k\in L}\left[\,{\rm{sim}}\,({s}_{ij},{t}_{k})\times {w}_{k}\right]$$

(1)

where s_ij are sentence embeddings, t_k are lexicon term embeddings, w_k are category weights (ranging from 1.0 for canonical theory constructs to 0.6 for causal mechanisms), and sim(⋅) is cosine similarity. Lexicon construction, hierarchical weighting scheme, and validation protocol are detailed in Supplementary Material S10.

Measurement convergence is assessed through: (1) pairwise correlations between explicit citation (binary), semantic density (continuous), and expert assessment (continuous); (2) Steiger’s Z-test comparing correlation strengths; and (3) cross-team consistency examining rank ordering across team types (pure CS, pure Planning, interdisciplinary). This protocol addresses the challenge of balancing transparent measurement with sensitivity to context-dependent expertise^134,135. All analyses are conducted in R 4.3.1 (α = 0.05, two-tailed), with code available in our public repository.

Research motivation classification

To uncover the driving forces behind urban AI research, we employ automated pattern matching on abstracts, titles, and introductions^136,137 to classify papers into four categories: (1) problem-driven-studies initiated by specific real-world crises (e.g., “urban flooding occurred”); (2) method-driven-research motivated by limitations of current solutions (e.g., “existing methods are insufficiently accurate”); (3) technology-driven-studies exploring novel technological applications (e.g., “deep learning can be applied to flood prediction”); and (4) general needs-driven-research addressing broad urban requirements without specific triggers. This taxonomy extends beyond traditional technology-driven versus problem-driven dichotomies¹³⁸, capturing nuanced distinctions between crisis-responsive and solution-improvement research.

Classification employs case-insensitive regex matching with contextual analysis (see Supplementary Material 3). Problem-driven patterns include crisis indicators ("occurred,” “emerged”) and temporal urgency markers; method-driven patterns capture performance critiques ("low accuracy,” “computationally expensive”) and comparative language; technology-driven patterns identify innovation phrases ("recent advances in,” “newly developed”) and technology-subject grammatical structures. Manual validation on 300 sampled papers achieved 85% inter-rater agreement (see Supplementary Material 3). For papers exhibiting multiple patterns, a semantic dominance hierarchy (Problem ≻ Method ≻ Technology ≻ General) determines classification based on priority zone analysis of titles and opening sentences. Papers without clear matches are assigned to “Unclear/Mixed” (147 papers, 13.1%) rather than forced into categories. Complete protocols appear in Supplementary Material 3. This framework aligns with recent advances in automated systematic review classification^139,140 and has been validated against manual coding using established text classification methods^141,142.

Theory-algorithm-data mapping construction

We develop a multi-method mapping framework that systematically connects urban theories, AI algorithms, and data sources by combining computational linguistics with knowledge engineering approaches^143,144. Building on recent advances in domain-specific knowledge graph construction^145,146, our framework adapts these methods for urban planning contexts through three core processes. First, NLP techniques extract computational requirements from theoretical principles^147,148; for instance, CPTED’s “maintain clear sightlines" translates to “visibility analysis" requirements. Second, algorithm capability profiles emerge from multiple sources, leveraging LLM-based knowledge extraction¹⁴⁹ with expert validation. Third, probabilistic co-occurrence models¹⁵⁰ identify data requirements by detecting algorithm-data source mention patterns within urban planning literature, thereby revealing implicit theory-implementation relationships^151,152.

The Stanford CoreNLP pipeline processes theory texts using PTBTokenizer, statistical sentence splitter, maximum entropy POS tagger, rule-based lemmatizer, SR parser, and neural dependency parser. Algorithm capability assessment evaluates seven dimensions (see Supplementary Material 4, Table S4): spatial analysis, temporal analysis, pattern recognition, prediction, classification, optimization, and real-time processing (scores 0–1). Association rule mining employs the Apriori algorithm with a minimum support of 0.05, a minimum confidence of 0.7, a maximum itemset size of 4, a lift threshold of >1.2, and a window size of 5 sentences. Pointwise Mutual Information quantifies theory-algorithm co-occurrence: PMI(theory, algorithm) = log[P(theory, algorithm)/(P(theory) × P(algorithm))]. To ensure mapping quality, a three-stage validation protocol combines statistical measures, expert annotation, and logical consistency checks^153,154, while acknowledging both technical and social assessment dimensions inherent to urban planning’s interdisciplinary nature¹⁵⁵. Final compatibility scores combine PMI (30%), expert scores (50%), and consistency checks (20%) through weighted averaging based on expert consensus¹⁵⁶, creating a robust bridge between theoretical principles and practical AI implementations.

For empirical analysis of the resulting mappings, we employ two complementary statistical approaches: instance-level distribution analyzes the proportion of algorithm instances across categories (Fig. 4), providing insights into implementation patterns, while paper-level distribution quantifies the proportion of papers employing each algorithm or data type (Supplementary Material 11, Table S11), revealing adoption patterns across the research community. This dual-level analysis enables a comprehensive understanding of both technical implementation choices and research practice trends.

Theory-driven multi-agent recommendation system

To bridge the theory-practice gap identified in our analysis, we propose a multi-agent recommendation system comprising five specialized agents that collaborate via asynchronous coordination^157,158,159. Each agent contributes domain-specific expertise through directed information flow, generating theory-grounded recommendations.

According to system scope and entry conditions, the system is designed for users who have preliminarily determined that AI-based approaches merit exploration. It does not assess whether AI is the most appropriate intervention—a determination requiring consideration of stakeholder preferences, institutional capacity, and the fundamentally social nature of many urban problems. Rather, the system transforms the question from “what can this algorithm do?” to “what theoretical principles should guide this intervention if AI is pursued?” Users encountering persistent low-similarity scores (σ < 0.70), capability gaps, or robustness failures (R < 0.85) should interpret these as signals warranting reconsideration of AI appropriateness.

Our design relies on three assumptions: (i) theoretical requirements exhibit sufficient independence for additive aggregation (∣ρ∣ < 0.35 for 94% of requirement pairs); (ii) expert knowledge can be reliably elicited (ICC > 0.80); and (iii) scenario perturbations follow bounded uniform distributions reflecting real-world planning uncertainties.

Scenario Analyzer Agent transforms unstructured challenge descriptions into structured representations across seven dimensions: domain, objectives, constraints, stakeholders, temporal scope, spatial boundaries, and data characteristics. Critically, the agent also extracts research objective type (prediction, explanation, optimization, or classification), which shapes algorithm matching independently of theoretical alignment. Scenario complexity is quantified as:

$$\xi (S)=\mathop{\sum }\limits_{i=1}^{7}{w}_{i}\cdot {c}_{i}(S)$$

(2)

where w_i are importance weights and c_i(S) ∈ [0, 1] are normalized measures. Table 10 provides specifications; complete operationalization with worked examples across three case studies appears in Supplementary Material 12, Table S12.

Table 10 Scenario complexity scoring specifications

Theory Retriever Agent employs BERT-based semantic matching (bert-base-uncased) to identify applicable theories^131,132. Theory-scenario alignment is quantified as:

$$\sigma ({T}_{i},S)=\cos ({{\bf{e}}}_{{T}_{i}},{{\bf{e}}}_{S})$$

(3)

where ${{\bf{e}}}_{{T}_{i}},{{\bf{e}}}_{S}\in {{\mathbb{R}}}^{768}$ are embeddings from the BERT [CLS] token. Theories with σ > 0.70 are relevant; σ > 0.85 indicates strong alignment. For complex challenges (ξ > 0.7), the agent identifies complementary theory combinations^160,161.

Algorithm Matcher Agent formulates selection as constrained optimization^162,163. Given candidates ${\mathcal{A}}=\{{A}_{1},\ldots ,{A}_{m}\}$ and requirements ${\mathcal{R}}=\{{r}_{1},\ldots ,{r}_{n}\}$:

$${A}^{* }=\arg \mathop{\max }\limits_{A\in {\mathcal{A}}}\left\{\mathop{\sum }\limits_{i=1}^{n}{w}_{i}\cdot {\rm{cap}}(A,{r}_{i})-\lambda \cdot {\rm{cost}}(A)\right\}$$

(4)

subject to ∑w_i = 1, w_i≥0, and cap(A, r_i)≥0.70 for critical requirements. Here, λ = 0.15 balances cost-performance trade-offs, and cost(A) ∈ [0, 1] combines computational time (60%) and memory (40%). The capability function:

$${\rm{cap}}(A,{r}_{i})=0.5\cdot {{\rm{PMI}}}_{{\rm{norm}}}(A,{r}_{i})+0.3\cdot {\rm{Expert}}(A,{r}_{i})+0.2\cdot {\rm{Consist}}(A,{r}_{i})$$

(5)

where PMI_norm is normalized pointwise mutual information, Expert scores derive from 8 specialists (ICC = 0.84), and Consist enforces logical consistency with mandatory thresholds for specific semantic tags (Supplementary Material 4, Table S5)^164,165.

Data Source Selector Agent evaluates sources using multiplicative aggregation^166,167:

$$Q({D}_{i})=\mathop{\prod }\limits_{j=1}^{6}{q}_{j}{({D}_{i})}^{{w}_{j}}$$

(6)

where weights are: relevance (w₁= 0.25), quality (w₂= 0.20), temporal coverage (w₃= 0.18), accessibility (w₄= 0.15), reliability (w₅= 0.12), and compatibility (w₆= 0.10). Critically, relevance (q₁) is computed dynamically via BERT semantic similarity between data source descriptions and scenario requirements—identical sources receive different scores across urban challenges (Supplementary Material 12, Table S13). Accessibility scores range from 1.0 (public datasets) to <0.5 (restricted sources). The system recommends data categories with documented precedent in urban research rather than hypothetical sources, with final selection requiring human validation.

Integration Validator Agent implements two-stage validation^168,169. Monte Carlo simulations (N = 1000) assess stability:

(7)

where τ = 0.70 is the performance threshold and ${R}_{\min }$= 0.85 the approval threshold. The performance function:

$${\rm{perf}}(S)=0.40\cdot {{\rm{Theory}}}_{{\rm{align}}}+0.30\cdot \min \{{{\rm{Alg}}}_{{\rm{cap}}},{{\rm{Data}}}_{{\rm{qual}}}\}+0.30\cdot {{\rm{Expert}}}_{{\rm{assess}}}$$

(8)

Each iteration applies bounded perturbations (±0.15) to 2–3 randomly selected dimensions, representing the 90th percentile of variations in 200 real-world projects. Solutions exceeding ${R}_{\min }$ advance to human validation through uncertainty sampling¹⁷⁰. Agents collaborate via structured protocols: initial analysis (Scenario → All), theory query (Theory → Algorithm), data requirements (Algorithm → Data) validation (All → Validator), and conflict resolution through iterative refinement.

BERT was fine-tuned on 1143 papers for 5 epochs (learning rate 3 × 10⁻⁵). Expert assessments used 7-point Likert scales; disagreements (ICC < 0.70) triggered facilitated discussion. Monte Carlo simulations ran in Python 3.10 with NumPy 1.24 (seed = 42). Code and data are available in our public repository.

Case study development

To demonstrate the practical application of our theory-driven recommendation system, we construct three representative case studies based on the distinct research motivation patterns identified in our corpus analysis: problem-driven, method-driven, and technology-driven. These cases are synthesized from archetypal research patterns observed across our analyzed corpus (n = 1123), capturing essential characteristics of each motivation type while maintaining generalizability². Case archetype development follows a systematic protocol to ensure representativeness: identifying recurring patterns for each motivation type, synthesizing composite scenarios incorporating common problem domains, reflecting typical algorithmic approaches, maintaining realistic constraints, and avoiding direct replication of published work. For each case, we simulate the system’s recommendation process by inputting the original research framing and generating theory-algorithm-data suggestions based on our established mapping framework. The transformations are evaluated across multiple dimensions, including theoretical grounding, solution comprehensiveness, and practical value generation¹²³. Rather than serving as empirical validations, these proof-of-concept demonstrations illustrate the system’s capability to identify relevant theoretical frameworks and transform technical solutions into theory-informed, multi-dimensional approaches that generate value beyond algorithmic performance metrics¹⁷¹. This approach aligns with recent methodological advances in design science research for urban informatics¹⁷² and theory-driven AI system evaluation (see Supplementary Material 6, Table S7).