Urban visual-spatial intelligence (UVSI)

The emergence of smart city initiatives based on embedding computers and digital communications into the contemporary city presents an unparalleled opportunity to understand and manage urban environments through a combination of Earth observation, geospatial methods, and artificial intelligence (AI)1. AI technologies are revolutionizing urban observation, sensing, imaging, and mapping2, enabling the integration of multiple data sources, including high-resolution satellite imagery, street-level imagery3, and crowd-sourced visual data4. These enabling allows the capture of multifaceted urban spaces with unprecedented levels of detail at different spatial scales and temporal resolutions. Many new sources of data through new forms of sensor technology are providing urban planners and policymakers with data that has never been available hitherto, which is enabling us to explore urban sustainability in new and innovative ways.

However, existing approaches often prioritize technological solutions over the lived experiences of urban residents, leading to a disconnect between urban monitoring and everyday life5,6. Urban Visual-Spatial Intelligence (UVSI)—defined as a human-AI collaborative framework that systematically integrates human perceptual capabilities with advanced artificial sensing systems to generate spatially explicit, actionable insights for sustainable urban development—emerges as a pivotal solution to bridge this gap5 (Fig. 1). Theoretically, UVSI is grounded in urban systems theory, geospatial data science, and human-AI collaboration, integrating principles from spatial analysis, social sensing, and human-environment interactions to enable multi-dimensional, real-time representation and predictive understanding of urban systems. Methodologically, UVSI operationalizes the fusion of heterogeneous urban data—including remote sensing, IoT sensor networks, mobile and social media streams—through spatial artificial intelligence, machine learning, and multi-scale visual analytics. It further incorporates social perception mining, semantic interpretation, and adaptive feedback loops to support continuous, dynamic urban monitoring, risk assessment, and adaptive urban management.

Fig. 1: The overall framework and core features of the Urban Visual-Spatial Intelligence (UVSI) system.
figure 1

By integrating human perceptual data with machine sensing data, the system achieves multi-dimensional and multi-layered fusion of urban spatial information. The left side represents the urban environment, while the central integration area demonstrates the collaborative collection and analysis of data by humans and AI. On the right, the results are translated into actionable insights addressing both socio-human challenges (health, economy, mobility, safety, etc.) and geo-environmental challenges (climate, disaster resilience, land use, environmental quality, etc.). The feedback loops at the top and bottom highlight the system’s adaptive and dynamic nature, enabling continuous optimization of urban management and sustainable, resilient city development through iterative interactions between historical and real-time data. Created with Flaticon.com.

Full size image

With the addition of diverse human-centered data (e.g., those derived from mobile devices, social media, and surveys) that reflect day-to-day individual experiences, this integration holds the potential to transcend and extend traditional smart city methods by recognizing that human perception and machine sensing can complement one another rather than compete. For instance, while machine sensing technologies autonomously collect objective environmental data through embedded systems (e.g., satellites, IoT sensors, drones) and provide extensive coverage of urban patterns and environmental parameters6, they gain additional value when combined with human perceptions of safety, liveliness, and overall well-being. Additionally, spatial data from points of interest, travel paths, mobile signals, and social media can deepen our understanding of urban dynamics7. Humans, acting as unique “sensors” through their devices and social media, provide a more comprehensive view of urban life8. These data sources reveal insights into human activities, behaviors, and social sustainability factors, such as environmental experiences, perceptions, and needs. Combined with more traditional socio-economic data relating to where urban activities and populations are located, this potentially provides us with new data sources that enable us to generate plans for much more sustainable cities.

The primary feedback loop within the UVSI framework in Fig. 1 ensures continuous interaction between urban environments, human experiences, and technological monitoring by integrating historical and real-time data to construct time-series models6. This mechanism enables dynamic and sustainable adjustments to urban management strategies, focusing on enhancing quality of life with respect to location, amenity, health, and mobility. The interaction between human and sensory perception in UVSI represents a dynamic relationship that enhances our understanding of urban environments. Sensors excel at capturing quantitative parameters with high precision, while humans excel at interpreting complex social and cultural contexts. The interactions between these two forms of perception are complementary. Human observations can ascribe social significance to sensor data, while sensor networks have the potential to broaden and deepen human observations of existing and emerging urban issues5,8. This dynamic interaction enables continuous refinement of urban monitoring systems and ensures that smart interventions remain grounded in human knowledge, needs, and experiences.

Urban interventions, including policy adjustments, infrastructure development, and community initiatives, address socio-economic and environmental challenges, thereby altering the urban environment. For instance, the UVSI system integrates IoT sensor networks to collect real-time environmental data (e.g., PM2.5), traffic data (e.g., vehicle flows), and infrastructure health data (e.g., bridge vibrations)5. Additionally, human perception data, capturing subjective experiences such as residents’ sense of safety, vibrancy, and well-being, are gathered through mobile applications and Volunteered Geographic Information (VGI)8. The system adheres to international privacy standards, such as the General Data Protection Regulation (GDPR), employing data anonymization, encryption, and transparent access control mechanisms to safeguard personal privacy and enhance public trust9. The effects of these interventions are continuously monitored through the UVSI system, utilizing multidimensional key performance indicators that include environmental metrics (e.g., PM2.5 concentrations, urban heat island effects), traffic efficiency metrics (e.g., congestion indices, public transit punctuality), and social well-being metrics (e.g., resident satisfaction, community engagement)10. This monitoring forms an adaptive cycle that shapes the urban environment. The closed-loop feedback approach ensures that urban development remains responsive to emerging challenges and the outcomes of prior interventions, fostering sustainable and resilient cities. The resulting UVSI (‘big’) data which is much bigger than anything we have been able to capture before offers extensive opportunities for research and development of more responsive urban sustainability initiatives that are grounded in the lived experiences of diverse populations. This Perspective introduces the use of UVSI for planning sustainable urban development and its policy relevance, identifies research gaps and emerging technologies, and outlines key research priorities for the coming decade.

Advancing UVSI research for global and regional policy relevance

Research using the UVSI methods extend into policy frameworks that align closely to the multiple United Nations Sustainable Development Goals (SDGs)11. By merging human and machine intelligence, we can better monitor urban progress towards the sustainability targets that are implicit but need to be identified and refined while ensuring that technological innovations are firmly grounded in human needs and experiences.

The transformative impacts of UVSI research are evident in its contributions to SDG Goal 11 – making cities and human settlements inclusive, safe, resilient and sustainable – on how we can ensure sustainable cities and communities remain prosperous11. UVSI research enhances urban planning and resource management by providing comprehensive knowledge and practices on energy consumption, urban heat islands, infrastructure resilience, and disaster monitoring. This will support climate action goals (e.g., the Paris Agreement which is to keep the increase in the global average temperature to well below 2 °C above pre-industrial levels and to pursue efforts to limit the temperature increase to 1.5 °C above pre-industrial levels) and promote community adaptation and mitigation strategies12. Furthermore, UVSI integrates human-centric, locational perception-based, and data-driven approaches, enabling urban policymakers to develop place-specific planning support systems (PSS) that facilitate evidence-based decision-making while acknowledging local socio-cultural dynamics. Transport plays an important role in moving towards this goal and this depends on moving away from fossil fuels.

By observing ecosystem changes generated by local peoples and sensor technologies, UVSI can significantly enhance the monitoring of urban biodiversity and green spaces, which have a significant impact on well-being and mental health, while supporting goals aligned with global initiatives such as the Kunming-Montreal Global Biodiversity Framework10. Integrating human observation with automated sensing ensures conservation and improvement efforts (e.g., increased resilience against weather extremes) are both effective and culturally appropriate.

Through interaction with local communities, UVSI research empowers participatory urban planning by providing data-driven tools to engage stakeholders and address socioeconomic disparities. It also facilitates co-designing solutions to existing issues. By uncovering inequalities among different populations and informing targeted interventions, UVSI helps create more equitable and inclusive urban environments5. This holistic approach, combining human perception with advanced technology, re-defines how cities are analyzed and managed, ultimately fostering more sustainable, resilient, and equitable urban environments worldwide.

Research gaps

There are many key research gaps in cities that can be best addressed through the UVSI framework.

One pressing gap is the lack of robust methodologies to effectively integrate qualitative human perception data with quantitative sensor data, due to differences in data format, structure, spatial-temporal scales, and biases, particularly when dealing with human perception data13. Current frameworks often treat these data streams separately, leading to a fragmented understanding of urban environments. For instance, human perception data collected through social media or surveys may be subjective and biased due to individual preferences, cultural backgrounds, and other factors. To address this, future research should focus on developing integrative models that combine these data types. This can be achieved through mixed-methods approaches, participatory mapping, and community-driven surveys. By doing so, we can create a more comprehensive understanding of urban dynamics that accounts for both subjective human experiences and objective sensor measurements14.

The development of AI in cities in terms of generative models is in its infancy, and it has not been well addressed with respect to its cultural sensitivity and contextualization. Most UVSI systems are designed with urban experiences in developed, advanced industrialized countries where English is lingua franca areas, and these applications often neglect the unique characteristics of cities in the Global South15. UVSI implementation must address disparities in technical infrastructure and expertise, particularly in these communities16. Research should focus on culturally sensitive frameworks that incorporate local knowledge and practices, examining community interactions with the urban environment. Engaging local stakeholders ensures solutions are contextually relevant and culturally appropriate. Embracing equity, diversity, and inclusion (EDI) principles is crucial, encouraging participatory approaches, empowering communities, and ensuring cultural responsiveness in urban planning17. Innovative approaches include collaborative research partnerships, capacity-building programs, and culturally sensitive technology design.

A critical methodological gap exists in developing systematic approaches to identify, quantify, and mitigate subjective biases in human perception data within UVSI frameworks. Individual biases—including cognitive, temporal, and experiential factors—can significantly skew urban perception data, while cultural biases may lead to misrepresentation of diverse community perspectives. To address these epistemological challenges, future research should establish standardized bias assessment protocols that combine multiple validation approaches: statistical methods for detecting outliers and systematic response patterns, comparative analysis across different demographic groups, and longitudinal studies to account for temporal variations in perception. Additionally, implementing ensemble methods that aggregate diverse viewpoints, employing blind validation techniques where observers assess urban conditions without prior knowledge of community demographics, and developing culturally adaptive survey instruments can enhance the scientific rigor of subjective data collection. Machine learning approaches for bias detection and correction, combined with transparent uncertainty quantification methods, will ensure that subjective experiential knowledge contributes meaningfully to evidence-based urban planning while maintaining scientific robustness.

Technological interoperability and data quality are additional significant gaps in AI research, particularly AI in cities, as it relies on big data of both a spatial and non-spatial kind. UVSI’s reliance on various technologies and data sources--from satellite imagery and ground-based sensors to crowd-sourced information--presents challenges of interoperability, consistency, and data quality. Professional surveying data offers high accuracy but at a high cost, while VGI is cost-effective but prone to bias. UVSI faces challenges in integrating diverse data quality and standardization due to varying formats, modalities, and protocols, impeding real-time synchronization and comprehensive analysis. Future research must prioritize developing open standards and protocols for seamless data sharing and integration across platforms, and systematic assessment of data quality and biases with every data release. Ensuring high data quality is crucial, addressing sensor calibration, data accuracy, and validation processes. Advanced data fusion techniques and machine learning algorithms can enhance the reliability of insights from diverse datasets, including sophisticated bias correction models that can identify and adjust for systematic biases in human perception data through pattern recognition and statistical normalization methods18. These techniques should incorporate confidence intervals and uncertainty measures for subjective data, enabling decision-makers to understand the reliability bounds of perception-based insights. Modality alignment using large language models (LLMs) is an attractive method for handling data with varying formats and protocols.

Using AI algorithms with UVSI raises privacy and ethical concerns, particularly regarding surveillance and data control9. A collaborative framework is urgently needed to protect personal data and maintain the autonomy of data providers, allowing them to actively contribute to UVSI systems. To address this, we should develop strong anonymization protocols, create transparent governance structures, and implement clear consent processes2. Advanced techniques are necessary to enhance resistance to adversarial and backdoor attacks while identifying and mitigating potential threats in urban systems2. Data process strategies should decentralize the UVSI process across various data sources, ensuring the confidentiality of sensitive information from individual urban data providers is not compromised.

Emerging trends and technologies

The global AI research landscape is rapidly evolving, with advancements in high-performance computing, real-time data processing, data compression, model optimization, and the growing role of citizen science and crowdsourcing. These trends could significantly influence the development of UVSI and enhance the effectiveness and responsiveness of urban management systems.

Advancements in edge computing and data compression are paving the way for real-time data processing. By processing data closer to their sources—such as sensors deployed throughout different city locations—edge computing reduces latency and enhances the responsiveness of urban monitoring and management systems. This capability is crucial for AI applications in areas like traffic management, emergency response, and environmental monitoring. Investigating the use of Internet of Things (IoT) devices alongside edge computing can further enhance real-time monitoring capabilities19.

Citizen science and crowdsourcing represent an influential movement in urban intelligence20, where community members actively participate in data collection and its updating. This approach democratizes urban data and enriches it with local knowledge and perspectives, helping systems to be closer to the citizens and their needs. Platforms enabling residents to report environmental conditions or urban issues can significantly enhance the data landscape’s richness and completeness. Research should explore best practices for integrating citizen-generated data, especially those collected from the Global South, into formal UVSI frameworks, ensuring that it is used responsibly and effectively and that no social group is harmed by the data collected. Maximizing participation and ensuring data quality by fostering trust and engagement within communities is essential.

Another emergent trend in AI in the urban domain is research into human mobility and new applications of motorized activities. The spatio-temporal series visual analysis based on human trajectories provides valuable insights into how individuals feel, move, and make decisions within urban environments. High spatial-temporal resolution volunteered geographic images facilitate real-time creation, revision, and updating of urban road maps while supporting traffic forecasting21. By integrating social media imagery and street view data, this approach promises to unveil new patterns of urban spatial structure and associated travel choices and frequencies, enabling effective monitoring of transportation infrastructure, traffic pattern analysis, and transportation emissions estimation. The role of vision systems in crowds and robots’ movements understanding is also becoming of great importance, preparing for sharing of the urban space between humans and autonomous systems alike22.

The granularity of urban visual spatial big data enhances our ability to examine social inequality and the uneven distribution of urban amenities across space and time. By integrating visual information from satellite and ground-based sources with citizen-science data, we can highlight disparities in infrastructure resource allocation and illustrate urban accessibility23. Advanced techniques in image processing and natural language processing applied to social media data can unveil the spatiotemporal dynamics of socioeconomic patterns and racial differences, challenging conventional perceptions of fixed boundaries associated with social spatial segregation.

Key research topics for the next decade

Contemporary urban development faces unprecedented challenges and opportunities in processing massive spatiotemporal datasets and understanding complex urban systems where rapid advancements in AI are changing the nature of the data that we are able to use to plan future cities. UVSI is a crucial framework for providing solutions. The following research priorities have emerged as key for the next five to ten years, aiming to create AI analytics improving the urban experience of all city residents while advancing sustainable urban development goals by narrowing technological gaps and addressing evolving urban demands11,24,25.

First, advancing urban visual-spatial analytics requires innovative methodologies for multi-modal and spatio-temporal data integration. A primary research priority is developing advanced, less computationally expensive, open-source vision-language models enhanced by spatial intelligence for multi-source data fusion, including remote sensing imagery, heterogeneous sensor networks, and crowdsourced information. To address potential biases in these data, the adoption of mixed-methods and community-driven surveys is crucial. These approaches can help contextualize data within the cultural and social frameworks of urban residents, ensuring a more balanced integration of subjective human experiences with objective sensor measurements. By enabling more accurate and culturally attuned monitoring of urban dynamics, these innovations are essential for overcoming computational challenges in real-time urban monitoring and analysis, directly supporting the creation of sustainable urban development strategies that reflect the true needs and conditions of urban environments.

Equally critical is the development of urban digital twins and modeling technologies26, where a wide array of tools and techniques associated with UVSI enables the creation of novel models built upon advanced digital methods to support sustainable urban development. The theoretical framework for urban digital twins requires substantial development in spatiotemporal modeling and simulation, with urban visual-spatial analytics as a key foundation. Key research areas include1: developing mathematical models of urban system structures and dynamics to optimize resource allocation and enhance urban resilience2; advancing multidimensional visualization techniques to engage citizens and private-sector stakeholders, thereby improving understanding of environmental and social impacts; and3 integrating cross-scale urban processes to ensure urban planning addresses challenges such as climate change and population growth. These developments will facilitate evidence-based urban resilience assessments and adaptation strategies, strengthening sustainable urban management by reducing energy consumption, improving infrastructure resilience, and promoting social inclusivity.

Methodological innovation in visual analytics and spatial decision support systems is another crucial research direction. This includes developing theoretical frameworks for collaborative decision-making, advancing human-computer interaction paradigms, and formulating novel approaches to network-based urban analysis. To address potential biases in decision-making processes, the integration of participatory methods is essential. These innovations are essential for addressing challenges in cross-regional data standardization and system integration, which directly impact the sustainability of urban planning. Additionally, current PSS lack mechanisms to integrate UVSI’s analytical capabilities for informing decision-making by urban planners and policymakers27 and it is essential to evolve and develop new tools for participation. Further research should focus on integrating UVSI into PSS to enhance their capabilities and effectiveness, enabling PSS to capture and process complex urban data while incorporating the lived experiences and aspirations of local communities, thereby supporting environmental and social sustainability.

Finally, the evolution of AI-driven UVSI technology necessitates rigorous investigation into ethical frameworks, privacy, data confidentiality, and inclusive methodologies to support sustainable urban development, which aims to create inclusive, safe, resilient, and sustainable cities and human settlements11. Data privacy risks, such as unauthorized access or data breaches, must be mitigated through robust encryption, anonymization, and transparent governance structures, adhering to international standards like the GDPR to enhance public trust and ensure the sustainability of urban management9. Technical dependence may introduce system vulnerabilities, such as AI algorithmic biases or cyberattacks, which threaten the long-term resilience of urban systems; these can be addressed through diversified technological designs, redundant systems, and continuous validation to ensure UVSI supports resource-efficient and environmentally friendly urban planning. Furthermore, social acceptance hinges on establishing trust through transparent communication and community engagement, while inclusive design, by integrating diverse population perspectives, addresses social inequalities and fosters sustainable, equitable urban environments. Future research must focus on developing theoretical foundations for equitable spatial intelligence, establishing robust protocols for democratic data access, and advancing methodologies for participatory spatial analysis. These developments are essential for realizing the vision of Artificial Intelligence for All Residents in All Cities while maintaining scientific rigor and social responsibility.