Abstract
With the deepening of urbanization, the spatiotemporal heterogeneity of theft crimes in New York City has become prominent, creating a demand for more accurate prediction. Existing models face limitations in capturing nonlinear correlations, integrating multi-source data, and generalizing to dynamic scenarios. This study proposes an LLM-enhanced Spatiotemporal Transformer (LLM-STT) model, which integrates multi-source spatiotemporal features (including taxi passenger flow proxy) and Gemma3-12B embeddings, with a lightweight fine-tuning scheme for Gemma3-1B. Its main explorations include LLM-based semantic encoding, quantifying feature coupling, and balancing performance and deployment feasibility. Experiments on hourly neighborhood-scale theft prediction in New York City show the model achieves an AUC of 0.91 and an F1 score of 0.83, demonstrating competitive performance against baselines. LLM embeddings and dynamic population features contribute positively, and the lightweight fine-tuned model outperforms the random baseline. These findings offer preliminary support for targeted crime prevention in similar urban contexts, with broader generalization requiring further validation.
Similar content being viewed by others
Data availability
The data that support the findings of this study are publicly accessible or derived from publicly available sources, in compliance with Springer Nature’s research data policy. Details on access paths, processing procedures, and citations are provided below: 1. Theft Crime Data Source: New York City Police Department (NYPD) Open Database, “NYPD Complaint Data Historic”. Access Link: https://www.kaggle.com/datasets/leilahmiller/2006-2023-nypd-complaint-data-historic Usage Scope: We filtered theft case records from November 2013–December 2015 and January 2020–December 2020, excluding duplicate reports, samples with abnormal latitude/longitude (outside the study area: 40.496\(^{\circ }\)N–40.915\(^{\circ }\)N, 73.7\(^{\circ }\)W–74.25\(^{\circ }\)W), and invalid timestamps. A total of 619,059 valid records were retained as binary prediction labels (1 = theft occurred, 0 = no theft occurred). Citation: Miller, L. (2023). 2006-2023 NYPD Complaint Data Historic [Dataset]. Kaggle. https://www.kaggle.com/datasets/leilahmiller/2006-2023-nypd-complaint-data-historic. 2. Taxi Passenger Flow Proxy Data Source: New York City Taxi and Limousine Commission (TLC) “TLC Trip Record Data”. Access Link: https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page Usage Scope: Hourly grid-level passenger flow indicators (inflow: number of destination passengers, outflow: number of origin passengers, net flow: inflow – outflow) were extracted from trajectory data covering January 2013–December 2015 and January 2020–December 2020. GPS coordinates of taxi pick-up/drop-off points were matched to the 14.3-hectare grid system (106\(\times\)139) used in this study. Missing values were imputed using the historical average of the same grid and time period. Citation: New York City Taxi and Limousine Commission (TLC). (2023). TLC Trip Record Data [Dataset]. TLC Official Website. https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page. 3. Multi-Source Spatiotemporal Data 3.1 Nighttime Light Data Source: Colorado School of Mines Earth Observing Group (EOG), “Visible Night Lights (VNL) – VIIRS/NPP Nighttime Light Data”. Access Link: https://eogdata.mines.edu/products/vnl/ Usage Scope: 500-meter resolution nighttime light intensity data (January 2013–December 2015, January 2020–December 2020) were resampled to the 14.3-hectare grid scale to represent regional activity intensity. Citation: Colorado School of Mines Earth Observing Group (EOG). (2023). Visible Night Lights (VNL) – VIIRS/NPP Nighttime Light Data [Dataset]. EOG Mines. https://eogdata.mines.edu/products/vnl/. 3.2 POI Data Source: OpenStreetMap Foundation, “OpenStreetMap Geospatial Data”. Access Link: https://osmfoundation.org/wiki/Main_Page Usage Scope: 12 categories of POIs (e.g., commercial, residential, public transportation) were extracted to calculate functional density and road network accessibility. Citation: OpenStreetMap Foundation. (2023). OpenStreetMap Geospatial Data [Dataset]. OpenStreetMap Official Website. https://osmfoundation.org/wiki/Main_Page. 3.3 Meteorological Data Source: U.S. National Weather Service, U.S. National Centers for Environmental Information (NCEI) (National Oceanic and Atmospheric Administration, NOAA), “Daily Summaries (Global Historical Climatology Network - Daily, GHCN-D)”. Access Link: https://www.ncei.noaa.gov/cdo-web/ Usage Scope: Weather type data (2013–2015, 2020) were spatially aligned to New York City’s administrative districts and matched to hourly time windows for constructing “weather-space interaction features”. Citation: U.S. National Weather Service, U.S. National Centers for Environmental Information (NCEI) (NOAA). (2023). Daily Summaries (GHCN-D) [Dataset]. NCEI Official Website. https://www.ncei.noaa.gov/cdo-web/. 4. Control Data (Socio-Economic Data) Source: U.S. Census Bureau (population density data); U.S. Bureau of Economic Analysis (BEA), “Gross Domestic Product (GDP) by County”. Access Link: https://apps.bea.gov/regional/ Usage Scope: Socio-economic indicators (GDP, population density) for January 2020–December 2020 were used to control for regional demographic and economic heterogeneity. Citation: U.S. Bureau of Economic Analysis (BEA). (2023). Gross Domestic Product (GDP) by County [Dataset]. U.S. Bureau of Economic Analysis. https://apps.bea.gov/regional/. 5. Code Availability The code used for model training (LLM-STT model implementation, spatiotemporal attention module, LLM lightweight fine-tuning) and data analysis (feature engineering, experimental result validation) is available from the corresponding author (Junjie Wang) upon reasonable request. All data used in this study have undergone standardization (Min-Max normalization, Z-score standardization), data cleaning (outlier removal, missing value imputation), and spatiotemporal alignment to ensure consistency with the model’s input requirements. No restricted or proprietary data were used in this research.
References
Brantingham, P. L. & Brantingham, P. J. Notes on the geometry of crime. In Principles of Geographical Offender Profiling 97–124 (Routledge, 2017)
Sherman, L. W., Gartin, P. R. & Buerger, M. E. Hot spots of predatory crime: Routine activities and the criminology of place. Criminology 27(1), 27–56 (1989).
Mansha, S., Rehman, A., Abdullah, S., Kamiran, F. & Yin, H. Locality aware temporal fms for crime prediction. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management 4324–4328 (2022)
Fu, H. et al. Augmented graph information bottleneck with type-aware periodicity heterogeneity for explainable crime prediction. Inf. Process. Manag. 62(6), 104227 (2025).
Shan, M., Ye, C., Chen, P. & Peng, S. Ada-gcnlstm: An adaptive urban crime spatiotemporal prediction model. J. Saf. Sci. Resil. 6(2), 226–236 (2025).
Jing, C. et al. A deep multi-scale neural networks for crime hotspot mapping prediction. Comput. Environ. Urban Syst. 109, 102089 (2024).
Guo, Y. Multimodal spatio-temporal fusion: A generalizable gcn-lstm with attention framework for urban application. Inf. Fus. 131, 104164 (2026).
Liang, W., Wang, Y., Tao, H. & Cao, J. Towards hour-level crime prediction: A neural attentive framework with spatial-temporal-categorical fusion. Neurocomputing 486, 286–297 (2022).
Butt, U. M., Letchmunan, S., Ali, M. & Sherazi, H. H. R. Start: A spatiotemporal autoregressive transformer for enhancing crime prediction accuracy. IEEE Trans. Comput. Soc. Syst. 12, 4650–4664 (2025).
Qin, Z. et al. Acsaformer: A crime forecasting model based on sparse attention and adaptive graph convolution. Front. Phys. 13, 1596987 (2025).
Lu, H., Chen, C., Ma, Y. & Ma, Y. Lightweight deep learning model for crime pattern recognition based on transformer with simulated annealing sparsity and cnn. Sci. Rep. 15(1), 20311 (2025).
Mechqrane, Y. & Elabbassi, I. From prediction to action: A constraint-based approach to predictive policing. In 31st International Conference on Principles and Practice of Constraint Programming (CP 2025) 29–1 (Schloss Dagstuhl–Leibniz-Zentrum für Informatik, 2025).
Shahmoradi, N., Alesheikh, A. A., Jafari, A. & Lotfata, A. Hybrid st-resnet and lstm approach for precise crime hotspot prediction. Sci. Rep. 15(1), 40754 (2025).
Reza, M. A., Bisaria, A., Advaitha, S., Ponnekanti, A. & Arya, A. Crix: Intersection of crime, demographics and explainable AI. In ICAART (2) 714–725 (2025)
Zhang, X., Liu, L., Xiao, L. & Ji, J. Comparison of machine learning algorithms for predicting crime hotspots. IEEE Access 8, 181302–181310 (2020).
Xia, L. et al. Spatial-temporal sequential hypergraph network for crime prediction with dynamic multiplex relation learning. arXiv preprint arXiv:2201.02435 (2022)
Li, Z., Huang, C., Xia, L., Xu, Y. & Pei, J. Spatial-temporal hypergraph self-supervised learning for crime prediction. In 2022 IEEE 38th International Conference on Data Engineering (ICDE) 2984–2996 (IEEE, 2022).
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S. & Huq, A. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd Acm Sigkdd International Conference on Knowledge Discovery and Data Mining 797–806 (2017)
Feng, J., Wang, S., Liu, T., Xi, Y. & Li, Y. Urbanllava: A multi-modal large language model for urban intelligence with spatial reasoning and understanding. arXiv preprint arXiv:2506.23219 (2025)
Liu, C. et al. St-llm+: Graph enhanced spatio-temporal large language models for traffic prediction. IEEE Trans. Knowl. Data Eng. 37, 4846–4859 (2025).
Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30, 6000–6010 (2017).
Deng, J., Chen, X., Jiang, R., Song, X. & Tsang, I. W. St-norm: Spatial and temporal normalization for multi-variate time series forecasting. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining 269–278 (2021)
Linning, S. J., Andresen, M. A. & Brantingham, P. J. Crime seasonality: Examining the temporal fluctuations of property crime in cities with varying climates. Int. J. Offender Ther. Comp. Criminol. 61(16), 1866–1891 (2017).
Reinhart, A. Point process modeling with spatiotemporal covariates for predicting crime. PhD thesis, Carnegie Mellon University (2016)
Benda, B. B. Survival analysis of criminal recidivism of boot camp graduates using elements from general and developmental explanatory models. Int. J. Offender Ther. Comp. Criminol. 47(1), 89–110 (2003).
Cozens, P. M., Saville, G. & Hillier, D. Crime prevention through environmental design (cpted): A review and modern bibliography. Prop. Manag. 23(5), 328–356 (2005).
Anderson, J. M., MacDonald, J. M., Bluthenthal, R. & Ashwood, J. S. Reducing crime by shaping the built environment with zoning: An empirical study of los angeles. University of Pennsylvania Law Review 699–756 (2013)
Bonilla Alguera, G. & Gutierrez Meave, R. Zoning out robbery? an empirical study in Mexico city. Hous. Policy Debate 32(4–5), 730–749 (2022).
Mohler, G. O., Short, M. B. & Brantingham, P. J. The concentration-dynamics tradeoff in crime hot spotting. In Unraveling the Crime-Place Connection 19–39 (Routledge, 2017).
Reinhart, A. & Greenhouse, J. Self-exciting point processes with spatial covariates: Modelling the dynamics of crime. J. R. Stat. Soc.: Ser. C: Appl. Stat. 67(5), 1305–1329 (2018).
Cohen, J. & Tita, G. Diffusion in homicide: Exploring a general method for detecting spatial diffusion processes. J. Quant. Criminol. 15(4), 451–493 (1999).
Sommer, A. J., Lee, M. & Bind, M.-A.C. Comparing apples to apples: An environmental criminology analysis of the effects of heat and rain on violent crimes in boston. Palgrave Commun. 4, 138 (2018).
Schinasi, L. H. & Hamra, G. B. A time series analysis of associations between daily temperature and crime events in Philadelphia, Pennsylvania. J. Urban Health 94(6), 892–900 (2017).
Anderson, C. A. & Anderson, K. B. Temperature and aggression: Paradox, controversy, and a (1998)
Erturk, E., Raynham, P. & Teji, J. U. Exploring the effects of light and dark on crime in London. ISPRS Int. J. Geo Inf. 13(6), 182 (2024).
Fotios, S., Robbins, C. & Farrall, S. Research note: Variation of the effect of ambient light level on crime frequency with type of crime and location. Light. Res. Technol. 56(3), 295–303 (2024).
Welsh, B. C., Farrington, D. P. & Douglas, S. The impact and policy relevance of street lighting for crime prevention: A systematic review based on a half-century of evaluation research. Criminol. Public Policy 21(3), 739–765 (2022).
Song, G. et al. Testing indicators of risk populations for theft from the person across space and time: The significance of mobility and outdoor activity. Ann. Am. Assoc. Geogr. 108(5), 1370–1388 (2018).
Puttock, S., Barros, U., Pinheiro, D. & Oliveira, M. Larger cities, more commuters, more crime? the role of inter-city commuting in the scaling of urban crime. arXiv preprint arXiv:2505.20822 (2025)
He, L. et al. Ambient population and larceny-theft: A spatial analysis using mobile phone data. ISPRS Int. J. Geo Inf. 9(6), 342 (2020).
Braakmann, N. Residential turnover and crime–evidence from administrative data for England and Wales. Br. J. Criminol. 63(6), 1460–1481 (2023).
Kadar, C. & Pletikosa, I. Mining large-scale human mobility data for long-term crime prediction. EPJ Data Sci. 7(1), 1–27 (2018).
Li, B., Shi, P. & Ward, A. R. Latent feature mining with large language models (2024)
Sarzaeim, P., Mahmoud, Q. H. & Azim, A. A framework for llm-assisted smart policing system. IEEE Access 12, 74915–74929 (2024).
Fatima, S. K., Zubair, T., Ahmed, N. & Khan, A. Autogen driven multi agent framework for iterative crime data analysis and prediction. arXiv preprint arXiv:2506.11475 (2025)
Kim, H. et al. Lapis: Language model-augmented police investigation system. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management 4637–4644 (2024)
Sarzaeim, P., Mahmoud, Q. H. & Azim, A. Experimental analysis of large language models in crime classification and prediction. In Canadian AI (2024)
Echterhoff, J., Liu, Y., Alessa, A., McAuley, J. & He, Z. Cognitive bias in decision-making with llms. arXiv preprint arXiv:2403.00811 (2024)
Mirza, V., Kulkarni, R. & Jadhav, A. Evaluating gender, racial, and age biases in large language models: A comparative analysis of occupational and crime scenarios. In 2025 IEEE Conference on Artificial Intelligence (CAI) 244–251 (IEEE, 2025).
Hu, E. J. et al. Lora: Low-rank adaptation of large language models. ICLR 1(2), 3 (2022).
Wang, E., Liu, W., Liu, W., Xiang, C., Yang, B. & Yang, Y. Spatiotemporal transformer for data inference and long prediction in sparse mobile crowdsensing. In IEEE INFOCOM 2023-IEEE Conference on Computer Communications 1–10 (IEEE, 2023).
Boyle, D. & Kalita, J. Spatiotemporal transformer for stock movement prediction. arXiv preprint arXiv:2305.03835 (2023)
Xu, M. et al. Spatial-temporal transformer networks for traffic flow forecasting. arXiv preprint arXiv:2001.02908 (2020)
Lv, X., Jing, C., Wang, Y. & Jin, S. A deep neural network for spatiotemporal prediction of theft crimes. Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci. 48, 35–41 (2022).
Chen, Y., Lin, F., Huo, J. & Yan, H. Designing specialized two-dimensional graph spectral filters for spatial-temporal graph modeling. In Proceedings of the AAAI Conference on Artificial Intelligence Vol. 39 11500–11508 (2025)
Arik, S.Ö. & Pfister, T. Tabnet: Attentive interpretable tabular learning. In Proceedings of the AAAI Conference on Artificial Intelligence Vol. 35, 6679–6687 (2021)
Nguyen-Sy, T. Optimized hybrid xgboost-catboost model for enhanced prediction of concrete strength and reliability analysis using monte carlo simulations. Appl. Soft Comput. 167, 112490 (2024).
Li, Z. et al. Urbangpt: Spatio-temporal large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining 5351–5362 (2024)
Zhang, X. et al. Interpretable machine learning models for crime prediction. Comput. Environ. Urban Syst. 94, 101789 (2022).
An, B. et al. Geopro-net: Learning interpretable spatiotemporal prediction models through statistically-guided geo-prototyping. In Proceedings of the AAAI Conference on Artificial Intelligence Vol. 39 11427–11435 (2025)
Lee, S., Ki, D., Hipp, J. R. & Kim, J. H. Analysing non-linearities and threshold effects between street-level built environments and local crime patterns: An interpretable machine learning approach. Urban Stud. 62(6), 1186–1208 (2025).
Albors Zumel, A., Tizzoni, M. & Campedelli, G. M. Deep learning for crime forecasting: The role of mobility at fine-grained spatiotemporal scales. J. Quant. Criminol. https://doi.org/10.1007/s10940-025-09629-3 (2025).
Zhao, X. & Tang, J. Exploring transfer learning for crime prediction. In 2017 IEEE International Conference on Data Mining Workshops (ICDMW) 1158–1159 (IEEE, 2017).
Downey, A., Islam, S. R. & Sarker, M. K. Predictive policing: A fairness-aware approach. Int. J. Artif. Intell. Tools 33(03), 2460005 (2024).
Funding
This work was supported by the Special Project for the Transformation of Scientific and Technological Achievements of Qinghai Province (Project No.: 2025-GX-143). The funder had no role in the design of the study, collection, analysis, or interpretation of data; drafting of the manuscript; or the decision to submit the manuscript for publication.
Author information
Authors and Affiliations
Contributions
T.M. (Tang Minghu) supervised the entire research process (including data processing, model development, and experimental design), collected and preprocessed multi-source data (theft crime data, taxi passenger flow data), implemented the core LLM-STT model code, and drafted the initial version of the manuscript.W.J. (Wang Junjie, Corresponding Author) conceived and designed the overall research framework, revised the manuscript critically for important intellectual content, coordinated the submission process, and communicated with the journal editorial office.B.X. (Bu Xuan) designed the spatiotemporal attention interaction module and ablation experiments, analyzed experimental results (including AUC/F1 score verification), and optimized model hyperparameters.Z.J. (Zhang Jiayi) completed feature engineering (temporal, geospatial, venue functional features), verified the model’s cross-temporal generalization ability (2020 dataset extrapolation test), and organized supplementary data figures.L.P. (Luo Peng) reviewed relevant literature on crime prediction and LLM applications, optimized the Gemma3-1B lightweight fine-tuning scheme (4-bit quantization + LoRA), and assisted in revising the “Data Availability Statement”. All authors read, revised, and approved the final manuscript.
Corresponding author
Ethics declarations
Ethical approval and consent to participate
Not applicable. All data used in this study are publicly available secondary datasets (e.g., Kaggle-hosted NYPD crime data, New York City TLC taxi trajectory data, Colorado School of Mines EOG nighttime light data) that do not involve human subjects, personal identifiable information, or experimental procedures requiring ethical oversight. No ethical approval or participant consent was required for the use of these public datasets.
Consent for Publication
Not applicable. This study does not include any personal data, individual-level information, images, or content that requires explicit consent for publication.
Competing Interests
All authors (Tang Minghu, Wang Junjie, Bu Xuan, Zhang Jiayi, Luo Peng) declare no known competing financial interests or personal relationships that could have appeared to influence the work reported in this manuscript.
Code Availability
Not applicable. The code used for model training (LLM-STT model implementation, spatiotemporal attention module, LLM lightweight fine-tuning) and data analysis (feature engineering, experimental result validation) is available from the corresponding author (Wang Junjie) upon reasonable request.
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Tang, M., Wang, J., Bu, X. et al. Urban theft prediction via LLM-empowered spatiotemporal transformer. Sci Rep (2026). https://doi.org/10.1038/s41598-026-45681-0
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-026-45681-0
