Evaluation

The methodology and findings were evaluated through several complementary checks: 1. NLP classification quality: the two-stage zero-shot NLI pipeline for crime-related and Broken Windows- Theory disorder posts was validated against 2,000 manually annotated posts using Cohen’s κ inter-rater agreement (κ = 0.90 at the gate stage; κ = 0.85 at the category-label stage). Sensitivity analyses were reported across alternative threshold configurations. 2. Statistical robustness: the primary Conditional Poisson within-LSOA fixed-effects regression was complemented by Negative Binomial models as overdispersion checks, run on both active and full LSOA samples to assess sensitivity to data sparsity. 3. Out-of-sample forecasting: six machine-learning models (Elastic Net, Histogram-based Gradient Boosting, Random Forest, XGBoost, LightGBM, and a stacking ensemble) were evaluated on temporally held-out test data (no random shuffling); hyperparameters were tuned via Optuna with Time Series Split cross-validation. Performance was reported using out-of-sample R² and RMSE across 1–6- month forecast horizons. 4. Interpretability: SHAP / TreeSHAP attribution validated that predictive features align with theoretical expectations (Broken Windows disorder, sentiment, crime mentions) and demonstrated stability of the dominant predictors across crime types and forecast horizons. 5. Ethics review: the project received Institutional Review Board approval at the host institution and additional REPHRAIN ethics review, ensuring alignment with proportionality, transparency, and harm-minimisation principles for online research.