Radar-vision multimodal fusion for dynamic target trajectory prediction and threat assessment in power transmission corridors

Abstract

Power transmission corridors face growing intrusion risks from unmanned aerial vehicles (UAVs), construction machinery, and wind-borne debris. Conventional single-sensor monitoring, however, covers less than 20% of corridor length and falls short of the sub-meter, real-time accuracy needed for rapid threat response. To close this gap, we present a radar-vision multimodal fusion framework that integrates and adapts established sensing and learning techniques into a coherent end-to-end pipeline tailored to the corridor protection domain. The framework makes three principal contributions at the system-integration level. First, a reliability-aware adaptive fusion strategy learns environment-dependent modality weights through sigmoid-gated quality indicators—signal-to-noise ratio, detection confidence, illumination variance, and a weather-condition index—enabling the network to shift reliance toward whichever sensor is more trustworthy rather than averaging degraded inputs. Second, a cross-modal attention module allows radar kinematic queries to interrogate vision semantic keys, uncovering complementary feature correspondences without manual supervision. Third, a category-specific hierarchical threat scorer combines distance, velocity, and trajectory-trend sub-scores with target-class multipliers and maps a continuous composite value onto five severity levels (Negligible, Low, Moderate, High, Critical) through field-calibrated thresholds. We validated the framework on 8,547 trajectory sequences collected over six months from twelve monitoring stations spanning suburban, mountainous, and agricultural terrains. Mean Absolute Error (MAE) of predicted positions reached 1.62 m at a 3-second horizon and 3.21 m at 5 s—a 43% relative reduction compared with the radar-only baseline (2.84 m at 3 s). Threat-level classification achieved 89.6% precision, 87.3% recall, and an 88.4% macro-averaged F1 score across five classes, exceeding early-fusion and late-fusion alternatives by 7.4 and 9.8% points respectively. Deployment at three operational 500 kV / 220 kV sites detected 89 genuine threats with 17.8 s mean warning lead time, 3.2% false-alarm rate, and 99.4% system uptime. Operator workload—measured as person-hours of continuous manual surveillance—fell by 67%, while instrumented detection coverage expanded from 18% to 94% of corridor length.