Learning to route in time and frequency domains: a dual-domain MoE transformer for multi-horizon forecasting

Abstract

Accurate long-term electrical load forecasting is required for reliable smart grid operation, yet it remains difficult due to multi-scale periodic patterns and non-stationary temporal variations across different prediction horizons. This paper presents MoE-Transformer, a dual-domain forecasting framework that learns to route representations in both the time and frequency domains through reinforcement learning. To mitigate spectral misalignment in multi-step forecasting, we introduce an Extended Discrete Fourier Transform (Extended DFT) that aligns the input spectrum with the frequency grid of the full prediction window. The proposed model incorporates parallel Mixture-of-Experts modules in the time and frequency domains (T-MoE and F-MoE), where domain-specific experts capture complementary temporal dynamics and spectral structures. Expert routing in each domain is modeled as an independent Markov Decision Process and optimized using reinforcement learning to jointly consider forecasting accuracy, routing consistency, and balanced expert utilization. Experiments on five benchmark datasets, including ETTh1, Electricity, and Traffic, across four forecasting horizons show that MoE-Transformer achieves MSE reductions of 50.9–56.9% relative to state-of-the-art baselines under matched training protocols. Relative to a same-capacity dense Transformer baseline on NVIDIA RTX 4090, sparse top-1 expert activation reduces peak GPU memory by $39.6 \pm 1.1\%$ and single-sample inference latency by $60.1 \pm 1.2\%$ (mean ± std over 5 runs), with measured absolute batched latency of $1.20 \pm 0.03$ ms per sample, supporting real-time forecasting deployment. Ablation results confirm the individual effects of Extended DFT, dual-domain modeling, and reinforcement-based routing, yielding performance gains of 5.8%, 4.6%, and up to 47.2%, respectively.