Factorization Machine-Enhanced Crossformer for Multivariate Time Series Event Detection in Epidemic

We propose FM-Crossformer, a novel architecture that replaces the traditional attention mechanism in Crossformer with factorization machine (FM) functions for epidemic event detection in multivariate time series. Traditional attention-based models face computational limitations and struggle to capture complex feature interactions in high-dimensional epidemic data. Our approach leverages the strengths of factorization machines to model pairwise feature interactions while maintaining the cross-dimensional and cross-temporal design of Cross- former. Let X ∈RN ×L represent a multivariate time series with N variables (e.g., infection rates, mobility indices, vaccination rates) and sequence length L. The traditional Crossformer attention mechanism Attention(Q,K,V) = softmax(QKT /√dk )V is replaced with a factorization machine function: FM(x) = w0 + n wixi + i=1 n i=1 n j=i+1 ⟨vi,vj ⟩xixj (1) where vi ∈Rk are latent factor vectors, and ⟨·,·⟩denotes the dot product. For the cross-dimension stage, we adapt the complete FM formulation: Hcross-dim = Wdim 0 + N Wdim i Xi,seg + i=1 N i=1 N j=i+1 where Xseg ∈RN ×s represents dimension-wise segments and Hcross-dim ∈Rs×d. For the cross-time stage: ⟨v dim dim i ,v j ⟩Xi,seg ⊙Xj,seg (2) Hcross-time = Wtime 0 + s t=1 Wtime t Hcross-dim t + s t=1 s t′ =t+1 ⟨v time time t ,v t′ ⟩Hcross-dim t ⊙Hcross-dim t′ (3) The event detection probability is computed as: P(eventt) = σ(WoutHcross-time t + bout) (4) where σ is the sigmoid function, and Wout and bout are learnable parameters. Traditional epidemic modeling relies on compartmental models (SIR, SEIR) and statistical approaches. Recent deep learning advances include LSTM-based models for COVID-19 forecasting (Chimmula & Zhang, 2020), graph neural networks for spatial-temporal epidemic prediction (Kapoor et al., 2020), and transformer architectures adapted for pandemic data analysis. Crossformer (Zhang & Yan, 2023) introduced cross-dimensional attention for multivariate time series, while factorization machines (Rendle, 2010) have proven effective for sparse feature interaction modeling in recommendation systems. Our FM-Crossformer architecture maintains the two-stage design of Crossformer while replacing attention computation with complete factorization machine functions. This approach reduces computational complexity from O(N2L2d) to O(N2sk+ s2k) where k is the latent factor dimension and sis the segment length. When s≪L, this provides computational savings while enabling the FM functions to naturally capture second-order feature interactions critical for epidemic dynamics, such as mobility and infection rate correlations as well as vaccination and transmission dependencies. The main contributions are: (1) Novel integration of factorization machines with transformer architecture for time series analysis; (2) Computationally efficient approach for multivariate epidemic event detection; (3) Enhanced modeling of feature interactions in high- dimensional epidemic data; (4) Theoretical framework demonstrating reduced computational overhead while maintaining interpretability through explicit feature interaction modeling. The proposed FM-Crossformer provides a computationally efficient alternative to attention-based models for detecting epidemic events such as outbreak of diseases, peak occurrences, and intervention effectiveness, while maintaining interpretability through explicit feature interaction modeling. Future work will validate performance across real-world epidemic datasets. References 1. Chimmula, V. K. R., & Zhang, L. (2020). Time series forecasting of COVID-19 transmission in Canada using LSTM networks. Chaos, Solitons & Fractals, 135, 109864. 2. Kapoor, A., Ben, X., Liu, L., Perozzi, B., Barnes, M., Blais, M., & O’Banion, S. (2020). Examining COVID-19 forecasting using spatio-temporal graph neural networks. arXiv preprint arXiv:2007.03113. 3. Rendle, S. (2010). Factorization machines. In 2010 IEEE International Conference on Data Mining (pp. 995-1000). IEEE. 4. Zhang, Y., & Yan, J. (2023). Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In International Conference on Learning Representations.