Applying Temporal Fusion Transformers for Air Quality Forecasting in Metro Manila

Dane Casey Casiño, Jezzel Faith Gier, Monique Mendoza, Aubrey Rose Quiacao, and Christine Joy Sorronda

DS413- Elective 4 (Deep Learning)

Introduction

Air quality forecasting plays a crucial role in mitigating health risks and guiding urban planning. Traditional models such as ARIMA or LSTM often face challenges in handling multivariate, irregular, and long-horizon time-series data. In rapidly urbanizing areas like Metro Manila, where pollution levels fluctuate due to traffic, weather, and seasonal effects, an advanced deep learning approach is necessary. The Temporal Fusion Transformer (TFT), proposed by Lim et al. [1], integrates recurrent layers, attention mechanisms, and gating to deliver accurate and interpretable multi-horizon forecasts.

Moreover, air pollution remains a critical global health issue, contributing significantly to respiratory and cardiovascular illnesses. Accurate forecasting of key pollutants such as PM2.5 is essential for early warnings and effective environmental management. While traditional models struggle with nonlinear interactions and long-term dependencies in complex datasets, Transformer-based architectures—originally developed for natural language processing—have emerged as powerful tools for sequential modeling. By leveraging self-attention mechanisms, they can capture intricate temporal patterns and cross-variable relationships, making them well-suited for air quality prediction tasks in complex urban environments like Metro Manila.

2 Application of Temporal Fusion Transformers

2.1 Data and Preprocessing

The Temporal Fusion Transformer (TFT) was applied to the task of 7-day PM2.5 forecasting across 12 cities in Metro Manila. The dataset consisted of air quality measurements collected between January and December 2024, yielding a total of 35,570 records with complete temporal coverage. The input variables included multiple air pollutants (CO, NO, NO₂, O₃, SO₂, PM₁₀, NH₃), meteorological indicators (temperature, humidity, wind speed), and temporal features (hour-of-day, day-of-week, month, weekend status). City identifiers were also incorporated to account for spatial heterogeneity. Data preprocessing involved forward-fill imputation to address missing values, normalization of numerical variables, and categorical encoding for temporal and spatial features. This ensured consistency and stability during model training.

2.2 Model Architecture

The TFT implementation utilized:

Encoder length: 30 days (historical context)

Decoder length: 7 days (forecast horizon)

Variable selection networks: To identify relevant inputs from multivariate time series

LSTM encoder-decoder layers: For short-term sequence modeling

Multi-head attention: For long-term dependency capture

Quantile loss function: To provide predictive intervals (0.1 to 0.9 quantiles)

The model was trained using chronological splitting with early stopping, incorporating 32 hidden units and 4 attention heads. Training utilized the Adam optimizer with a learning rate of 0.01 and gradient clipping for stability.

2.3 Experimental Results

The TFT model demonstrated exceptional performance in PM2.5 forecasting:

Overall Performance Metrics:

MAE: 0.513 μg/m³

RMSE: 0.618 μg/m³

R²: 0.9975

Day-wise Forecast Accuracy:

Table 1. Key Model Performance Metrics:

Day	MAE (μg/m³)	RMSE (μg/m³)	R²
1	0.4630	0.5514	0.9971
2	0.3760	0.4291	0.9951
3	0.4203	0.4435	0.9751
4	0.5125	0.6051	0.9735
5	0.4867	0.6428	0.9978
6	0.5047	0.5541	0.9988
7	0.8276	0.9501	0.9953

The model achieved a 91% reduction in validation loss during training, demonstrating effective learning of complex temporal patterns without overfitting.

3	Impact and Benefits

3.1 Enhanced Forecasting Accuracy and Interpretability

The TFT model achieved exceptional accuracy with MAE below 0.83 μg/m³ across all 7 forecast days, significantly outperforming traditional time-series approaches. Similar improvements in forecasting accuracy have been documented in prior studies applying Transformer-based models to air pollution prediction tasks, underscoring their capacity to model complex temporal and cross-variable dependencies [2]. Moreover, the architecture’s inherent interpretability through attention mechanisms provided insights into feature importance, revealing that nitrogen oxides (NO, NO₂) and temporal patterns (hourly, weekly variations) were primary drivers of PM2.5 variability across Metro Manila. This interpretability transforms forecasting from a black-box prediction to an explainable decision-support tool, consistent with prior findings that emphasize interpretability as critical for real-world scientific applications of machine learning [3].

3.2 Robustness in Real-World Conditions

Metro Manila's air monitoring infrastructure faces challenges, including sensor inconsistencies and data gaps. The TFT demonstrated remarkable resilience, maintaining high performance (R² > 0.97) despite these real-world data quality issues. This robustness is aligned with prior research showing that deep learning approaches can adapt effectively to noisy or incomplete urban air quality datasets, thereby ensuring reliable performance in less-than-ideal monitoring environments [4]. The model’s ability to handle multivariate inputs with missing values represents a significant advancement over conventional statistical methods that require complete datasets, making it well-suited to the realities of Metro Manila’s monitoring infrastructure.

3.3 Policy-Relevant Probabilistic Forecasting

Unlike point forecasts that provide single predictions, the TFT’s quantile output enables probabilistic forecasting with uncertainty quantification. This capability is particularly valuable for risk-aware decision-making, allowing health authorities to issue targeted advisories based on probability thresholds, allocate resources more effectively during high-pollution episodes, and communicate risk levels more transparently to the public. Prior studies have highlighted the importance of uncertainty-aware forecasting in environmental and health applications, where overconfidence in single predictions can lead to poor policy outcomes [1]. The 7-day forecasting horizon provides sufficient lead time for preventive measures while maintaining clinical relevance for respiratory health protection, a crucial aspect for cities with recurring pollution events.

3.4 Scalability and Generalizability

The implemented framework demonstrated successful application across 12 cities in Metro Manila, indicating strong potential for scalability to other urban areas in the Philippines. The architecture’s flexibility allows the incorporation of additional data sources, including satellite observations and traffic patterns, for enhanced forecasting capability. Similar approaches combining deep learning with heterogeneous environmental and urban datasets have proven effective in scaling solutions to broader regions [2]. This generalizability supports nationwide air quality management initiatives while accommodating local contextual factors, reinforcing the role of TFT as a foundation for scalable, AI-driven environmental governance.

Conclusion

The Temporal Fusion Transformer has demonstrated exceptional capability for air quality forecasting in Metro Manila's complex urban environment. The model achieved outstanding accuracy (MAE: 0.513 μg/m³, R²: 0.9975) across 7-day forecast horizons while providing interpretable insights into pollution drivers. This implementation successfully addresses key challenges in urban air quality forecasting, including multivariate input handling, long-term dependency capture, and real-world data irregularities.

TFT has significant ability to produce probabilistic forecasts, which supply a spectrum of possible outcomes rather than a single deterministic estimate. These results confirm TFT's superiority over traditional methods for environmental forecasting tasks, particularly in settings with complex temporal dynamics and multiple influencing factors. The model's interpretability features enable evidence-based policy decisions, while its probabilistic outputs support risk-aware public health interventions.

Taken together, the findings highlight TFT’s superiority in handling complex temporal dynamics and multiple influencing factors over traditional methods, such as ARIMA or LSTM. Traditional approaches often fail when data is incomplete or highly nonlinear. Temporal Fusion Transformers thrive under these conditions, making it a strong candidate for real-world operational deployment in environmental monitoring systems.

Future work could extend this application through:

Integration of satellite-based remote sensing and traffic data

Finer spatial resolution (barangay-level forecasting)

Real-time deployment for operational forecasting

Transfer learning to other Philippine urban centers

Enhanced interpretability visualizations for stakeholder engagement

In conclusion, this study establishes a strong foundation for AI-driven environmental management in the Philippines. By demonstrating that advanced deep learning architectures such as Temporal Fusion Transformers can deliver accurate, interpretable, and actionable forecasts, it underscores the transformative role of artificial intelligence in urban sustainability. The results not only advance the technical frontier of time-series forecasting but also provide a pathway toward healthier, more resilient, and better-managed cities.

5	References

[1]	Lim, B., & Arik, S. Ö. (2019). Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting. NeurIPS.

[2]	Yi, X., Zhang, J., Wang, Z., Li, T., Zheng, Y. (2020). Deep distributed fusion network for air quality prediction. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2586–2595.

[3]	Lagerquist, R., McGovern, A., Ebert-Uphoff, I. (2021). The importance of interpretability in machine learning for science applications. Computers & Geosciences, 157, 104943.

[4]	Zheng, Y., Liu, F., Hsieh, H.P. (2022). U-Air: When urban air quality inference meets big data. ACM SIGKDD Explorations Newsletter, 16(2), 1–20.