Failure Modes & Known Limitations
Honest, structured documentation of where our models perform poorly or produce unreliable results. This page is updated as new limitations are identified.
Why we publish this: Scientific integrity requires acknowledging limitations. Policy-makers and reviewers should not make decisions based solely on our results without understanding the conditions under which the models break down.
High severityRL Optimisation
RL Training Instability at High Renewable Shares
All tested RL algorithms exhibit training instability when the renewable share exceeds ~85% of installed capacity. The reward signal becomes noisy due to high curtailment variability, causing policy gradients to oscillate.
Conditions
Occurs consistently when: renewable share > 85%, no storage constraint, stochastic demand ±20%.
Mitigation
Reward shaping with curtailment penalty smoothing partially mitigates this. Full resolution requires explicit storage modelling in the action space (planned in Phase 4).
Medium severityForecasting
LSTM Forecast Degradation During Price Spikes
The LSTM demand/price forecast model significantly underestimates the magnitude of electricity price spikes (>3σ events). During the 2021–2022 European gas crisis, retrospective validation shows the model would have underforecast peak prices by 40–60%.
Conditions
Price spike events: TTF gas price > €100/MWh, or geopolitical supply disruptions.
Mitigation
A separate spike detection module (logistic regression on gas futures) was added in LSTM-v2.1 to flag anomalous conditions. Forecast confidence bands are widened automatically when spike risk is elevated.
Medium severityEnergy System Model
TIMES Model Underestimates Grid Flexibility Value
The TIMES benchmark model uses a simplified demand response representation that underestimates the value of demand-side flexibility by approximately 15–25% compared to RL-derived shadow prices.
Conditions
Affects all TIMES-derived results. Not a failure of the RL model — rather a known limitation of LP-based energy system models.
Mitigation
RL results are presented as primary; TIMES used only for calibration and constraint checking.
Low severityRL Optimisation
DQN Cannot Handle Continuous Action Spaces
DQN requires discretisation of the continuous investment action space (0–5 GW per technology). A 5-level discretisation was used, limiting precision. This explains DQN's lower performance relative to continuous-action algorithms (PPO, SAC, TD3).
Conditions
Inherent to DQN algorithm design.
Mitigation
Included in comparison only to benchmark against the continuous-action family. Noted in all comparison tables.
Low severityData
EPİAŞ Data Gaps for Pre-2015 Period
EPİAŞ intraday electricity price data is only available from 2015. Longer-horizon LSTM training uses estimated pre-2015 prices derived from fuel cost regression, introducing uncertainty in early training data.
Conditions
Affects LSTM models trained on price series starting before 2015.
Mitigation
Pre-2015 data is flagged as 'estimated' in the dataset. LSTM training ablation shows negligible impact on forecast accuracy for the test period (2020–2023).