This section discusses the future research building on this dissertation. The guidance is organized into thematic blocks: (1) methodological advances (Graph Neural Networks, hierarchical RL, robust optimization), (2) high-resolution geospatial modelling, (3) real-world deployment and barrier integration, (4) digital twin frameworks, (5) short-term operational optimisation, (6) sector coupling, (7) international cooperation, (8) socio-political aspects.
Note: This part gives in-depth discussion on future research direction. Refer to section 5.4 for a summary.
Methodological Advances.
1. Energy Network Topology Modelling Via Graph Neural Networks.
This thesis adopts flat state representations (446-dimensional vectors), handling regional capacities as separate features and overlooking the spatial dependencies of the transmission network. Agents can reason about transmission constraints, congestion and regional interdependencies owing to the ability of Graph Neural Networks (GNNs) to model grid topology as graphs (see Section 2.6.3 for the preliminary). This focuses on future work where agents will learn spatial patterns with graph-based state encodings replacing flat representations. As an example transmission congestion propagates through the graph which impact the decision on where to deploy regionally. We anticipate a utilization of approximately 85% compared to the current baseline utilization of 72%. **Issues:** Need training time raised by 15% computational overhead as well as line capacities impedances topology necessary as detailed data at transmission network
2. Multi-Agent Hierarchical Reinforcement Learning:
Currently, flat hierarchy utilized in multi-agent reinforcement learning (e.g. seven regional agents, one level). Hierarchical reinforcement learning (RL) separates decision making into various levels of abstraction (1) high-level: expansion planning (annual, 26-year horizon, this work) (2) mid-level: deployment scheduling (monthly, seasonal adjustments) and (3) low-level: dispatch (hourly, real-time balancing). Next, we will create a new hierarchical reinforcement learning (RL) framework that achieves decision making at three levels. It will enable optimization at strategic, tactical, and operational timeframes simultaneously. Integrated planning-operations optimization and a reduction in computational complexity (hierarchical decomposition). Improved solution quality through coordination across different timescales. One of the many challenges is reward design which means balancing short-term operational costs against long-term strategic objectives. The second one is training complexity that is learning for multiple levels, training multiple agents.
3. Robust Optimization Under Deep Uncertainty:
The present framework account for parametric uncertainty (costs ±30%; demand ±20%) using ensembles of Monte Carlo modelling; it does not account for deep uncertainty (an unknown probability distribution, structural uncertainty, ambiguous objectives). Solid optimization approaches (such as info-gap theory and robust satisficing) help find solutions robust across large ranges of uncertainty without requiring exact probability distributions. Future work consists of applying robust optimization principles in an RL framework so that agents can learn policies that are robust to deep uncertainty (unknown technology breakthroughs, structural demand shifts, regime changes). The expected benefits are that robustness in extreme scenarios will improve. Sensitivity to distributional assumptions will reduce. Robust optimization is often more expensive, and there is a trade-off between robustness and optimality.
4. Transformer-Based Long-Term Forecasting:
Present-day ENSM utilizes LSTM-driven architecture for temporal modeling. Transformer architectures have outperformed CNNs and RNNs in multiple domains like NLP, Vision, and Forecasting. Next up, we will develop the Transformer-based forecasting model that will replace LSTMs components. Through the attention mechanism based we will focus on long-distance dependencies (26-year horizon) and multi-scale patterns (annual trends, decadal cycles, technology lifecycles) We will be able to do a better long-term extrapolation, we can deal with non-stationary patterns, and resulting model will train faster (can be parallelized vs. sequential LSTM). Transformers require really large datasets to achieve best results but the energy data available is limited (broadly speaking). Also, the approach is not as easy to interpret (the attention weights are less intuitive than LSTM hidden states).
5. The Combining of Two Carbon Markets
The current framework models carbon pricing as an exogenous parameter of $50/tonne, USD 2023. The EU and voluntary carbon markets have created carbon prices with their dynamic pricing, which depends on the balance between supply and demand. Further work can endogenize carbon pricing within the RL environment through modeling carbon market dynamics, along with allowance allocation, trading mechanisms and price formation, allowing agents to optimize capacity expansion accounting for carbon price volatility and market interactions. This is likely to lead to more plausible policy analysis, insight into best carbon price levels, and understanding of market-based policy effectiveness. The challenges here are market complexity (not just a single actor with strategic behaviour, information asymmetry) and data requirements (historical carbon price data, market structure).
High-Resolution Geospatial Energy Modeling.
1. Provincial-Level Granularity (81 Provinces):
The current framework aggregates Turkey into 7 regions, losing intra-regional heterogeneity (Section 5.3, Limitation 2). By modeling at the provincial level, the 81 provinces of Vietnam were used for disaggregation to depict finer spatial patterns of local resource endowments, demand, and policies context. In the future, we intend to expand state space up to provincial level so 446D goes up to 2500D, develop hierarchical RL where provincial agents become regional coordinators and national planner to optimal siting at provincial scale. The anticipated benefits are improved use of resources as the solar and wind potential of a region varies by 2-3 times, and better matching of demand as provincial demand centers have been identified. Due to local generation, the need for transmission will reduce. The challenges faced are computational complexity, data requirements and coordination complexity. E.g., state space explosion which was 5.6× larger.
2. Modeling at Grid & Cell Level (1-5 Km Resolutions):
Ultra-high-resolution modelling (grid cells 1-5 km) allows for precise resource (solar irradiation, wind speed, topography) assessment, optimal siting (avoiding protected areas, minimizing transmission), and environmental impact assessment (biodiversity, land use, visual). For future work the team will integrate GIS data, like land use, protected areas and transmission corridors, with RL optimization. This will allow agents to learn optimal siting taking multiple constraints into account including resource, environmental and social constraints. Less impact on environment and improved social acceptance (avoiding sensitive areas) and better routing. The problem we are working with has a very large state space, consisting of millions of grid cells. It also suffers from computational intractability, for which we will require approximation methods like hierarchical RL, spatial aggregation, sampling and others.
Real-World Deployment and Barrier Integration.
1. Full-scale implementation involving all 7 regions 26-year horizon, with ongoing monitoring and adaptation of management systems, allows for verification of model predictions vs outcomes from what gets deployed.
2. Incorporate implementation barriers (like permitting delays, financing constraints, social acceptance) into RL optimization. Do not only impose them as constraints. This is modeled as state variables that affect the feasibility of actions which inherently have institutional capacity, regulatory complexities and stakeholder dynamics.
3. Users and stakeholders co-create, together with the developer, the optimization function for the drone swarm. Iterative policy refinement and preferencing their elicitation through workshops.
4. Comparative Pilots - Implementing multiple pilots in different regions such as resource-rich vs. demand-center, developed vs. developing, to test generalizability and nuances affecting model performance.
Digital Twin Framework for Real-Time System Monitoring and Adaptive Optimization.
Traditional energy planning models have offline mode: create optimal pathways, implement policies, and evaluate outcomes ex post. Digital Twin structures allow for continuous monitoring of the system, continual updating of the model, and continual improvement of appropriate policies that connect planning and operations.