*Problem Categories in Energy Transition Planning: Four main problem categories (Forecasting, Optimization, Integration, Validation) and their sub-problems. Shows the methodological challenges addressed in this dissertation.*
Designing optimal pathways for Turkey's renewable energy transition towards the 2050 technical planning horizon (aligned with the 2053 net-zero political commitment) encapsulates many interconnected and interdependent problems in forecasting, optimization, integration, and validation. The following section lays out the four types of problems that motivate this dissertation.
Forecasting Challenges for Long-term Energy Capacity Projections
It is methodologically challenging to forecast the energy capacity for a long-run horizon of 26 years (2024–2050). Moreover, there are conventional forecasting approaches do not suitably resolve this issue. Conventional statistical methods such as Auto-Regressive Integrated Moving Average (ARIMA), Seasonal ARIMA (SARIMA), Vector Auto-Regression (VAR), etc. are based on linear or log-linear assumptions that fail to capture intricacies of renewable energy deployment's non-linear technology adoption behaviour. The adoption of technology typically follows an S-form. Rogers [rogers2003diffusion] theorized this S-form; Bass [bass1969new] quantified this S-form. The adoption is slow at first, then it accelerates exponentially, thereafter the market saturates. Linear models tend to underestimate growth during periods of acceleration, while overestimating growth during periods of saturation. In particular, forecasts with more than 10 years horizon typically show very large errors (with published estimates of 20-40%, depending on technology and context; [hong2016forecasting]; [@lago2018forecasting]).
Deep learning techniques have achieved great results in short (hours to months) term forecasting by learning complex temporal patterns from data. More specifically, Long Short Term Memory (LSTM) networks, Gated Recurrent Units (GRUs), and Transformer architectures are now the state-of-the-art forecasting methods. Nonetheless, there are serious limitations on these models for multi-decade energy forecasting. They learn statistical patterns, operate as black boxes and don't use knowledge on physical constraints. This results in physically unrealistic predictions, such as negative installed capacity, capacity factors above physical limits, e.g. (CF > 1.0), or energy generation that violates basic energy balance equations. Furthermore, large training dataset is another requirement in deep learning models. But the historical renewable energy data for Turkey provides insufficient samples for learning long-term projection patterns. Furthermore, using data outside of the model's training introduces a lot of uncertainty. For instance, take models trained on past growth rates, which do not apply to future policy regimes, new technology maturation, or future economic conditions.
Current methods used to quantify uncertainty in forecasts are ineffective for characterizing uncertainty in forecasts used for making policy decisions. Forecasts that focus on point estimates (deterministic forecasts for example a point value) tend to ignore various uncertainties. Establishing prediction intervals from past residuals through the assumption of stationary error distribution violates during structural shifts, such as accelerated installation of renewable energy sources. Monte Carlo simulation methods can produce uncertainty distributions but tend to be not calibrated. For example, the coverage probability may equal 80%, whereas the nominal level is equal to 95% (e.g. 95% confidence intervals miss true values in 20% of cases) [hyndman2021forecasting].
In addition, the majority of forecasting studies look at technologies as a whole (e.g. total installed solar capacity), rather than disaggregating by technology subtypes (i.e. utility-scale versus distributed versus roof-top PV versus concentrated solar power), resource quality classes (i.e. Class I-IV wind sites); and/or regional distributions. Masking important differences in costs, capacity factors, grid integration requirements, and deployment timelines this aggregation. Turkey's renewable resource distribution shows significant spatial diversity, with solar irradiation varying from the Black Sea region value of 1,120 kWh/m²/year to a Mediterranean value of 1,900 kWh/m²/year [menr2024strategic]. The capacity factor of wind for an inland location is 25%, while that for a premium offshore site is 45%. Aggregated forecasts cannot inform policy design for specific regions or technologies.
Interdependencies make forecasting difficult in multi-technology and multi-region contexts. The deployment of solar and wind does not happen independently in a vacuum as they both compete for the same limited grid capacity, transmission infrastructure, and financing. The deployment of storage relies on intermittent renewable performance. Rate of fossil fuel retirement depend on renewable availability and policy mandates. The interdependencies of all these (technologies) must be captured by using multivariate forecasting frameworks. However, most published literature has conducted modeling using univariate time series models for each technology. This is done in isolation, ignoring inter-technology interdependencies.
Optimization Challenges for Multi-Objective Energy Transition Planning
The energy transition pathway for Turkey is a high-dimensional, multi-objective, long-horizon, stochastic sequential decision-making problem which traditional optimization methods cannot solve. The decision space includes a large number of energy technologies, regions and time steps annually (2024–2050), giving rise to an enormous solution space [ringkjob2018energy].
We must think about conflicting criteria at the same time as they are multi-objective problems. Economic objectives aim to minimize the total system costs (capital expenditure, operating expenditure, and financing costs). These costs are estimated at $200-450 billion, USD ($2023) over 2024–2050 depending on deployment pathways. The environmental objectives intends to achieve maximum CO₂ emission reduction of ≥87% from 152 MtCO₂ (2024 baseline) to <20 MtCO₂ (2050). The objectives of energy security emphasize the need to reduce import dependency as well as diversify supplies. The adequacy constraints imposed by grid reliability objectives ensure that supply meets demand in all hours – with a reliability reaching or exceeding 95%. Socio-economic objectives include generating employment, regional equity and a just transition for fossil fuel workers. Having objectives regarding technology diversity will avoid over-concentration in single technologies which can be vulnerable to systemic risks.
These goals show basic trade-offs which cannot be handled by a simple weighted scalarization. In other words, minimizing the costs means delayed investment (due to technology costs decline and discounting) while maximizing emission reduction means immediate aggressive deployment. While onshore wind costs less ($30-40/MWh), its benefits are more concentrated in certain areas. Distributed solar, on the other hand, has a higher cost ($60-80/MWh) but creates benefits in many places and reduces the need for new transmission. For example, natural gas combined cycle plants with firm capacity (capital cost of $700/kW; operating cost of$50/MWh) can provide grid flexibility but at the cost of emissions. Battery storage does provide flexibility with zero emissions but at a higher cost ($300/kW declining to$150/kW by 2035).
The long decision horizon (26 years) makes things complicated. In 2024, the investment decisions we make will affect system performance and costs within 2024–2050 and beyond. Benefits will occur over the course of decades, while costs will also emerge over decades. Standard optimization methods that employ Net Present Value (NPV) or discounting do not weight long term benefits appropriately. The social discount rates used in optimizing climate policy, which range between 3 and 7%, render costs and benefits beyond 15 to 20 years virtually irrelevant to optimizing objectives. This conflicts with intergenerational equity principles [stern2007economics]; [2017]; [Stiglitz]; [et]; [al.].
Long-term energy planning is characterized by uncertainty. Predictions of Electricity Demand for 2050 varied from 550 TWh to 750 TWh depending on the combined assumptions on GDP growth (2.5-6% CAGR), impreferences.bib'deki anahtarlarla eşleştirovement in energy intensity (1-3%/year) and electrification of transport (EV) and heating. Solar PV financing costs have dropped 85 per cent over 2010-2023 but future learnings remain uncertain as costs could range $200-600/kW by 2050 [iea2023c]. The evolution of policy is highly uncertain. For example, carbon pricing could vary between$0-200/tCO₂, renewable support mechanisms could shift between FITs, auctions, and tax credits, and grid codes could require different interconnections. Climate change influences hydroelectric production (±20% variation from year to year), wind capacity factors (±10% variation) and demand for cooling/heating.
Standard optimization methods provide inadequate solutions to these challenges linear programming, mixed-integer linear programming, and deterministic scenario optimization. The objective functions and the constraints need to be convex and linear in linear programming. This is violated in the case of non-convex technology learning curves, network effects, and minimum efficient scale constraints. The discrete choice technology permitting mixed-integer programming performs poorly at scale. In this case, the problem with thousands of integer variables requires excessive computation and gives no guarantee of optimality [bertsimas1997introduction]. The examination of a limited number of expert-defined scenarios (typically 3-10) in scenario optimization does not explore the vast solution space of any of these pathways. Deterministic solutions are brittle solutions that fail with perturbations in the value of the parameters.
Integration Challenges: Bridging Forecasting and Optimization
According to Prina [prina2020integration], forecasting and optimization are usually treated as a sequential, decoupled processes in most energy system studies. A regular workflow is one in which (1) external forecasting models (often constructed by international bodies like IEA, or IRENA or national planning bodies) issue demand and cost forecasts; (2) forecasts are then input as fixed parameters to optimization models; (3) optimization solves for capacity expansion plans on the assumption that forecast values will be realized exactly. There are several critical deficiencies to this decoupled approach.
To begin with, uncertainty in the forecast is not transferred to optimization decisions. Models of forecasting unavoidably produce uncertain predictions, but optimization algorithms treat input parameters as deterministic. Systems for robust optimization and stochastic programming exist that handle uncertainty. However, they require forecast uncertainty distributions as inputs. Moreover, these two systems are not typically linked in practice.
Moreover, there is no feedback mechanism from optimization to forecasting. Helpful insights on optimisation that reveal critical variables, binding constraints or implausible forecast trajectories can't inform us how to improve the forecasting model. This suggests that tighter cooperation is needed in grid integration modeling and forecasting.
Third, energy balance consistency is not guaranteed between forecast and optimization. Models that forecast total generation through planned capacity and average capacity factors. For example, optimization models schedule hourly and seasonal dispatch for demand profiles. Forecasted annual generation may be inconsistent with the integral of optimized dispatch schedules, and forecasted capacity factors may be inconsistent with the optimized utilization rates. Errors in modeling lead to increasing uncertainty.
Fourth, iterative refinement opportunities are missed. A set of policy assumptions or investment paths is made in the initial forecast. Optimization might show that such trajectories are not optimal. Improving the quality of the solution through updating forecast and re-optimizing. Most studies have not made any iterative refinement since implementing such bidirectional iteration is technically complex and computationally expensive with standard tools. Chapter 3 (Methodology) describes how to implement a bidirectional forecast-optimization integration.
Validation and Statistical Rigor Challenges
Energy transition assessments often suffer from a lack of validation, insufficient benchmarking, and a lack of statistical testing. As such, one cannot have high confidence in the findings of these studies, and their usefulness for policy is limited. There are many specific shortcomings in the literature.
First, out-of-sample validation is rare. Forecasting studies often report performance metrics on the training data or the entire dataset. The mistake may lead to overfitting, where models learn historical patterns by heart rather than generalizable relationships, giving inflated performance estimates. Temporal data needs special attention; regular k-fold cross-validation disrupts temporal ordering, causing data leakage where information from the future influences the past. Performing proper temporal validation using expanding or rolling windows is costly and seldom executed.