Methodology Overview

Interactive exploration of all models, methods, and algorithms from Forecast and Reinforcement. Click to expand, filter by category, and inspect equations, parameters, and architectures.

Forecast models

HNSM, ENSM

Algorithms

RL + evolutionary + fuzzy

446

State dims

RL environment

119

Action dims

Hierarchical

Reward components

Multi-objective

Feature categories

Forecast

Reward Function — Multi-Objective RL

The reward function guides the reinforcement learning agent toward Turkey's 2050 energy transition goals. It is a weighted sum of 8 components, each normalized to [0, 1]. The total weight is always 1.00 — when you increase one component, others decrease proportionally.

Formula: R = Σᵢ wᵢ · rᵢ, where Σwᵢ = 1. Each rᵢ ∈ [0, 1]. Thesis §5.2.

Weight Sliders — Total: 1.00 ✓

Total weight: 1.00 ✓

Drag a slider to change a weight. Other weights adjust automatically so the total stays 1.00.

renewable share

25%

Target 90% renewable share; growth + target bonus

capacity growth

20%

218 GW target by 2050; ~3808 MW/yr growth

cost efficiency

25%

CostCalculator optimization (investment, O&M, carbon)

emission reduction

15%

(current - next) / current; 87% reduction target

grid stability

10%

Penalty if stability < 0.5

investment balance

1 - std_dev of action (diversification)

import dependency

Renewable share increase reduces import dependency

energy efficiency

Efficiency change reward

Component Details — Formulas, Targets, Rationale

r_res = (res_share - res_share_prev) × growth_bonus + target_bonus × 𝟙(res_share ≥ 0.90)

Rationale: The primary policy objective is decarbonization. This component incentivizes both year-over-year growth in renewable share and a bonus when the 90% target is reached. Aligns with Turkey's 2053 Net-Zero commitment and EU Green Deal.

Implementation: Growth bonus: linear reward for each percentage point increase. Target bonus: one-time reward when renewable share crosses 90%. Normalized to [0, 1] for weighted sum.

Reference: Thesis §5.2.1, Eq. 5.4

Code: Reward calculation

# Reward calculation (Reinforcement)
component_rewards = self._calculate_component_rewards(state, action, next_state)
technology_scores = self._calculate_technology_scores(state, next_state)
penalty = self._calculate_penalty(state, action, next_state)
bonus = self._calculate_bonus(state, action, next_state, done)
total_reward = self._calculate_total_reward(
    component_rewards, technology_scores, penalty, bonus)

Forecast Models

Combines mathematical S-curve fitting with neural network capacity factor prediction. Physical energy balance: G = K × CF × 8760/1000.

sources/Forecast/advanced_neural_physical_forecasting.py

Equations (LaTeX)

Capacity:

K(t) = K_max / (1 + e^(-α(t - t₀)))

Generation:

G = K × CF × 8760/1000 (TWh)

Parameters

K_max: Saturation capacity (GW), fit via scipy.optimize.curve_fit
α: Growth rate, bounds [0.005, 0.3]
t₀: Inflection year, bounds [2005, 2030]

Architecture

CapacityForecastingNN

→ Linear(25→128)
→ ReLU
→ Linear(128→64)
→ ReLU
→ Linear(64→1)

CapacityFactorNN

→ Linear(25→512), LayerNorm, LeakyReLU, Dropout(0.2)
→ Linear(512→512), LayerNorm, GELU, Dropout(0.2)
→ Linear(512→256), LayerNorm, SiLU, Dropout(0.15)
→ Linear(256→256), LayerNorm, Mish, Dropout(0.15)
→ Linear(256→128), LayerNorm, ReLU, Dropout(0.1)
→ Linear(128→128), LayerNorm, LeakyReLU, Dropout(0.1)
→ Linear(128→64), LayerNorm, ReLU, Dropout(0.05)
→ Linear(64→1), Sigmoid

Code: S-curve fitting

# S-curve fitting (scipy)
from scipy.optimize import curve_fit

def logistic_s_curve(t, K_max, alpha, t0):
    return K_max / (1 + np.exp(-alpha * (t - t0)))

popt, pcov = curve_fit(logistic_s_curve, years, capacity,
    p0=[200, 0.06, 2022],
    bounds=([max(K_true), 0.005, 2005], [250, 0.3, 2030]))

Feature Engineering

S-Curve & Uncertainty

Validation & Benchmarks

13 Algorithms

Algorithm Comparison Table

ID	Full Name	Category	Hyperparams
PPO	Proximal Policy Optimization	RL	lr=3e-4, gamma=0.99, clip_ratio=0.2
DQN	Deep Q-Network	RL	lr=1e-3, gamma=0.99, batch_size=64
A3C	Asynchronous Advantage Actor-Critic	RL	—
SAC	Soft Actor-Critic	RL	lr=3e-4, tau=0.005, alpha=0.2
TD3	Twin Delayed DDPG	RL	lr=3e-4, policy_noise=0.2, policy_freq=2
GA	Genetic Algorithm	Evolutionary	population=100, mutation_rate=0.1, crossover_rate=0.8
PSO	Particle Swarm Optimization	Evolutionary	swarm_size=50, inertia=0.9, cognitive=2, social=2
DE	Differential Evolution	Evolutionary	population=50, mutation_factor=0.5, crossover_rate=0.9
FuzzyQL	Fuzzy Q-Learning	Fuzzy	—
NeuroFuzzy	Neuro-Fuzzy Systems	Fuzzy	—
MARL	Multi-Agent RL	Multi-Agent	—
MAML	Model-Agnostic Meta-Learning	Meta-Learning	—
ProgressiveNN	Progressive Neural Networks	Meta-Learning	—

State & Action Space

Technology capacities29

Cost metrics250

Performance metrics150

Cumulative costs4

Environmental1

Economic4

Infrastructure1

Fuel2

Time1

Energy demand/supply4

Level 1: Main categories (7)

solar, wind, hydro, geothermal, biomass, battery, hybrid

Level 2: Sub-technologies (25)

utility_pv, commercial_pv, land_based_wind, offshore_wind, ...

Level 3: Resource class (10)

excellent, good, fair, poor

Level 4: Scale (3)

utility, commercial, residential

Level 5: Scenario (4)

moderate, advanced, conservative, breakthrough

Level 6: Fine-tuning (70)

10 dims per main category × 7

Environment & Cost

Bidirectional Integration

Forecast provides 2024–2050 projections → Reinforcement optimizes policy pathways → Policy updates Forecast scenario assumptions. Fixed-point iteration until convergence.

Loading diagram…

4–7

Iterations to convergence

89%

Gap reduction

Uncertainty Propagation

Detailed model cards for each forecast model and reinforcement learning algorithm. Expand to view mathematical formulations, parameters, architecture, performance, and project usage.

Forecast Model Cards (2)

📐 Mathematical Formulation

General Form: K(t) = K_max/(1 + β·exp(-γ·t)) or K_NN(t)
CF(t) = f_NN(X_t), G(t) = K(t) × CF(t) × 8.76
Theoretical Foundation: Stage 1 — Logistic S-curve for capacity. Stage 2 — Neural network correction when S-curve fit fails (R²<0.85). Physical Balance: G = K × CF × 8760/1000 enforced at all times.
Key Innovation: Combines parametric interpretability (S-curve) with neural flexibility (MLP fallback).

K(t) = K_max / (1 + exp(-α(t - t₀))) via scipy.optimize.curve_fit
If curve_fit fails or R² < 0.85: K(t) = CapacityForecastingNN(X_t; θ)
G = K × CF × 8760/1000  (TWh)

🔢 Parameter Definitions

K_max — Maximum Saturation Capacity (GW)
α (alpha) — Logistic Growth Rate (year⁻¹)
t₀ — Inflection Point Year
ε_NN(t) — Neural Network Residual/Fallback (GW)
CF(t) — Capacity Factor (dimensionless [0, 1])
G(t) — Annual Generation (TWh/year)

Fitted Parameters

K_max ≈ 239.4 GW
α ≈ 0.097 year⁻¹
t₀ ≈ 2024.4

🏗️ Model Architecture

Stage 1: scipy.optimize.curve_fit (Levenberg-Marquardt), maxfev=20,000
Stage 2: CapacityForecastingNN 25→128→64→1 (ReLU)
CapacityFactorNN: 7 layers, 512→256→128→64 hidden, LayerNorm, LeakyReLU/GELU/SiLU/Mish, Dropout 0.2–0.05
Loss: L_total = λ₁·L_capacity + λ₂·L_generation + λ₃·L_physical

📊 Performance Metrics

Generation: R² = 0.9799, MAE = 3.1 TWh, RMSE = 4.0 TWh
Capacity: R² = 0.9087, MAE = 1.6 GW, RMSE = 2.1 GW
Best among 17 benchmarks (17–40% improvement)
Physical consistency: Energy balance error <0.1%, CF violations 0%

📉 Uncertainty Quantification

Bootstrap: 30 runs, 90% CI, PICP = 94.2%
Monte Carlo Dropout: 100 passes, PICP = 97.3%
MPIW: 11.8 GW for 2050 (±5.4% of total capacity)

💼 Project Usage

ENSM forecasts capacity/generation 2024–2050
MAPPO uses forecasts for cost/technology planning
Bidirectional loop: 4–7 iterations until convergence (<1 GW gap)
Outputs: Capacity 218.9–221.1 GW by 2050, Generation 620–724 TWh/year (95% CI)

📐 Mathematical Formulation

Primary: K(t) = K_max/(1 + exp(-α(t-t₀))) via curve_fit
Fallback: GRU/NeuralSCurve if R² < 0.85
Physical: G = K × CF × 8760/1000

K(t) = K_max / (1 + exp(-α(t - t₀)))
If curve_fit fails: K(t) = GRU(X_t; θ)

🔢 Parameters

K_max ≈ 239.41 GW
α ≈ 0.0972 yr⁻¹
t₀ ≈ 2024.39

🏗️ Architecture

Stage 1: curve_fit (logistic), success if R² ≥ 0.85
Stage 2: 4-layer GRU, 512 hidden, input 106 features
CapacityFactorNN: 7 layers, Sigmoid output

📊 Performance

R² = 0.9799 (generation)
35% better than 17 benchmarks
78 Diebold–Mariano tests (p < 0.001)
PICP 95.4%, MPIW 11.8 GW

Reinforcement Learning Algorithm Cards (14)

🏆 Best Performance

Economic: Total Cost $226.79B (lowest). vs SAC: $37.21B savings (14.1%). vs MARL: $72.21B savings (24.2%).
Environmental: Renewable Share 97.34%, CO₂ Reduction 90.6% (152→14.30 Mt)
Reliability: Success Rate 100%, Reserve Margin ≥15%, Energy Balance 0 violations
Stability: σ = 0.012 (3× lower than SAC/TD3)

📐 Mathematical Formulation

L_CLIP(θ) = E_t[min(r_t(θ)A_t, clip(r_t(θ), 1−ε, 1+ε)A_t)]
r_t(θ): Policy ratio = π_θ(a_t|s_t) / π_θ_old(a_t|s_t)
A_t: Advantage via GAE(λ=0.95), ε = 0.2 (clip ratio)

L_CLIP = E[min(r_t·A_t, clip(r_t,1-ε,1+ε)·A_t)]
GAE: A_t = δ_t + (γλ)δ_{t+1} + ...

📝 Algorithm Description

7 regional agents: Marmara (41% demand), Aegean (15%), Mediterranean (14%), Central Anatolia (16%), Black Sea (7%), Eastern (4%), SE (3%)
CTDE: Centralized Training (global critic), Decentralized Execution (local policies)
119-dim action: 7 main categories × hierarchical decomposition

🎯 State Space (446 dims)

Capacity Matrix: 26 tech × 7 regions = 182
Generation/Demand: 91
Economic: 84 (CAPEX, O&M, learning curves, fuel, carbon, financing)
Cumulative Emissions: 42
Storage/Transmission: 47

🎮 Action Space (119 dims)

Level 1: Inter-regional allocation (7)
Level 2: Technology portfolio (26)
Level 3: Annual deployment (26)
Level 4: Transmission expansion (28)
Level 5: Storage sizing (14)
Level 6: Retirement/retrofit (18)

🎁 Reward Function

R = −0.35·Cost + 0.30·Emission + 0.20·RE_share + 0.10·Reliability + 0.05·Equity + 0.05·Self_sufficiency − 0.05·Penalties

⚙️ Hyperparameters

clip_ratio: 0.2, GAE λ: 0.95, γ: 0.95
lr: 3e-4, Adam β₁=0.9, β₂=0.999
Actor: 446→[256,256]→119 (Tanh)
Critic: 446→[256,256]→1
Epochs: 10, Batch: 64, Rollout: 2048

💼 Project Usage

Bidirectional loop with ENSM: 4–7 iterations
RE Share: 94.12% → 97.34% (+3.22 pp)
Cost: $238.4B → $226.79B ($11.6B savings)

🏆 Best Performance

Same as PPO (MAPPO is multi-agent PPO with CTDE)
7 regional agents, centralized critic, decentralized execution

📐 Mathematical Formulation

PPO clipped objective with multi-agent extension
Global critic V(s₁,...,s₇, a₁,...,a₇)
Local policies πᵢ(aᵢ|sᵢ)

← System Architecture · Forecasting → · Results →