Methodology & Data Sources

This page documents the modelling approach, assumptions, and data sources used by the forecasting and dispatch model.

Two-stage participation model

The backtester uses a two-stage model to separate FR availability revenue from energy arbitrage revenue without double-counting the same physical capacity.

Stage 1 — day-ahead capacity allocation. For each day D, the model decides how many MW to commit to FR services versus hold back for arbitrage. Two signals are compared using only information available at the end of day D-1:

Capacity is allocated proportionally, fr_fraction = fr_value / (fr_value + arb_value), so more MW flows toward whichever stream looks more attractive that day — without all-or-nothing switching.

Stage 2 — intraday dispatch. Within the allocated arbitrage MW, the selected strategy schedules charge/discharge against forecast prices and realises revenue against actual day-D prices. The same price signal drives both stages.

The MPC engine tracks SoC continuously at 30-minute resolution, enforcing the FR headroom band [10%, 90%] as a hard constraint at every step of the planning horizon. The remaining simplification is that Stage 1 allocation is a daily heuristic rather than being jointly co-optimised with the intraday LP — see known limitations.

Dispatch strategies

Intraday dispatch is driven by a rolling Model Predictive Control (MPC) linear programme, re-solved at every 30-minute settlement period. At each period t the LP plans over a -period (-hour) horizon, returns only the first period's decision, then re-solves — a receding-horizon approach reflecting the real constraint that dispatch must be committed before future prices are known.

LP formulation. Decision variables are charge power p_chg[t], discharge power p_dis[t], and state of charge SoC[t+1] over the horizon. The objective maximises net arbitrage revenue minus cycling degradation cost:

maximise  Σ price[t] × (p_dis[t] − p_chg[t]) × 0.5h  −  cycling_cost × Σ p_dis[t] × 0.5h

subject to:

Mutual exclusion of simultaneous charge and discharge is handled by LP relaxation: because the objective penalises cycling, simultaneous charge and discharge is never optimal at a positive spread, so no binary variables are required. Solved with the CLARABEL interior-point solver bundled with cvxpy (Diamond & Boyd, 2016).

Three price signals are benchmarked — all run the identical dispatch engine; only the forecast fed to the LP differs.

Strategy Price signal fed to LP What it represents
Perfect Foresight Actual day-D wholesale prices Theoretical ceiling — needs advance knowledge of the future
Naive (D-1 prices) Yesterday's 48 half-hourly prices Zero-skill floor; any real model must beat this
ML Model Random Forest forecast for day D Realistic best case using features available at end of D-1

Dispatch decisions execute unconditionally at actual prices. Per-period revenue can be negative when forecast error causes an unfavourable trade — that is the realistic operational outcome and is intentional.

The foresight ratio summarises how much of the theoretical ceiling each strategy achieves. For LP-based joint co-optimisation of arbitrage and frequency response in GB, see Swierczynski et al. (2021).

Ancillary service availability revenue

Wholesale energy arbitrage revenue

Negative clearing prices

GB frequency response auctions clear below zero more often than is widely appreciated: 13.5% of auction records in this dataset (8,134 of 60,054) have a negative clearing price, concentrated in DR High (5,205 records) and DM High (2,816). As the storage fleet has grown, procurement volumes have been outpaced and the High-side services in particular have tipped into oversupply.

These are included in revenue. Negative prices are a real feature of a maturing flexibility market, and excluding them overstates FR income — by roughly 12% of total modelled revenue on this dataset. Earlier versions floored clearing prices at zero, which silently removed those records; the floor is now opt-in rather than a default.

Capacity allocation treats them differently, and deliberately so. The Stage 1 allocator splits each block's capacity in proportion to the value of each stream, and capacity cannot rationally be allocated toward negative expected value — so the FR signal used for the split is clamped at zero. Two consequences:

Without the clamp the proportional split is undefined: a negative numerator produces negative committed MW, which propagates into the LP's power bounds, and the denominator can cross zero and make the fraction unbounded.

Availability factor

Cycling wear cost and battery degradation

ML price forecast model

A Random Forest regressor predicts the 48 half-hourly APXMIDP prices for day D using features available at the end of day D-1.

Why Random Forest? The feature set is tabular (lagged prices, generation-mix ratios, temporal encodings) rather than sequential; trees need no feature scaling, are robust at this data size, and give interpretable importances. This is consistent with the electricity price forecasting literature, which finds tree-based methods competitive against deep learning on short-horizon day-ahead tasks (Lago et al., 2021; Weron, 2014).

Features (all available at end of day D-1):

Reading the REPD fleet series. REPD is a planning database published quarterly, with two consequences. Recent months are revised upward as projects are confirmed operational — the Q4 2025 extract put March 2026 at 3.4 GW, while the Q2 2026 extract puts the same month at 5.0 GW, so figures for the most recent year are provisional. And the extract always lags the half-hourly price data, so trailing months are projected from a 12-month linear trend anchored at the last measured value, keeping the cumulative series monotonic. Projected months are flagged is_extrapolated: currently of months (measured through , projected to ). Months before the first REPD entry are zero; months after the last carry the most recent measured capacity forward rather than dropping to zero.

Train/test split. A strict temporal split: training ends before , so the model never sees future prices. Training uses an expanding window; the held-out test period runs from to the end of the data. The split date is held fixed as data is extended, so each refresh adds to the out-of-sample period rather than to training — and the test window stays longer than a full year, so seasonal performance can be assessed across a complete annual cycle.

Known limitations. Tree-based models cannot extrapolate beyond price ranges seen in training; electricity price forecasting is inherently noisy; and the model improves dispatch quality on average without eliminating error on individual days. Current metrics and feature importances are on the Forecasting & Dispatch page.

Known limitations

Not yet modelled

Approximations in the current MPC LP

Battery degradation (not yet modelled)

A real asset degrades through calendar ageing (capacity fade at rest) and cycle ageing (accelerated by depth of discharge, C-rate and temperature). A more complete model would track state-of-health across the backtest, apply a degradation-aware dispatch policy trading short-term revenue against cycle-life consumption, and incorporate chemistry-specific degradation curves (NMC, LFP), which differ materially. The cycling wear cost is a simplified financial proxy and does not capture the compounding, path-dependent nature of real degradation.

Data sources

Coverage is read from the processed datasets at build time rather than hardcoded, so this table cannot drift out of date when the pipeline is re-run. Both NESO and Elexon APIs are fully public and require no key. NESO Data Portal · Elexon Insights · DESNZ REPD

Literature & references

Electricity price forecasting

MPC dispatch engine & LP solver

BESS dispatch optimisation & co-optimisation

Battery degradation modelling

GB BESS market context