Forecast reliability
JADE ANALYSISHow much confidence should be placed in this forecast under conditions like these? A forecast is treated as a relationship: prediction → subsequent observation.
Sydney Northern Beaches — point −33.78, 151.29 (Freshwater/Manly) and Terrey Hills AWS (BOM 066059 / GHCN ASN00066059)
Tmax MAE
0.7438 °C
n=1675 · bias 0.0626 · RMSE 1.0327
Rain Brier score
0.1875
vs Terrey Hills · climatology rate 0.4436
“No rain” then rain
16.4%
51 of 310 days with forecast p ≤ 10% recorded ≥0.2 mm.
Skill through time
Yearly MAE, forecast Tmax minus Terrey Hills. NO SIGNIFICANT CHANGE DETECTED
Pettitt test on yearly MAE (forecast − Terrey Hills Tmax)
Rain-probability calibration
Reliability versus Terrey Hills occurrence. Dashed line is perfect calibration. Bins with n < 8 are omitted.
Afternoon rain timing
INSUFFICIENT EVIDENCEHourly station observations are not in the v0.1 extract, so afternoon-rain timing skill is not reported.
Skill by regime
INSUFFICIENT EVIDENCENo reproducible East Coast Low / synoptic classifier is implemented in v0.1.
Skill by lead time
MODEL OUTPUTOpen-Meteo Previous Runs API — hourly fields, daily maximum of each run
| Lead | n | MAE | Bias | RMSE |
|---|---|---|---|---|
| lead_0_same_run | 790 | 0.8154 | -0.0456 | 1.1407 |
| lead_1_day | 790 | 1.0259 | 0.067 | 1.4347 |
| lead_3_day | 790 | 1.2157 | 0.0777 | 1.7175 |
| lead_5_day | 790 | 1.5877 | -0.0806 | 2.1609 |
Lead 0 is the same-day archived run. previous_day N is the model state from N days earlier. Not a Bureau of Meteorology official archive. Isolated misses are not treated as degradation.
Sources and limits
Open-Meteo Historical Forecast API — archived model forecast (not BOM official forecast) (2022-01-01 to 2026-08-17).
- · Open-Meteo is not the Bureau of Meteorology official forecast. Skill here is a forecast-as-relationship demonstration, not a BOM audit.
- · Terrey Hills AWS is inland and elevated (~199 m); coastal Freshwater/Manly can differ, especially in sea-breeze and orographic rain.
- · ERA5-based archive observations are not independent of some NWP systems.
- · Isolated misses are not treated as evidence that forecasting has degraded.