Skip to content
Released models / daily development benchmark

1.1 vs 1.2.

Pressure-based intensity. Track position. Track direction.

Complete & verified1.2: real mean of 50
1,473 daily forecasts · 270 equally weighted stormsOne typhoon-day = one case. Lead errors → daily score → storm score.
+6 to +120 hours · 50-member meanSame cases, valid times and common label masks within each comparison.

Track uses all 1,473 starts / 270 storms; pressure uses 134 matched starts / 40 storms. The other 1,339 starts lack the frozen 1.1 intensity inputs, not zero error. DeepMind Mini has not run on this daily cohort. Development evidence, not an untouched test.

Lower is better

Intensity · central pressure

JMA central-pressure MAE · hPa

  1. Trackformer 1.113.53 hPa
  2. Trackformer 1.212.84 hPa
  3. DeepMind MiniNot evaluated
40 storms · 134 daily starts · 2,637 paired leads
Lower is better

Track · position

Mean position error · km

  1. Trackformer 1.1798.4 km
  2. Trackformer 1.2471.2 km
  3. DeepMind MiniNot evaluated
270 storms · 1,473 daily starts
Lower is better

Track · direction

Six-hour heading error · degrees

  1. Trackformer 1.151.58 °
  2. Trackformer 1.234.96 °
  3. DeepMind MiniNot evaluated
270 storms · 1,473 daily starts

1.1 · frozen native ensemble1.2 · 50 distinct input-perturbed membersDeepMind Mini · not evaluated here, not zero

The mean JMA pressure error is slightly lower for 1.2, but its paired 95% whole-storm interval [-2.68, 1.20] hPa includes zero. No universal intensity improvement is established.

Example · pressure through time

2024141N03142 · issue 2024-05-22 00:00 UTC. All 134 matched starts are selectable; the first example is not cherry-picked.

9659901015Central pressure (hPa)+6h+24h+48h+72h+96h+120hObserved JMA best trackTrackformer 1.1Trackformer 1.2 · mean of 50Trackformer 1.1: 1,005.57 hPa at +6hTrackformer 1.2 · mean of 50: 1,000.25 hPa at +6h
● JMA best track● Trackformer 1.1● Trackformer 1.2 · mean of 50
Observed · JMA
Unavailable
Verification only
Trackformer 1.1
1,005.57 hPa
Reference unavailable
Trackformer 1.2 · mean of 50
1,000.25 hPa
Reference unavailable

Same physical hPa scale and exact valid UTC; no shifting or stretching. Reference gaps stay unavailable, not zero. Hover the graph or move the slider to inspect each time.

All 20 forecast times · observed, 1.1 and 1.2 values
EWINIAR · issue 2024-05-22 00:00 UTC · central pressure in hPa
Lead / valid UTCObserved JMA1.11.2 · mean of 50
+6h2024-05-22 06:00 UTCUnavailable1,005.57 hPa1,000.25 hPa
+12h2024-05-22 12:00 UTCUnavailable1,007.00 hPa999.96 hPa
+18h2024-05-22 18:00 UTCUnavailable1,007.40 hPa999.77 hPa
+24h2024-05-23 00:00 UTCUnavailable1,007.08 hPa999.97 hPa
+30h2024-05-23 06:00 UTCUnavailable1,006.35 hPa999.16 hPa
+36h2024-05-23 12:00 UTCUnavailable1,005.47 hPa998.19 hPa
+42h2024-05-23 18:00 UTC1,004.00 hPa1,004.47 hPa996.20 hPa
+48h2024-05-24 00:00 UTC1,006.00 hPa1,003.20 hPa992.93 hPa
+54h2024-05-24 06:00 UTC1,004.00 hPa1,001.87 hPa989.02 hPa
+60h2024-05-24 12:00 UTC1,006.00 hPa1,000.60 hPa984.68 hPa
+66h2024-05-24 18:00 UTC1,004.00 hPa999.30 hPa981.35 hPa
+72h2024-05-25 00:00 UTC1,006.00 hPa997.62 hPa978.94 hPa
+78h2024-05-25 06:00 UTC1,004.00 hPa996.08 hPa978.35 hPa
+84h2024-05-25 12:00 UTC1,002.00 hPa994.54 hPa977.83 hPa
+90h2024-05-25 18:00 UTC998.00 hPa993.49 hPa977.34 hPa
+96h2024-05-26 00:00 UTC994.00 hPa992.52 hPa977.05 hPa
+102h2024-05-26 06:00 UTC990.00 hPa991.79 hPa976.80 hPa
+108h2024-05-26 12:00 UTC980.00 hPa990.99 hPa976.68 hPa
+114h2024-05-26 18:00 UTC975.00 hPa990.50 hPa976.74 hPa
+120h2024-05-27 00:00 UTC970.00 hPa990.81 hPa976.65 hPa
Other verified measures, lead curves & source data
Secondary diagnostics, unchanged. Each row keeps its own matched coverage; agencies are not mixed.
Measure1.11.2Coverage
JMA pressure-curve similaritysimilarity · higher is better0.70740.7118134 starts / 40 storms
USA pressure-curve similaritysimilarity · higher is better0.72370.6637131 starts / 40 storms
Mean track errorkm · lower is better798.4471.21473 starts / 270 storms
+120 h track errorkm · lower is better1,646.41,031.61473 starts / 270 storms
Six-hour direction errordegrees · lower is better51.5834.961473 starts / 270 storms
Centred route-shape similaritysimilarity · higher is better0.75440.88371473 starts / 270 storms
Geographic path similaritysimilarity · higher is better0.53450.65601473 starts / 270 storms
Frechet path distancekm · lower is better1,658.11,045.81473 starts / 270 storms
JMA central-pressure MAEhPa · lower is better13.5312.84134 starts / 40 storms
USA central-pressure MAEhPa · lower is better12.8412.55134 starts / 40 storms

Pressure error by forecast lead · JMA

01019+6h+24h+48h+72h+96h+120h1.1: +6h 4.51 hPa; +12h 5.35 hPa; +18h 6.63 hPa; +24h 8.55 hPa; +30h 10.40 hPa; +36h 11.42 hPa; +42h 12.73 hPa; +48h 14.50 hPa; +54h 15.65 hPa; +60h 16.37 hPa; +66h 17.12 hPa; +72h 17.93 hPa; +78h 17.17 hPa; +84h 16.43 hPa; +90h 16.47 hPa; +96h 16.03 hPa; +102h 16.16 hPa; +108h 16.20 hPa; +114h 15.65 hPa; +120h 15.95 hPa1.2: +6h 5.62 hPa; +12h 7.25 hPa; +18h 8.85 hPa; +24h 10.88 hPa; +30h 12.11 hPa; +36h 12.82 hPa; +42h 13.28 hPa; +48h 14.66 hPa; +54h 15.20 hPa; +60h 14.96 hPa; +66h 15.14 hPa; +72h 15.70 hPa; +78h 15.26 hPa; +84h 14.49 hPa; +90h 14.09 hPa; +96h 14.15 hPa; +102h 14.16 hPa; +108h 13.77 hPa; +114h 12.80 hPa; +120h 12.44 hPa
● Trackformer 1.1● Trackformer 1.2 · mean of 50

Track position error by forecast lead

08891778+6h+24h+48h+72h+96h+120h1.1: +6h 66.11 km; +12h 133.48 km; +18h 202.36 km; +24h 273.29 km; +30h 346.90 km; +36h 419.88 km; +42h 493.57 km; +48h 568.09 km; +54h 644.13 km; +60h 722.62 km; +66h 803.59 km; +72h 886.62 km; +78h 973.28 km; +84h 1060.58 km; +90h 1151.58 km; +96h 1244.97 km; +102h 1343.59 km; +108h 1443.19 km; +114h 1544.06 km; +120h 1646.40 km1.2: +6h 48.28 km; +12h 91.13 km; +18h 129.12 km; +24h 164.48 km; +30h 199.07 km; +36h 234.93 km; +42h 272.48 km; +48h 311.48 km; +54h 353.47 km; +60h 399.38 km; +66h 449.27 km; +72h 502.49 km; +78h 558.32 km; +84h 616.18 km; +90h 677.66 km; +96h 740.93 km; +102h 808.80 km; +108h 880.08 km; +114h 954.91 km; +120h 1031.63 km
● Trackformer 1.1● Trackformer 1.2 · mean of 50

All 1,473 original forecast hashes and equal-storm route scores were reproduced. All 134 actual 50-member intensity replays passed their original-pressure and member/hash checks. No weights, forecasts or calibration were changed.

Daily plan SHA-256 965184f1e5ab3fcb52e304f39b38583a14f753870aa2295f2d01942a441dcfd6

1.2 source checkpoint SHA-256 f194a23d3f91ea76ad776dfad942fabd669eeae8b3fd665815463095367e9ee0

Public benchmark API ↗ · Verification receipt · All daily route/storm scores · Pressure masks and diagnostic report

Wind/radius diagnostics remain in the report, not the main charts. Basin-wide pressure-field error is a separate 1.2-only diagnostic (2.72 hPa); no missing 1.1 field score is fabricated. Curve-similarity diagnostics remove mean and amplitude; the actual hPa examples above do not.

Development archive Earlier checkpoints, regional cohorts, TIP and training diagnostics