Skip to content

Median skill vs NBM, by hours ahead (validation)

+1 h+17.9%+6 h+10.6%+12 h+6.2%

VALIDATION · NOV–DEC 2024 · BLIND 2025 LOCKED

Research preview · NASA model test not run yet

Can a NASA weather foundation model sharpen local forecasts?

Every weather forecast office has small, predictable temperature errors: a little too warm here, a little too cool there. We built a small AI model for each of the 116 National Weather Service offices in the lower 48 that learns those habits and corrects the official forecast for the next 1 to 12 hours. In our tests it beats the official forecast at 100 of 116 offices and loses at none (the rest are roughly tied). Now we're testing whether a large NASA–IBM weather model can fix the one region where it still struggles: the northern Plains. A first piece of that test (arm B) has run; the NASA-model part (arm C) has not.

wfx tests whether frozen features from Prithvi WxC, the NASA–IBM weather and climate model trained on NASA's MERRA-2 reanalysis, help a small per-office network correct the National Blend of Models' 1–12 hour temperature forecasts.

Research project. Not an official NWS, NOAA or NASA product. Results are our own validation runs and are not peer reviewed. The blind-test year has not been scored.

The app maps today's NWSCnn skill vs NBM for all 116 offices.

01 · What this is

A small correction model for every forecast office.

Every office's model starts from one trained on the Sterling, Virginia office, the National Weather Service office that forecasts for Washington, DC and Baltimore.

NWSCnn is a compact convolutional network, about 494,000 parameters (≈2 MB) per office, that learns to correct 2 m temperature forecasts at 1 to 12 hours ahead for the 116 National Weather Service forecast offices in the lower 48 states. Each office gets its own model, pretrained on the Sterling, VA office (LWX) and then fine-tuned on its own area.

In plain terms, the bar to clear is the official forecast. Technically, it is the National Blend of Models (NBM), NOAA's calibrated blend that forecasters already use. Scores are an MSE skill score against NBM on the office's 2.5 km grid, checked against NOAA's URMA analysis: positive means NWSCnn had lower error.

02 · Status

Arm B has a first result. The NASA-model arm hasn't run.

The numbers in section 03 come from earlier validation runs of today's per-office models, not from the new test.

  • Done NASA Earthdata account created.
  • In progress Colab notebook for the WxC features and the NWSCnn loader for them.
  • Partly done Arm B (shared core): first results at the 14 test offices, shown in the app as an experiment (sc1).
  • Not started Arm C (shared core + NASA weather model): no Colab run, no WxC features, and no results. This page will be updated when there are.
Open the app

Today's validation skill for all 116 offices, office by office.

03 · Where it stands today

Good on average. The northern Plains are still the weak spot.

Skill vs NBM median, best GroupNorm model per office
+11.5%
At a typical office, squared temperature error is about 11% lower than the official forecast (typical error about 6% smaller).
116 offices; 100 beat NBM (95% CI above zero), 0 lose. gn1 at 56 offices, gn1lr4 at 60. Validation, Nov–Dec 2024
Northern Plains vs NBM
+3.2 to +5.1%
Aberdeen SD, Bismarck ND, Grand Forks ND and Sioux Falls SD: roughly tied with the official forecast; by 12 hours out, it runs slightly too cold (about 0.4–0.7 °F).
ABR, BIS, FGF, FSD: ties with NBM (95% CI crosses zero), slightly negative by +12 h, still a cold bias of −0.22 to −0.38 °C (−0.4 to −0.7 °F) at +12 h
Dodge City (DDC) vs NBM
+11.2%
Dodge City, Kansas: was losing to the official forecast; after tuning, its squared error is about 11% lower.
with gn1lr4 (+15.5 / +11.9 / +7.5% at +1 / +6 / +12 h); gn1 was −14.0%, losing to NBM

Getting the Plains to this point took a lower fine-tune learning rate (gn1lr4) and choosing the better model office by office, and the northern Plains still fade and run cold by +12 h. That suggests the model is missing something about how the larger-scale weather evolves over the next 12 hours, which is exactly what a global weather model is built to know. The WxC test asks whether a foundation model can close that gap and remove the per-office tuning. See every office in the app

04 · The three-arm test

Same 14 offices. Three versions.

The 8 pilot offices (BOU, BOX, LOX, MFL, MPX, OAX, PSR, SEW) plus the 6 Plains offices (ABR, BIS, DDC, FGF, FSD, GID).

  1. A

    Today's versionone model per office · gn1

    The GroupNorm per-office models (gn1), unchanged. The baseline to beat. (Not the same as the headline numbers above, which pick the better of gn1 and gn1lr4 at each office.)

  2. B

    Shared brainone shared core, small office add-on

    One core network trained across all offices. Only a small office embedding and output head are trained per office, with the core frozen.

  3. C

    Shared brain + NASA weather modelshared core + Prithvi WxC

    Arm B, plus frozen Prithvi WxC +6 h and +12 h forecast features for each office, computed on Google Colab.

Rules

Validation data only. The blind test year, 2025, stays locked and unseen. Training uses Nov 2023–Oct 2024; validation is Nov–Dec 2024.

What counts as a pass

If C doesn't clearly beat both A and B, especially in the Plains, the NASA model isn't worth its cost and we drop it.

05 · The Prithvi family

Two NASA–IBM foundation models, two very different jobs.

Prithvi WxC is a large AI model from NASA and IBM that learned how the atmosphere evolves by studying NASA's weather "replay" since 1980 (MERRA-2, below). We don't retrain it; we borrow what it "notices" about the big-picture weather around each office.

Why the northern Plains: cold air outbreaks and fast-moving fronts come from hundreds of miles away. A small local model can't see that far; the big model can.

Prithvi-EO-2.0 is NASA–IBM's land-imagery model; we don't use it here.

Technical details: Prithvi-EO-2.0

Prithvi-EO-2.0 is for land imagery: a vision transformer pretrained on NASA's Harmonized Landsat and Sentinel-2 (HLS) data at 30 m, used for tasks like crop mapping, floods and landslides. It is not a weather model, and we are not using it here.

Prithvi WxC is the weather and climate model: a 2.3-billion-parameter transformer built by NASA and IBM Research and trained on 160 variables from MERRA-2. It takes the state of the atmosphere at two times and predicts a single state, which can be in the future. It comes in two versions: a general pretrained model, and a rollout version tuned further for forecasting. We use the rollout version.

The paper reports that WxC does "well to exceptionally well" at the shortest leads, 6 and 12 hours, particularly for surface temperature, with skill falling below Pangu-Weather after about 66 hours. The short end is the only part NWSCnn needs.

06 · What MERRA-2 is

NASA's record of the global atmosphere since 1980.

An "instant replay" of Earth's atmosphere: NASA combines satellite measurements, weather balloons, aircraft and ground stations with a weather model to rebuild what the air looked like every few hours since 1980.

Technical details: MERRA-2 specs and the variables WxC uses

The Modern-Era Retrospective analysis for Research and Applications, Version 2, produced by NASA's Global Modeling and Assimilation Office (GMAO) with the GEOS model and data assimilation system (version 5.12.4). A reanalysis combines a fixed, modern forecast model with decades of observations to build one consistent, gap-free picture of the past atmosphere.

Grid
0.5° latitude × 0.625° longitude, about 50 km. Global: 361 × 576 points.
Vertical
72 model layers. Prithvi WxC uses 14 of them, at nominal pressures from 985 to 48 hPa.
Time step
3-hourly for the 3-D fields; 1-hourly for many single-level fields such as 2 m temperature and 10 m wind.
Record
1980 to present. The first long-term global reanalysis to assimilate space-based aerosol observations.
Access
Free from NASA's GES DISC archive with a NASA Earthdata login, in NetCDF.
Latency
About 3 weeks after each month ends, usually between the 20th and 25th of the next month.

20 surface variables

U10 V10 T2M QV2M PS SLP TS TQI TQL TQV GWETROOT LAI EFLUX HFLUX Z0M LWGEM LWGAB LWTUP SWGNT SWTNT

10 variables × 14 levels

U V OMEGA T QV PL H CLOUD QI QL · at 985 970 925 850 700 600 525 412 288 245 208 150 109 48 hPa

Static

PHIS FRLAND FROCEAN FRACI, plus day-of-year and hour-of-day

07 · How WxC uses MERRA-2

Masked reconstruction plus forecasting.

Because the replay is assembled afterward, it can include a few observations from slightly after the forecast time, so we treat these results as a research test, not a live-forecast score.

Technical details: inputs, pretraining, rollout tuning, what we take out

Inputs. Two MERRA-2 snapshots, 20 surface plus 10 × 14 level variables each (160 per time), on the 360 × 576 grid, together with a 1980–2019 climatology for the target time and the static fields. The model predicts the departure from climatology, so it doesn't waste effort relearning the seasonal and daily cycle.

Pretraining. Phase one hides 50% of the input and asks the model to fill it in and predict ahead at the same time. Leads are drawn from 0, 6, 12 or 24 hours, with the two inputs 3 to 12 hours apart. It trained on 1980–2019 data for 100,000 steps on 64 A100 GPUs.

Rollout tuning. The forecast version drops the masking and fixes both the input spacing and the lead to 6 hours, then is tuned over 1 to 3 chained steps. Two steps of 6 hours gives our +12 h.

What we take out. For each 12Z start time, the notebook feeds MERRA-2 at 06Z and 12Z, rolls forward to +6 h and +12 h, and keeps a box about 1.5° beyond each office's area: 14 surface and low-level channels, stored as float16. A reanalysis can include observations from shortly after its nominal time, so these features are a research input, and we read the results with that in mind.

08 · The limit

Good for research. Not live, as built.

The replay is published about three weeks late, so this setup can test the idea on past weather but can't run a live forecast yet.

MERRA-2 arrives about three weeks after each month ends. That is fine for training and for testing on past dates, but a live 12-hour forecast can't wait three weeks for its input.

If arm C wins, the next question is whether a near-real-time analysis can stand in for MERRA-2. Possibilities include NASA GMAO's GEOS-FP, from the same family of GEOS systems, or NOAA GFS analyses. Neither has been tested, and WxC was trained only on MERRA-2, so any swap would need its own check.

09 · The compression idea

One backbone, small heads.

The lesson from a recent model-compression study: most of the value sits in one shared backbone, and each new task only needs a small head.

Technical details: Du et al. and the compression result

Du et al. (2025, arXiv:2512.01181) distilled Prithvi-EO-2.0-300M into a student 16× smaller (1.2 GB to 73 MB) that matched the full model on five tasks, kept the encoder frozen, and swapped in small task heads. They then ran it on the International Space Station. That work is about land imagery, not WxC, but the lesson carries: most of the value sits in one shared backbone, and each new task only needs a small head.

NWSCnn today is the opposite: 116 full copies of the same network. Arm B tests the shared-core version on its own first. If B holds up, WxC's features only have to be wired into one core, not 116 models. If C beats B, the gain comes from WxC itself and not from the restructuring.

10 · The pipeline

Big model in the cloud. Small files home.

The big NASA model runs once on a free cloud GPU; only small per-office files come back to train our models.

Technical details: the four pipeline steps and why frozen features
  1. 01

    NASA Earthdata

    MERRA-2 files from GES DISC, signed in with a free Earthdata account. The login lives in Colab's private secrets, never in code.

  2. 02

    Colab + Prithvi WxC

    The 28.4 GB rollout weights download to Colab's disk and run in float16 on a free T4 GPU, +6 h and +12 h.

  3. 03

    Per-office features

    One small file per office and split (train, validation). Only these come back. No credentials inside.

  4. 04

    NWSCnn arm C

    The shared core plus WxC features trains and is scored on our Mac, against A and B.

Why frozen features, not fine-tuning

The rollout checkpoint alone is a 28.4 GB file, far too big to fine-tune on the Mac that also trains the 116 office models. Running WxC once, frozen, on Colab's free GPU and keeping only its outputs costs nothing and leaves the 2.3B weights untouched. This also follows the WxC paper's own pattern, where the pretrained model is "the typically frozen core" with a few new layers trained on top.

Glossary

Terms used on this page.

Show or hide the terms
NBM
National Blend of Models: the National Weather Service's official computer-forecast blend. The bar we try to beat.
URMA
Unrestricted Mesoscale Analysis: NOAA's best after-the-fact temperature map, used as the answer key.
MSE skill score
How much lower our mean squared error is than the official forecast's. +11.5% means squared error is 11.5% lower, about 6% lower typical error. It does not mean "11.5% more accurate."
MAE (mean absolute error)
The average size of the temperature error, ignoring whether it was too warm or too cool. The live page's accuracy tab uses it.
MAE skill
How much smaller our average temperature error (MAE) is than the official forecast's. Positive = better. It is used on the live page and is not the same as the MSE skill score used in the validation results here.
Lead time, "+6 h"
How many hours ahead the forecast is for.
2 m temperature
Air temperature about 6 feet above the ground: what your weather app shows.
Reanalysis
A reconstruction of past weather that combines a fixed weather model with observations. MERRA-2 is one.
Foundation model
A large AI model pretrained on lots of data that can be reused for many tasks. Prithvi WxC is one.
Frozen features
Outputs taken from a model without retraining it.
Rollout
Running a forecast model forward step by step, each step feeding the next.
CI (confidence interval)
The range the true value likely falls in. "95% CI above zero" means we're confident the office beats the official forecast.
CONUS
The contiguous United States: the lower 48 states.
12Z
12:00 UTC: 7 a.m. EST / 8 a.m. EDT.
float16
A compact way to store numbers that halves file size.
Colab, T4 GPU
Google Colab is a free cloud notebook service; a T4 is the graphics processor it provides for running AI models.
Office codes
Three-letter NWS office IDs, e.g. LWX = Sterling, VA (Washington/Baltimore), ABR = Aberdeen SD, BIS = Bismarck ND, FGF = Grand Forks ND, FSD = Sioux Falls SD, DDC = Dodge City KS.

Sources

Read the primary sources.

  1. Schmude, Roy, et al. (2024). Prithvi WxC: Foundation Model for Weather and Climate. arXiv:2409.13598.
  2. NASA/IBM. Prithvi-WxC-1.0-2300M model card and rollout checkpoint (prithvi.wxc.rollout.2300m.v1.pt, 28.4 GB), Hugging Face.
  3. NASA IMPACT. Prithvi-WxC code, GitHub.
  4. NASA GMAO. MERRA-2 overview.
  5. NASA GES DISC. MERRA-2 inst3_3d_asm_Nv (M2I3NVASM) dataset summary: resolution, 72 levels, 3-hourly, ~3-week latency.
  6. NASA Earthdata Forum. Common data access problems at the GES DISC: MERRA-2 usually available between the 20th and 25th of the next month.
  7. Gelaro, R., et al. (2017). The Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2). Journal of Climate, 30(14), 5419–5454.
  8. IBM/NASA/Jülich. Prithvi-EO-2.0 model card; Szwarcman et al. (2024), arXiv:2412.02732.
  9. Du, A., et al. (2025). First On-Orbit Demonstration of a Geospatial Foundation Model. arXiv:2512.01181.
  10. NASA Earthdata. Earthdata Login (free registration).

NWSCnn numbers on this page come from our own validation runs (Nov–Dec 2024) and are not peer reviewed. The blind-test year has not been scored.