Skip to content

ForecasterHQ is the only platform that gives indie forecasters a live accuracy score based on real NWS observation data. Here's how the scoring works and why it's changing how people evaluate weather forecasters.

0-6"
Snowfall forecast of NOV/13-14/2026, 5 observations

For decades, forecast accuracy has been the exclusive language of government agencies and research institutions. NOAA publishes annual verification statistics. The National Hurricane Center releases detailed post-season analysis of track and intensity errors. ECMWF publishes model verification scores that forecasters use to benchmark their tools.

None of that infrastructure existed for the person publishing storm forecasts for their local community on a WordPress site.

That's changing. Forecaster HQ now gives every indie forecaster something that previously only existed inside government weather offices: a live, NWS-data-backed accuracy score tied to their public forecast record.

Here's how it works - and why it's becoming one of the most important signals in indie weather forecasting.


Why Forecast Accuracy Is Hard to Measure

The intuitive definition of a "good forecast" is easy: you said it would snow 6 inches and it snowed 6 inches. But weather doesn't cooperate with simple point estimates, and a forecasting methodology that penalizes uncertainty ranges punishes honest communicators.

The practical challenges in measuring forecast accuracy:

You need a prediction made before the event. Post-hoc forecasts - explaining what happened after the storm - are common and useful, but they're not the same as a prediction with verifiable timing. Accuracy scoring requires a published forecast with a timestamp that predates the event.

You need specific, measurable predictions. "Heavy snow possible" cannot be scored. "4-8 inches in the northern zone, 1-3 inches in the southern zone" can be. Any scoring methodology requires quantified predictions at a geographic level.

You need enough observations to be statistically meaningful. A forecast that covers three counties and gets one observation from a backyard weather station isn't scoreable in any rigorous sense. You need a minimum number of independent observations spread across the predicted region before the score means anything.

You need to handle model uncertainty appropriately. A forecaster who issues a wide range (4-12 inches) and hits it is doing something different from a forecaster who issues a narrow range (5-7 inches) and hits it. Both should be scored as accurate, but the latter represents higher precision and should be recognized as such.

Forecaster HQ's verification system was built to handle all of these constraints.


How Forecaster HQ Scores Storm Forecasts

Forecaster HQ's accuracy system applies to storm forecasts - structured, event-based predictions published before a storm arrives, with snowfall accumulation ranges drawn per geographic region on a map.

The scoring flow:

Step 1: The forecast is published with a timestamp. When a Forecaster HQ forecaster publishes a storm forecast, it's locked with a creation timestamp. The regions and predicted ranges are immutable after publish - there's no editing predictions after the event.

Step 2: After the storm ends, the forecaster (or Forecaster HQ) fetches NWS observation data. Forecaster HQ connects to NWS weather station observation data and IEM (Iowa Environmental Mesonet) Local Storm Reports. For a storm forecast over New England, this means pulling actual snowfall measurements from every available NWS cooperative observer station, COOP station, and official observer within the forecast domain.

Step 3: Observations are mapped to the forecasted regions. Each observation point is matched to the geographic region it falls within. If a forecaster drew 8 regions on their storm forecast map, every observation gets assigned to the region that contains it.

Step 4: A minimum observation threshold filters noise. Regions with fewer than 5 independent observations are marked "pending" - there isn't enough data to make a meaningful accuracy call. Only regions with 5 or more observations enter the scoring calculation.

Step 5: Each region is scored verified or missed. For regions that pass the observation threshold:

  • Verified: At least one observation falls within the predicted accumulation range
  • Missed: All observations fall outside the predicted range (either above the max or below the min)

Step 6: The overall forecast is scored. The accuracy score reflects the percentage of scoreable regions that were verified. A forecast with 6 scoreable regions where 5 were verified gets an 83% score. A forecast where all 8 regions were verified gets 100%.


The Minimum 5 Observation Rule: Why It Exists

The 5-observation minimum is the most important design decision in the system, and it deserves an explanation.

Snowfall observations are noisy. A single station can read significantly different from a station three miles away, depending on micro-terrain, elevation, snow board placement, and measurement timing. A region "hit" by one observation in a dense urban area and "missed" by a second observation in a rural township could go either way depending on which reading you trust.

Five observations per region doesn't guarantee accuracy, but it makes gaming the system significantly harder. It also filters out under-observed regions - a forecaster who draws 12 tiny regions over rural terrain where NWS has sparse coverage would have most of those regions come back "pending" rather than scoring inaccurately.

The tradeoff: some legitimate forecasts don't get fully scored because the observation network isn't dense enough in the forecast area. This happens most often in rural western states where NWS cooperative observer networks are sparse. It's a known limitation, and Forecaster HQ displays it transparently - pending regions are labeled as pending, not counted as misses.


What Gets Scored vs. What Stays Pending

Not every storm forecast generates a full accuracy score. Three conditions affect scoreability:

1. Observation network density. Urban and suburban areas in the Northeast, Midwest, and Pacific Northwest have dense enough NWS observation networks to score most storm forecasts fully. Rural areas in the Great Plains, Mountain West, and parts of the South often return more pending regions than verified/missed ones.

2. Storm timing. IEM Local Storm Reports (informal but dense observations) are available in near-real-time but may take 24-48 hours to fully populate. If verification data is fetched too quickly after a storm, regions that would have been verified may still be pending. Forecaster HQ recommends waiting at least 48 hours post-storm before fetching verification data.

3. Forecast region size. Small, tightly-drawn regions are harder to score than broad regional polygons. A forecaster who draws 15 small county-level regions will have more pending regions than one who draws 5 regional zones, because each small region needs 5 independent observations and small regions often have fewer stations within their boundaries.


How the Score Appears on a Forecaster's Public Profile

Once verification is complete, the accuracy score becomes part of a forecaster's public record on Forecaster HQ.

Each past storm forecast shows:

  • Whether it was verified or missed, per region
  • A strip plot visualization showing the predicted range as a band and each actual observation as a dot on a number line
  • The overall accuracy score for that forecast
  • The observation count per region

A forecaster's profile aggregates these individual forecast scores into a track record:

  • Total forecasts published
  • Percentage of forecasts that were scored (vs. pending)
  • Verification percentage across all scored regions

The track record is public, timestamped, and tied to real NWS data. It's not a self-reported claim. It's an independently verifiable record.


Why This Changes the Trust Equation

Before verification scoring existed, the standard way to evaluate an indie forecaster was social proof: follower counts, how often they were right in memorable events, testimonials from readers. Those signals are real, but they're manipulable and they don't survive the news cycle.

An accuracy score tied to NWS observations is different in a few important ways:

It's pre-committed. The prediction goes in before the storm. There's no retroactive editing. The forecaster either called it or didn't.

It's third-party-sourced. NWS observation data comes from a federal network that has no stake in any forecaster's reputation. The data is public, the methodology is documented, and anyone can pull the same station readings to double-check the score.

It compounds over time. A single verified forecast is nice. A verified track record spanning two hurricane seasons and 30+ storm forecasts is the weather equivalent of the CSU archive - accountability that becomes authority.

Readers who are evaluating whether to subscribe to a forecaster's newsletter or pay for a premium tier now have something concrete to anchor on. Not "I've heard good things" but "this forecaster has been scored against NWS data on 18 storm forecasts and has an 89% verified rate."


Building Your Own Accuracy Score

For indie forecasters who aren't yet on Forecaster HQ, the path to a verified track record starts with publishing structured storm forecasts before events - with specific regional predictions, quantified accumulation ranges, and a clear timestamp.

The verification feature on Forecaster HQ handles the scoring automatically once a storm ends. There's no manual data entry, no spreadsheets, no post-hoc calculations. The forecaster publishes, the storm happens, the data gets fetched, and the score appears.

For forecasters who want to understand the methodology in detail before publishing their first scored forecast, the Forecaster HQ storm forecast verification tool article covers the technical flow end-to-end.

For forecasters building toward a long-term public track record, how to build a public forecast track record covers the strategic framing - why the archive matters, how long it takes to build meaningful credibility, and what to do during the first season when the sample size is small.


The Honest Limitations

Forecaster HQ's accuracy scoring is real and meaningful, but it's not perfect, and overstating it would undermine the whole point of having a verification system in the first place.

Binary pass/fail has limitations. The current system scores each region as verified or missed - there's no partial credit for a prediction that was close but outside the range. A 3-6 inch forecast where the event came in at 7 inches is scored as a miss even though the forecaster clearly had the right magnitude and only missed the upper bound.

NWS observation networks have gaps. Sparse observation coverage in rural areas is a real constraint. Some storm forecast domains simply won't generate enough observations for a fully scored result.

Accumulation only. The current system scores snowfall accumulation. Wind speed, ice accumulation, and timing accuracy aren't scored in v1. These features are on the roadmap.

None of these limitations make the score less useful. They make it honest. The goal isn't a perfect number - it's a transparent, pre-committed record that readers can evaluate with appropriate context.

That's what forecast accountability actually looks like.


Want to see your accuracy score? Start publishing - it's free โ†’

Publish your own storm map, and see the observations land on it

Draw the regions, set the ranges, publish. When the storm is over, the observations are drawn inside your regions.

Publish your forecast