Reality check

In layers. Reported regional users and revenue anchors are imposed by construction. Behaviour needs matching definitions and checks separate from calibration. Feature effects have no fair held-out test. Rival launches are what-ifs.

The public-target counts load from the saved scorecard.

Only the Google ranking card (E07), the ad-budget shock card (E10) and Pinterest's own ad-load card (E09) rest on something measured or said about Pinterest. For every other rival launch, no published measurement of its effect on Pinterest was found.

This page shows the public record. None of the simulated results is a forecast.

Loading the public record.

What the engine can be checked against

Matching an input is calibration. Comparing an unused output needs the same definition, population and period, or an explicit assumption that makes the gap acceptable.

What we did about the gaps tonight

The engine now exports quantities that can be put next to public figures. Two pre-registered checks miss: app daily-to-monthly users is 0.21 against a 0.24 to 0.31 reference band, and saves per monthly user are 6.3 a month against at least 10.2 if Pinterest's 2023 weekly save figure still holds. Both rest on unresolved definitions, so neither counts as an independent test.

We also held out each of the four public calibration targets in turn and selected worlds on the other three. With each acceptance gate withheld, three target medians stayed within tolerance: weekly to monthly users, US and Canada growth, and global growth. Outbound clicks per user missed (1.66 against 2.7, plus or minus 0.9). These targets informed model development earlier, so this tests how much world selection depends on each gate, not validity on new data; method in research/25-leave-one-out.md. A backtest against past quarters found rest-of-world users growing too slowly in the engine. A proposed regional growth prior, fitted only to figures published before each start date, cuts the typical miss from 3.8% to 1.6%; a simple seasonal rule still does better at 1.1%. The prior is opt-in until approved (research/24-engine-backtest-fix.md). A full 192-world batch with it passes a pre-stated promotion rule: every kept world meets the original checks, regional growth reaches its targets, rankings within each scenario group barely move (Kendall tau 0.96 to 1.00), only 3 of 402 headline readings flip sign, all within noise, and the three bets keep their readings (web/batch-compare.json, docs/promotion-runbook.md).

We then raised activity until the engine reached those levels and reran all 67 scenarios and the three registered bet checks under that one change. The three decision readings on Bets did not change. Some feature and outside-event rankings did, and seven readings near zero flipped sign, so single readings near zero should not be trusted. A live audit of 360 Pinterest results is on Bets too. Method: research/21-grounding-round.md and research/23-reality-gap-sensitivity.md.

The saved scorecard separates reconciled totals, calibration, prior evidence and independent comparisons. Read the data and source links. The existing tests below answer different questions.

Tests against what really happened.

These ask different questions. A replay checks a historical story. Loose experiment comparisons check model responses. The ledger checks reported totals. None establishes that a feature effect is right.

Loading the test files.

Method and limits

Dates and quotes were collected from public sources on 2026-10-12. Some launch dates were confirmed from dated search results rather than by reading the full page. Medium and low confidence flag those limits and weaker sources. Many earnings-call quotes come from a third-party transcript site; the link beside each quote identifies its source. Year-on-year figures use Pinterest's reported quarterly monthly users.

The before figure is from the last full quarter before the event. The after figure is from four quarters after the event's quarter. Those comparisons do not measure an effect. Regional figures here are reported totals. Any country or state figures elsewhere in the simulator are allocations, not observed counts.