Source file DATA-AUDIT.md · last changed 2026-10-15 · history

Earth Desk — Data Audit

Verified 2026-10-14. Every endpoint below was checked; anything unverified says so. Tiers: LIVE (daily or faster) · PERIODIC (weekly/monthly) · ANNUAL · STATIC (one-off, dated snapshot) · CLOSED (exists but no open access)

The site shows the tier on every panel. A STATIC panel says "as of "; a LIVE panel shows last-fetch time.


1. Commodity meters (reserves vs. consumption)

Feed Endpoint Format Auth Cadence License Tier
USGS MCS 2026 ScienceBase item 69837e43b66b01367d7ec7c7, file MCS2026_Commodities_Data.csv; DOI 10.5066/P1WKQ63T CSV none Annual, Feb CC0 ANNUAL
Energy Institute Review 2026 energyinst.org/statistical-review/resources-and-data-downloads (2026 XLSX is form-gated; 2025 direct link confirmed) XLSX email form Annual, June attribution; redistribution needs permission ANNUAL
EIA API v2 api.eia.gov/v2/{petroleum/sum, petroleum/cons, natural-gas/cons, coal/consumption-and-quality, international, steo}/data JSON free key (register) weekly (WPSR Wed 10:30 ET) / monthly public LIVE (US) / PERIODIC (world)
OWID grapher ourworldindata.org/grapher/{slug}.csv?csvType=full&useColumnShortNames=true CSV none mirrors upstream CC BY 4.0 ANNUAL
IEA Critical Minerals dataset iea.org/data-and-statistics/data-product/critical-minerals-dataset XLSX free IEA login annual CC BY 4.0 ANNUAL (login)
Prices — IMF PCPS imf.org/-/media/files/research/commodityprices/monthly/external-data.xlsx XLSX none monthly cite IMF PERIODIC
Prices — FRED PCOPPUSDM (copper); siblings for silver/cobalt unconfirmed JSON free key monthly FRED terms PERIODIC
Prices — World Bank Pink Sheet thedocs.worldbank.org/.../CMO-Historical-Data-Monthly.xlsx (path segment changes yearly) XLSX none monthly, ~2nd business day CC BY 4.0 PERIODIC
Data center energy/water IEA Energy & AI 2025 (XLSX, login); LBNL 2024 US report (PDF) PDF/XLSX login / none one-off CC BY / public STATIC

Decisions - Meter arithmetic: reserve (USGS, annual) minus cumulative consumption since the reserve date, interpolated daily from the last annual production figure. Label: "Interpolated from USGS MCS 2026; reserves as of Jan 2026; production rate = 2025 estimate." - Fossil fuels: EI reserves are year-end 2020 and have not been refreshed. Meter shows "2020 reserves ÷ latest EIA consumption" and says so in the panel. - Lithium has no free daily or monthly price feed (not in IMF PCPS or Pink Sheet). Lithium meter carries volume only, no price. - Data center meter is STATIC: LBNL 176 TWh (4.4% of US, 2023); 325–580 TWh by 2028. Water: no public dataset. Panel shows the model and its date; no fake daily tick. - EI license: cite with attribution; do not mirror the XLSX in the repo.

2. Water

Feed Endpoint Format Auth Cadence Tier
Lake Mead / Powell usbr.gov/uc/water/hydrodata/reservoir_data/{921 Mead | 919 Powell}/{csv|json}/{17 storage AF | 49 elevation ft | 42 release}.csv (full period of record; take tail) CSV/JSON none daily LIVE
USGS Water Data API api.waterdata.usgs.gov/ogcapi/v0/collections/{latest-continuous, daily, field-measurements, monitoring-locations} GeoJSON free key real-time/daily LIVE
Water Quality Portal waterqualitydata.us/data/Result/search?characteristicName=Nitrate;Arsenic;...&siteType=Well&statecode=...&mimeType=csv CSV none nightly LIVE
USGS national PFAS in groundwater ScienceBase 64d167cbd34ef477cf3c0266, DOI 10.5066/P93RXTKJ CSV none 2024 one-off STATIC
US Drought Monitor usdmdataservices.unl.edu/api/USStatistics/GetDroughtSeverityStatisticsByAreaPercent?aoi=conus&startdate=&enddate=&statisticsType=1 CSV/JSON none weekly (Thu) PERIODIC
NASA GRACE-DA drought indicators nasagrace.unl.edu (weekly folders; PNG + binary grids; drought.gov XYZ tiles) grids/tiles none weekly PERIODIC
JPL GRACE mascon TWS (global) PO.DAAC TELLUS_GRAC-GRFO_MASCON_CRI_GRID_RL06.3_V4, DOI 10.5067/TEMSC-3JC634 NetCDF 0.5° free Earthdata login monthly, ~3-mo lag PERIODIC
FAO AQUASTAT dashboard export only; World Bank API mirrors ER.H2O.FWST.ZS, ER.H2O.FWTL.ZS CSV/JSON none annual; many values are long-term averages ANNUAL
WRI Aqueduct 4.0 water stress files.wri.org/aqueduct/aqueduct-4-0-water-risk-data.zip GPKG/CSV none 2023 one-off STATIC
Jasechko 2024 global well trends zenodo.org/records/10003697 (1,693 aquifers; 66% of records released) CSV none one-off STATIC

Decisions - Legacy waterservices.usgs.gov is being retired Nov 2026–Feb 2027. Build on api.waterdata.usgs.gov from day one. - "Clean vs polluted aquifers": no national percent-exceedance feed exists. Compute it from WQP (nitrate >10 mg/L, arsenic >10 µg/L) per state, nightly, and show the sample count so a thin state is visibly thin. PFAS from the 2024 USGS static layer. - "Fresh available" globally: Aqueduct 4.0 water-stress polygons (static 2023) plus World Bank/AQUASTAT withdrawal-as-%-of-renewable (annual). Global GRACE TWS anomaly computed from the JPL mascon grid monthly. - US gets the live tier; the rest of the world gets annual and static tiers with that stated.

3. Oceans

Feed Endpoint Format Auth Cadence Tier
NOAA NCEI Marine Microplastics ncei.noaa.gov/products/microplastics; archive DOI 10.25921/d1bs-3m40 (revised 2026-10-06) CSV/GeoJSON none as-needed PERIODIC
Eriksen 2023 global trend 1979–2019 github.com/wincowgerDEV/ocean_plastic_modeling (LocationsCleanedWithWind.csv, 11,777 stations) CSV none one-off STATIC
Baleen whale ingestion (Kahane-Rapport 2022) github.com/shirelkr/risk-of-microplastic-ingestion-by-filter-feeding-megafauna XLSX none one-off STATIC
Mussel contamination Cefas UK 2022 (data.gov.uk); NCEI accession 0290836 (SF Bay 2018) CSV none one-off STATIC
IAEA MARIS (Cs-137 etc.) maris.iaea.org (JS app); CKAN mirror data.iaea.org/datastore has a working datastore API per dataset NetCDF / CSV none irregular PERIODIC
WHOI Our Radioactive Ocean ourradioactiveocean.org — dormant since 2019 HTML none dead STATIC (baseline)
TEPCO / MOE ALPS monitoring tepco.co.jp/en/decommission/progress/watertreatment/monitoring; shorisui-monitoring.env.go.jp/en PDF / graphs none ~daily (TEPCO) LIVE but no CSV
NOAA Coral Reef Watch DHW coastwatch.pfeg.noaa.gov/erddap/griddap/NOAA_DHW.csv?... ERDDAP none daily LIVE
NOAA OISST SST coastwatch.pfeg.noaa.gov/erddap/griddap/ncdcOisst21Agg.json?sst[(last)]... ERDDAP none daily LIVE
Copernicus Marine BGC (pH, O2) GLOBAL_ANALYSISFORECAST_BGC_001_028 NetCDF free registration daily LIVE (login)
GPGP extent (Lebreton 2018) figshare 5873142 (GPGP_contours.shp, 10/100/1000 g/km²) SHP/GeoTIFF none 2015–16 snapshot STATIC
River plastic inputs (Meijer 2021) figshare 14515590 (31,819 outfalls, t/yr); OWID plastic-entering-ocean has country totals SHP / CSV none one-off STATIC

Decisions - Microplastics: a dated point map, not a global index. NCEI itself warns concentrations are not comparable across methods. Panel says so. - Filter-feeder "percent of diet" does not exist as a metric anywhere. The honest panel is per-species ingestion estimates with their study, region and date (e.g., blue whale ~1.1e7 pieces/day, California Current, 2022). Not synthesized into a percentage. - Radioactivity: MARIS point samples (dated) plus WHOI 2014–2018 baseline. Fukushima tritium is published daily by TEPCO but only as PDFs; scrape or transcribe manually with the date shown, or omit from the daily job. - SST and heat stress are the cleanest live ocean feeds and go in first.

4. Air

Feed Endpoint Format Auth Cadence Tier
OpenAQ v3 api.openaq.org/v3/locations, /locations/{id}/latest, /countries JSON free key (X-API-Key), 60/min, 2,000/hr near-real-time LIVE
WAQI api.waqi.info/feed/{city}/?token= + map-bounds JSON free token real-time LIVE — license risk
NASA GIBS WMTS gibs.earthdata.nasa.gov/wmts/epsg4326/best/{Layer}/default/{date}/{set}/{z}/{row}/{col}.png (AOD layers confirmed; NO2 layer ID to verify) tiles none daily LIVE
NOAA Mauna Loa CO2 gml.noaa.gov/webdata/ccgg/trends/co2/co2_daily_mlo.csv (# comment header) CSV none daily LIVE
NOAA global CH4 gml.noaa.gov/webdata/ccgg/trends/ch4/ch4_mm_gl.csv CSV none monthly PERIODIC
IQAir World AQ Report PDF + Tableau only, no CSV (2025: 9,446 cities, 143 countries) PDF none annual ANNUAL (manual)

Decisions - OpenAQ is the backbone (license per location, exposed in the API; attribute each). WAQI's terms require a written agreement for for-profit public use; skip it or get the agreement first. - Heaviest vs lightest: compute from OpenAQ latest PM2.5 by country/city at refresh time; show station counts so sparse countries aren't misread. - Key stays server-side in the refresh job.

5. Cleanup

Feed Endpoint Format Auth Cadence Tier
IBI Global Biochar Market Report 2025 biochar-international.org (exec summary PDF free; full $400) — 520,810 t biochar 2025 vs 180,150 t 2023 PDF none annual ANNUAL
Biochar Europe Market Report 2025/26 biochareurope.eu — 205 plants, 361 kt/a capacity end-2025 PDF none annual ANNUAL
Puro.earth registry registry.puro.earth/issuances (public table, filter Biochar; 595 issuances); API partner-only HTML none (UI) continuous LIVE UI / CLOSED API
CDR.fyi api.cdr.fyi/v1 — partner access only JSON closed continuous CLOSED
OWID plastics (OECD) ourworldindata.org/grapher/global-plastics-production.csv, plastic-waste-mismanaged.csv, share-plastic-waste-recycled.csv CSV none base year 2019 STATIC
Chemical recycling / pyrolysis capacity Fraunhofer UMSICHT map s.fhg.de/crmap — Europe: 18 plants operational, 289 kt/a; 2,799 kt/a planned (Oct 2025) HTML none periodic PERIODIC (manual)
Global Forest Watch alerts POST data-api.globalforestwatch.org/dataset/gfw_integrated_alerts/latest/query/json JSON free key, domain-bound daily LIVE

Decisions - Biochar: no open time series. Scorecard = IBI annual production + Biochar Europe plant count + Puro public issuance count (scraped, dated). CO2 offset derived from tonnage only with the stable-carbon caveat (process temperature dependent). - Plastic pyrolysis capacity: nothing structured exists globally. Europe from Fraunhofer; US/Asia hand-curated from company disclosures (Agilyx, Plastic Energy, Brightmark, Plastic 2 Green) with source links. This is the section where Randy's own technical commentary carries the weight. - Plastics flow numbers are anchored at 2019 (OECD base). Label it.


What the globe can honestly show on day one

Live layers: US reservoir levels, US drought, US groundwater sites, US air (OpenAQ global where stations exist), global SST and coral heat stress, Mauna Loa CO2, tropical deforestation alerts, NASA aerosol tiles.

Periodic: US drought, GRACE groundwater (US weekly, global monthly), methane, commodity prices.

Annual/interpolated: every commodity meter, fossil fuel R/P, water stress, biochar.

Static with date: microplastics points, radioactivity points, GPGP extent, river plastic inputs, PFAS wells, data center energy, plastics flows (2019).

Not shown as numbers: filter-feeder "percent of diet"; global aquifer pollution percent; data center water; pyrolysis capacity outside Europe (shown as a curated list with sources instead).


Keys the refresh job needs

EIA (free), OpenAQ (free), USGS Water Data (free), GFW (free, domain-bound), NASA Earthdata (free, for GRACE), Copernicus Marine (free, optional for pH), FRED (free, optional).