From filings to forecasts.
The v1 ingestion pipeline we are building: six stages from a document appearing on a regulator's website to an updated assumption in a capacity expansion model. This page describes the current design; components will change as prototypes are tested.
01Watch
Headless-browser watchers poll ERCOT's monthly interconnection queue report, protocol revision requests, PUCT filings, and Texas Legislature bill activity. Moratoria, datacenter consumption taxes, and market rule changes appear first as documents. Change detection wakes the rest of the pipeline only when a document is new or revised.
02Parse
Filings arrive as layout-heavy PDFs, and many are scans. A parsing layer converts each document to structured text and tables, with a vision-language model handling pages that conventional OCR reads poorly.
03Extract
An open-weight language model, running on our own GPUs, reads each parsed document and emits records against a fixed schema: project name, capacity, county, point of interconnection, queue status, projected commercial operation date. For policy documents, the extraction maps each change to the model parameters it affects. Constrained decoding means the model can only produce valid JSON; the schema is enforced during generation.
04Validate
Every record is validated against typed schemas, cross-checked against ERCOT's settlement-point registry and the prior state of the queue, and scored for confidence. Low-confidence extractions go to a human review queue instead of into the database.
05Store
Assumptions live in a versioned database. Each value carries provenance: the source document, the page, and the extraction that produced it, so a price scenario can be traced back to the filing that changed it. When the queue report revises a project, the history of that project's assumptions is preserved.
06Model
The structured assumptions export directly into capacity expansion modeling: buildout, retirements, load growth, and policy parameters. The expansion model produces the long-run price scenarios our siting and valuation work consumes, and a change in a bill or filing can be traced through to its effect on a price scenario.
The target schema.
A mock extraction record. Values are illustrative, not from a real filing.
$ pipeline extract --source gis-report-2026-06.pdf --page 41 { "project": "Example Solar + Storage", "county": "Pecos", "capacity_mw": { "solar": 240, "storage": 200 }, "poi_kv": 138, "queue_status": "IA executed", "projected_cod": "2028-Q2", "source": { "doc": "gis-report-2026-06.pdf", "page": 41 }, "confidence": 0.93 }
Training the extraction models.
The extraction models are fine-tuned for this document domain: a frontier model labels a corpus of historical filings, and that labeled set trains smaller open-weight models that run locally. The result is extraction that runs privately and is cheap enough to re-run across the full document history when the schema changes. GPU compute is the binding constraint on this work: training, fine-tuning, and batch extraction all run on local hardware.
The full stackWorking on the same problem?
We're a small shop and we answer our own email.