Skip to content
core-data-updater

core_data_updater.jobs.fundamentals

jobs/fundamentals.py — FundamentalsFlow: statements · filings · documents.

Five census tables off ONE clock that nothing else in this container shares: a company files when it files. There is no session, no partition schedule, no publication calendar — so this flow is EVENT-DRIVEN end to end:

UPDATE  fundamentals_update  the FILING SENSOR back-walks completed EDGAR
                             indexes from its durable cursor; the job pulls
                             the spine + recent periods for exactly the hit
                             CIKs/symbols
UPDATE  fundamentals_filings the filings for the whole roster, nightly —
                             the catch-up for anything the sensor missed
DOCUMENTS fundamentals_documents after every new filings product; drains
                             bounded pointer debt when save_documents=true
BUILD   fundamentals_build   full statement history over the stored roster
                             — manual, max_concurrent=1, replace scoped
BACKFILL fundamentals_backfill  nightly re-pull of the recent periods that
                             drift until filed — 11:00 UTC, born STOPPED

Live receipts (2026-08-17, real calls): income/balance/cash AAPL+XOM+JPM × 8 quarters → 24 rows each, and the FULL field-map audit came back clean: 31/31, 53/53, 39/39 mapped fields populated. No rename. get_filings 76 rows across 3 CIKs (10-K + 10-Q) filing_events 992 (Fri 08-14) · 895 (Thu) · 630 (Wed) form entries filing_events weekend 0 rows, ok=True — EDGAR answers 403 and that is a CORRECT empty, so the sensor survives Saturdays get_filing_document 15,170 bytes for AAPL’s 0000320193-25-000073

WHY THE SENSOR IS THE PRIMARY TRIGGER. Statements are the one stream where a calendar is actively misleading: a company can file Q3 in October or in January, restate in March, and file an 8-K any morning. Polling every symbol daily would burn the whole FMP budget re-reading unchanged history. So the sensor reads ONE cheap artifact — EDGAR’s daily form index — filters it to our CIK set, and the job pulls only for companies that actually filed.

Laws carried here, identical to the other four flows: configs read LIVE per step; every landing through CoreDataRepo.land on the FUNDAMENTALS policies; state-as-params (the CIK map is READ from raw.equities and PASSED — the feed never queries); the run ledger opens idempotently; fulfill-or-name; the FATAL law in _land; and ZERO SQL, wire-parsing, or conversion in this file.

Classes

FundamentalsFlow

Bases: CoreDataFlow

The fundamentals flow — FUNDAMENTALS_FLOW.md §5, method for method.

backfill_balance_sheets

backfill_balance_sheets(cfg, fetch_ciks: dict) -> None

backfill_cash_flows

backfill_cash_flows(cfg, fetch_ciks: dict) -> None

backfill_income_statements

backfill_income_statements(cfg, fetch_ciks: dict) -> None

build_balance_sheets

build_balance_sheets(cfg, fetch_ciks: dict) -> None

build_cash_flows

build_cash_flows(cfg, fetch_ciks: dict) -> None

build_filings

build_filings(cfg, fetch_ciks: dict) -> None

The whole spine over the stored roster, all configured forms.

build_income_statements

build_income_statements(cfg, fetch_ciks: dict) -> None

fetch_ciks

fetch_ciks(cfg) -> dict

The CIK roster from raw.equities, read ONCE and PASSED.

THE CIK IS THE WHOLE FLOW’S KEY. EDGAR knows nothing about tickers: the filing spine, the daily form index, and every document path are CIK- addressed, so a symbol without one is invisible to this flow entirely. Returned as {"ciks": [(symbol, cik), ...], "active": {...}}: all-history preserves duplicate-symbol legal-entity incarnations for SEC, while active stays the unique map the sensor needs. Statement legs de-duplicate the all-history list back to vendor ticker calls.

A FAILED read RAISES — a metadata outage must block loudly rather than land a green run that filed nothing.

filings_ready

filings_ready(ctx, upstream_done)

THE FILING SENSOR — fires when one of OUR companies actually filed.

Reads cheap completed Eastern-time EDGAR daily form indexes from the durable sensor cursor through today, filtered to our CIK set. A hit means a company we track filed, so only those exact CIKs and de-duplicated symbols are handed to the job in run tags. No hit means nothing changed and the job does not run — polling 20,000 symbols to re-read unchanged history would burn the FMP budget for nothing.

WEEKENDS AND HOLIDAYS: EDGAR publishes no index and answers 403. The feed returns a CORRECT empty (no gap, ok=True) — so this reads as ‘nobody filed’, which is exactly true, and the sensor stays alive instead of dying every Saturday (the 2026-08-17 fix).

The requires gate confirms equities has been bootstrapped. The sensor then reads the current active CIK map; the gate itself is not an asset-age/freshness assertion. The run key includes an accession-set digest, so a late addition to the same index date is not deduplicated behind an earlier tick.

setup

setup(cfg: Any) -> None

Build the runtime and stamp sensor-targeted filing work.

update_balance_sheets

update_balance_sheets(cfg, fetch_ciks: dict) -> None

update_cash_flows

update_cash_flows(cfg, fetch_ciks: dict) -> None

update_documents

update_documents(cfg) -> None

Fetch + store the primary document bytes for spine rows that have none yet, then STAMP the pointer on the spine.

OFF by default (save_documents): documents are the expensive leg and most consumers only need the spine. The debt read is the repo’s — enrichment_debt("filings", column="doc_location") — so this job never writes SQL to find what is missing.

update_filings

update_filings(cfg, fetch_ciks: dict) -> None

The filing spine per company, from EDGAR’s submissions feed.

The policy COALESCES doc_location: the spine lands with the pointer NULL and the document leg stamps it later, so a spine re-run can never null out a pointer that already exists.

update_income_statements

update_income_statements(cfg, fetch_ciks: dict) -> None

The increment — recent periods only (8 = two years of quarters).