core_data_updater.jobs.metadata

jobs/metadata.py — MetadataFlow: the ROOT island (everything reads its tables).

Design record: METADATA_FLOW.md (pre-stamped from the records; §7 decisions taken per recommendation, 2026-08-16). Shape: DAILY metadata_daily the roster heartbeat: universes (land_universe + stamp_tradable) · bounded SEC CIK resolution · catalogs · constituent diffs — cron 00:15 UTC Tue-Sat, born STOPPED WEEKLY metadata_weekly venues · calendar re-union · SEC/info enrichers — cron Sat 06:00 UTC, born STOPPED BUILD metadata_build the genesis run that unblocks prices + events — manual, max_concurrent=1 RECONCILE metadata_reconcile the rename hole, closed off events’ landed symbol_changes — fired by the AFTER gate on that asset ALONE, born STOPPED BACKFILL metadata_backfill the three enrichment holes — cron 10:00 UTC, born STOPPED

Laws carried here: configs read LIVE per step (per-step rows, §7-D1; enabled comes back VISIBLE); every landing through CoreDataRepo — land() on the METADATA policies (NULL_KEY drops undated windows/null-cik names, counted), land_universe for the incarnation tables (three-strata resolution, the transition rule, ABSENCE-NEVER-FLIPS, half-assembly refusal), the merge verbs for enrichment (entity fill-never-overwrite · etf-info fresh-fact-wins on the ACTIVE row) and mutate_windows for atomic close+open membership changes — this file carries ZERO SQL, ZERO wire parsing, ZERO conversion. State-as-params: stored open windows, CIK maps and rosters are READ from raw tables and passed to feed verbs. Anchor-failure ≠ no-changes: a None diff skips that index NAMED and closes NOTHING. Cron-primary by design (§5.4) — the root island has no grounded probe; downstream sensors compose off these asset keys instead.

Classes

MetadataFlow

Bases: CoreDataFlow

The metadata flow — METADATA_FLOW.md §5, method for method.

backfill_entity_info

backfill_entity_info(cfg) -> None

The EDGAR hole: ACTIVE equities carrying a cik but NO sic.

backfill_etf_holdings

backfill_etf_holdings(cfg) -> None

The N-PORT hole: ACTIVE etfs with NO holdings rows at all (the absent-rows arm of enrichment_debt — the child table IS the fact).

backfill_etf_info

backfill_etf_info(cfg) -> None

The etf/info hole: ACTIVE etfs with no etf_company fact landed.

build_calendar

build_calendar(cfg) -> None

build_constituents

build_constituents(cfg) -> None

Full event-history windows (GSPC → 1957) — replace scoped to the SERVED index codes (§7-D2: stale-free rebuilds). The stored open windows are passed into the feed as a destructive-write baseline: a 200-empty/truncated current anchor fails before exact replacement.

build_crypto_catalog

build_crypto_catalog(cfg) -> None

build_entity_info

build_entity_info(cfg) -> None

build_equity_ciks

build_equity_ciks(cfg) -> None

build_etf_ciks

build_etf_ciks(cfg) -> None

build_etf_holdings

build_etf_holdings(cfg) -> None

build_etf_info

build_etf_info(cfg) -> None

build_exchanges

build_exchanges(cfg) -> None

build_index_catalog

build_index_catalog(cfg) -> None

build_universe_equities

build_universe_equities(cfg) -> None

build_universe_etfs

build_universe_etfs(cfg) -> None

reconcile_renames

reconcile_renames(cfg) -> None

Close the OLD incarnation of every CORROBORATED rename — the ABSENCE-NEVER-FLIPS hole (§4, §9): the roster assembly can only see that FB stopped appearing, and absence must never flip a row, so the dead ticker stays is_active forever. events’ landed raw.symbol_changes is the witness; close_renamed_incarnations carries the composed UPDATE and both guards (already-inactive rows untouched · no new_symbol row = no close).

Reconciling off a STALE rename stream is the one way this step can be wrong — which is why it has its OWN job (metadata_reconcile, 2026- 08-17), fired by the AFTER gate on the symbol_changes asset and by nothing else. It used to ride metadata_backfill, and that dragged the three enrichment legs along on every rename landing: only THIS step needs a fresh symbol_changes.

refresh_calendar

refresh_calendar(cfg) -> None

update_constituents

update_constituents(cfg) -> None

Stored open windows → fresh anchors → atomically close+open.

Each index’s departure instructions and new-window feed rows enter one repository mutation. Its SQL statement is the transaction boundary: an opening failure rolls back the closes for that index. The feed receipt still drives fatal/gap verdicts and the opening/closing counts retain their distinct monitoring stages.

A None diff = the anchor FAILED: that index skips NAMED and closes NOTHING (vendor down ≠ everyone left the index).

update_crypto_catalog

update_crypto_catalog(cfg) -> None

update_entity_info

update_entity_info(cfg) -> None

update_equity_ciks

update_equity_ciks(cfg) -> None

update_etf_ciks

update_etf_ciks(cfg) -> None

update_etf_holdings

update_etf_holdings(cfg) -> None

update_etf_info

update_etf_info(cfg) -> None

update_exchanges

update_exchanges(cfg) -> None

update_index_catalog

update_index_catalog(cfg) -> None

update_universe_equities

update_universe_equities(cfg) -> None

update_universe_etfs

update_universe_etfs(cfg) -> None