Skip to main content

RSMS results — Azure Table Storage + JSON blobs

Canonical path: rsms-worker-function runs the engine → rsms_results_storage upserts PlumeByTime / PlumeByMile (numpy.float32 payloads, mile_step from SSOT — 0.2 mi) and uploads minified edge_timeseries.json, mass_balance.json to RSMS_RESULTS_BLOB_CONTAINER. rsms-api-fastapi/app/results reads tables + blobs; notebook prototypes are out of scope for production uploads.

Sudhir validation artifacts (if present in repo clones) remain reference-only coding guides — not an execution prerequisite.


Contracts (design lock)

Aligned with ssot/rsms-ssot.json, rsms-results-storage-design.md:

AreaLocked
Partition keys, row keys, table namesSSOT azureTableStorage + app/results/constants.py
time_index ↔ hours, mile_index ↔ milesmile_step = 0.2 matching implementation
JSON blobsedge_timeseries.json, mass_balance.json prefixes {riverbasin_id}/{scenario_id}/

Writer pipeline (worker-owned)

Implement rsms-worker-function/rsms_results_storage/ — entrypoint write_scenario_results_to_azure (parse .TIM / .PLT / .MAS, interpolate to mile_step, upsert PlumeByTime / PlumeByMile, emit edge + mass JSON blobs). concentration_tolerance from the scenario entity drives edge thresholds.

Exit: Successful worker run → API GET …/cxplt/by-hour / …/mass-balance usable for that scenario ID.


Worker — engine integration

Operational checklist: rsms-worker-implementation-plan (W1–W4 history + operations).

Runtime: engine_workflow loads RiverMileIndex.csv from the riverbasin blob container (not notebooks).


Scenario persistence (two JSON bags)

Implemented (app/scenario_bags.py): Azure Scenarios entity stores spill_definition + simulation_parameters JSON strings alongside status; legacy flat scalar columns remain readable for migration.


Supplementary backlog

Anything still open belongs in rsms-master-tasklist.md.

TopicNotes
Generate NFQ from CTL (notebooks)Historical reference onlynfq_from_api.py + worker tests are authoritative; production does not run Jupyter.

API — reads (results router)

Routes under /riverbasins/{riverbasin_id}/scenarios/{scenario_id}/resultsapp/results/router.py, service.py.

Open API follow-ups: see rsms-master-tasklist.md ( edge-profile, optional perf tuning).


Cleanup & frontend

Historical note: stray Parquet-helper scripts and UX copy implying Parquet uploads have been cleared; meta.dense_grid semantics remain.

Any new cleanup belongs in rsms-master-tasklist.md unless it requires an SSOT meta.schemaVersion bump.


Azure ops (open checklist)

Operational items (Table provisioning/RBAC/App Settings parity, measured P95 read latency vs Sudhir ~0.13 s benchmark, optional caching) are tracked only in rsms-master-tasklist.md so they do not drift across multiple canonical lists.


Risk register (still relevant)

RiskMitigation
Dense grid (zeros are real)API meta.dense_grid; charts treat tails as zeros
edge_timeseries.json semantics driftBlob schema version (schema_version / kind) asserted in writers + parsers

Removed: dual-write Parquet migration (complete) — see archive/README.md only for archaeology.


Dependencies

Document
rsms-results-storage-design
rsms-master-tasklist
rsms-worker-implementation-plan
archive — superseded Parquet/DuckDB-era specs