hydrotrek-chat-agent-implementation-review
Documentation status: CURRENT Architecture review of the DWD chat agent stack (CopilotKit → MAF → MCP → FastAPI). Use alongside chat-agent-mcp-development-guide.md for implementation workflow.
DWD chat agent — implementation review
Audience: Developers and reviewers evaluating chat agent design, API-to-tool mapping, and roadmap for hydraulics/SCADA-style prompts.
Captured: 2026-10-15
Related:
- chat-agent-mcp-development-guide.md — mandatory workflow for changing agent/MCP behavior
- dwd-mcp-agent-scenario-create-v1.md — agent scenario provenance
- dwd-swmm-suite-sync.md — DWD vs SWMM agent/MCP patterns
1. End-to-end architecture
dwd-frontend-react (CopilotKit + CopilotAppContext)
│
▼
dwd-copilot-server (HttpAgent proxy — no system prompt)
│
▼
dwd-api-fastapi POST /agent (MAF + OpenAI + composed playbooks)
│
▼
dwd-mcp-fastmcp (FastMCP tools → httpx → FastAPI)
│
▼
dwd-api-fastapi REST + specialty routes → Azure (tables, blobs, Parquet, SCADA DB)
| Layer | Repo path | Role | Intelligence here? |
|---|---|---|---|
| CopilotKit UI | dwd-frontend-react | Chat, JWT, live twin/scenario/asset context | No — passes context to /agent |
| Runtime proxy | dwd-copilot-server | CORS + HttpAgent → AGUI_AGENT_URL | No system prompt |
| MAF agent | dwd-api-fastapi/app/agent/ | Agent Framework, instructions/*.md, middleware | Yes — routing guidance via playbooks |
| MCP server | dwd-mcp-fastmcp/tools.py | Tool definitions, HTTP proxy, formatters | Execution + response shaping |
| Product API | dwd-api-fastapi/app/ | Auth, caps, persistence, DuckDB/Parquet, SCADA | Yes — business rules and data access |
Skills vs production agent
Cursor Agent Skills (.cursor/skills) are for IDE workflows only. They do not load into the production MAF agent.
Production behavior comes from:
dwd-api-fastapi/app/agent/instructions/*.md(composed at startup)- MCP tool docstrings and
Annotatedparameter descriptions
Design order (MAF-first)
When adding or changing behavior, the suite standard is:
- API / MCP enforcement — reject invalid actions; cap resources; omit dangerous tools.
- Tool shape & dedicated tools — resolvers, composite reads (
get_scenario_details), actionable error strings. - MAF middleware / CopilotKit context — session facts (selected twin), MCP connected/disconnected hints.
- Composed instructions — thin
00_core.md+ domain playbooks. - Avoid duplicating tool routing tables in prose when tool docstrings + manifest already define behavior.
2. How APIs become MCP tools (“pythonify”)
The MCP server is a REST adapter, not a second domain layer.
- Handler in
dwd-mcp-fastmcp/tools.py—@mcp.tool, clear docstring,Annotated[..., "…"]param descriptions (critical for the model). - HTTP —
dwd_apifor/api/v2/...,fastapi_clientfor FastAPI-only paths (e.g./twins/.../network). - Auth — forward
Authorizationwhen present; otherwiseX-DWD-MCP-Secret(and agent-origin headers for trusted mutations). - Errors as strings —
_call()returns HTTP/error text instead of raising so the model can explain failures in chat. - Optional formatters — reshape API JSON for operators (e.g.
get_scenario_details→agent-summarywithdisplaySections).
Two tool tiers in use
| Tier | Examples | User questions |
|---|---|---|
| Thin proxy | get_scenario, get_node_timeseries | “Return this API payload for known parameters.” |
| Intent-shaped composite | get_scenario_details, get_twin_details, network formatters | “What are this scenario’s settings?” (aligned with Edit scenario UI) |
Recommendation: Copy the composite pattern for hydraulics and SCADA Q&A — not more raw 1:1 OpenAPI mirrors.
Current MCP surface (summary)
Registered in dwd-mcp-fastmcp/tools.py (see README tool table — table may lag code):
- Twins / scenarios: list, get, details, create, update, run, unqueue, definition
- Results (entity-centric):
get_node_timeseries,get_link_timeseries,get_model_summary,get_model_status_report - Specialty: fireflow, leakage
- Network (INP): summary, section, search
get_hydraulics_metrics was removed; metric codes come from the EPANET SSOT (same as UI).
3. Industry patterns (short glossary)
| Pattern | Idea | DWD fit |
|---|---|---|
| Tool-first on product API | LLM chooses tools; backend owns truth | Current default — good for audit, UI parity, auth |
| Fat tools / workflows | One tool = one user intent | Underused — best next step for hydraulics Q&A |
| OpenAPI / MCP auto-exposure | Generate hundreds of tools from spec | Usually bad at scale (context bloat, wrong granularity) |
| Text-to-SQL / ad hoc query | Model writes queries against a schema | Possible for SCADA DB; needs strict guardrails |
| RAG on docs | Answer “how does PDA work?” | Complements tools; does not replace result reads |
| Separate compute plane | In-memory engine session (SSC openswmm.mcp) | DWD uses persisted runs + Parquet — no in-proc EPANET session for chat |
DWD should not copy SSC engine tools literally (e.g. session get_flooding_summary). Port the user intent, backed by completed scenario results and SCADA APIs.
4. API first, or tool-only?
Default: API first for anything that touches persisted data, auth, or product semantics.
| Situation | Where to implement |
|---|---|
| New persisted behavior, UI might use it, caps/validation | FastAPI route/service first, then MCP tool |
| Chat-only phrasing / which tool to call | Playbook (app/agent/instructions/NN_*.md) |
| Chat-only response shape (no new server capability) | MCP formatter or agent-summary / display registry |
| Purely conversational (definitions, EPANET concepts) | Instructions only — no API |
Tool-only without API is appropriate for:
- Thin orchestration in MCP (call two existing endpoints and merge — still backed by APIs), or
- Ephemeral session-local helpers (SSC engine pattern — not DWD’s data plane).
Anti-pattern: mutations (create/run scenario) implemented only in MCP without FastAPI enforcement.
5. Example future prompts vs current stack
“Which nodes are flooded?” / “Which nodes are under pressure?”
Gap: Partially implemented at the API layer; not exposed to the agent via MCP.
FastAPI already supports bulk result reads, for example:
GET .../results/metric_slice?time=&objectSubType=Junction&metric=EN_PRESSURE(and otherEN_*metrics from SSOT)- Timestep snapshots:
time_node/time_link
These routes live in dwd-api-fastapi/app/results/routes.py. They are not registered in dwd-mcp-fastmcp/tools.py.
What is exposed: get_node_timeseries, which requires 1-based node_index, SSOT node type, and metric enum — suitable for a known asset, not for “which nodes?” across the network.
The model cannot reliably answer those prompts without either hundreds of per-node calls or intent tools / analytics primitives (see section 6).
Domain note: In EPANET/DWD, “flooded” may mean different things than in SWMM — e.g. low pressure / PDA deficiency, tank overflow (EN_CANOVERFLOW), etc. Product language should be explicit in tool names and docstrings.
“Generate pump curves for the available SCADA data”
Larger gap: SCADA is served from dwd-api-fastapi/app/scada/routes.py (Postgres per twin). Network and scenario result tools do not replace SCADA tag time series + pump linkage.
There is no MCP tool surface for SCADA reads today (only scenario fields like scadaAvailable in details/update).
Needs: API for pump/tag series (and optionally curve-fit) → MCP intent tool → playbook (units, SCADA DB configured, time window limits).
6. Generic support without coding every API call
Fully generic (“any question → auto tool”) — no, not safely at production quality. Models need bounded, named capabilities with clear inputs/outputs.
Partially generic — yes, in layers:
| Approach | Genericity | Effort | DWD status |
|---|---|---|---|
| A. Analytics primitives | Medium — model picks metric + time + filter | Low–medium | metric_slice, time_node, entity_at_time in API; not in MCP |
| B. SSOT-driven metric catalog in tool docs | Medium | Low | app/core/ssot_metrics.py; use SSOT labels in docstrings |
| C. Intent tools with parameters | Low genericity, high UX | Per intent | get_scenario_details is the template |
| D. Server-side “views” / agent summaries | High for fixed reports | Medium | Extend model-hydr-summary, registries, new /agent-summary-style routes |
| E. OpenAPI → MCP codegen | High tool count, low quality | One-time | Not recommended as primary strategy |
| F. NL → structured query (validated) | High if schema stable | High | Future for SCADA; allowlisted tables/metrics |
Recommended generic core: (A + C + D)
- Primitives: a small set of MCP tools on existing result routes (
metric_slice, list hydraulic times, bridge tosearch_network_elementfor ID ↔ index). - Intents: composites where operators repeat the same question (“junctions below X psi at T”, “worst 10 pressures”).
- Summaries: when the UI would show a report, add agent-oriented endpoints so MCP stays thin.
Avoid N tools for N HTTP paths and avoid exposing the full OpenAPI surface to the model.
Output size: Raw metric_slice for all junctions can be huge. Formatters should return counts, thresholds, top-K, EPANET IDs — not full arrays in chat.
7. Strengths of the current implementation
- Clear separation — proxy vs MAF vs MCP vs API; single place for prompts (
instructions/). - Security model — JWT + MCP secret,
origin=agent, caps — not prompt-dependent. - UI parity — scenario/twin
agent-summary+ display registries mirror Edit dialogs. - Operational hardening — stateless OpenAI, AG-UI tool order, live selection injection (see development guide troubleshooting).
- Contributor workflow — chat-agent-mcp-development-guide.md is a usable internal standard.
8. Gaps and risks
| Area | Observation |
|---|---|
| Results / hydraulics Q&A | Rich Parquet API; MCP still entity-centric timeseries. Bulk routes not exposed to the agent. |
| SCADA | API exists; no MCP tools; no results/SCADA playbooks. |
| Tool catalog vs code | dwd-mcp-fastmcp/README.md table may lag tools.py. |
| Playbooks | Core + scenario/twin details + lifecycle only — no hydraulics/SCADA playbook. |
| Index vs ID | Network search returns EPANET IDs; timeseries wants engine index — easy for the model to get wrong without a bridge tool. |
| SSC comparison | SWMM MCP has in-memory analysis tools; DWD should port intent, not the session model. |
9. Suggested roadmap
Phase 1 — Analytics primitives (high leverage)
- MCP: wrap
metric_slice,time_node(or POST variants); document how to obtaintime(CopilotsimulationStartUtc/simulationEndUtcor list hydraulic times). - Playbook:
20_hydraulics_results.md— completed run required, SSOT metric names, no raw JSON dumps. - Formatter: optional threshold / top-K in API or MCP so the model does not filter thousands of rows in prose.
Phase 2 — Intent tools for operator prompts
- e.g.
list_junctions_by_pressure(...)— internallyread_metric_slice+ index/ID map. - Disambiguate “flooded” vs “low pressure” vs tank overflow in naming and docs.
Phase 3 — SCADA and pump curves
- API for pump SCADA series and/or curve generation.
- MCP intent tool + playbook; reuse twin/scenario SCADA flags from agent-summary.
Phase 4 — Governance
- Keep checklist: API enforcement → agent-summary or formatter → MCP → playbook → tests.
- Tool budget: prefer composites over new thin proxies (rough guide: stay near ~25 MCP tools unless primitives are shared).
10. FAQ (direct answers)
How do we translate APIs into meaningful tools?
Name and document user intents, not routes. Use composite reads where the UI already defines meaning (get_scenario_details). For raw data APIs, add primitives with guardrails (time, metric, top-K) and formatters that return counts, extremes, and IDs.
API first or straight to agent tool?
API first for persistence, auth, and shared UI behavior. MCP may orchestrate multiple API calls without a new route when logic is chat-only and non-reusable; heavy domain logic belongs in FastAPI/services.
Generic support without every API call?
Use a small set of query primitives plus SSOT-backed metric vocabulary, and add named intent tools where multi-step composition is too error-prone for the model. Do not rely on one MCP tool per OpenAPI operation.
11. Code touchpoints (quick reference)
| Concern | Path |
|---|---|
| MAF agent build | dwd-api-fastapi/app/agent/copilot.py |
| Playbooks | dwd-api-fastapi/app/agent/instructions/*.md |
| MCP registration | dwd-mcp-fastmcp/tools.py |
| MCP HTTP client | dwd-mcp-fastmcp/api_client.py |
| Bulk results API | dwd-api-fastapi/app/results/routes.py |
| SCADA API | dwd-api-fastapi/app/scada/routes.py |
| SSOT metrics | dwd-api-fastapi/app/core/ssot_metrics.py |
| Agent summaries | .../agent-summary on twins/scenarios; app/agent/scenario_summary.py, twin_summary.py |
| Copilot proxy | dwd-copilot-server/lib/copilotRuntime.js |
Implementation workflow for changes: chat-agent-mcp-development-guide.md.