Skip to main content

DWD Dashboard Design

I asked Cursor to look at our existing code and recommend a design approach.

The Prompt​

Look at the controllers/endpoints that we have available in the exterior API project at C:\git\gqc\hydrotrek-dwd-suite\dwd-api-aspnet

Look at what fake data we're showing in the current frontend dashboard.

Help me design (no code yet) the proper way to tie our frontend dashboard to appropriate API endpoints. I would like to consider the following:

  1. Auto updating values in the dashboard every ___ seconds (configurable)
  2. Most efficient and industry-standard way to get the dashboard values, i.e. should it be a single API endpoint, a few broad categories of endpoints, or each individual endpoints.
  3. Any other considerations we should think of?

Current State Analysis​

Available API Endpoints​

The API provides comprehensive endpoints for:

  1. Hydraulic Data: Individual controllers for each entity type (Pump, Tank, Valve, Pipe, Junction, Reservoir) with hydraulic data
  2. Quality Data: Similar structure for water quality parameters
  3. Summary Data: ModelHydrSummaryController for aggregated system metrics
  4. Scenario Management: Full CRUD operations for scenarios
  5. Heatmap Data: CSV endpoints for visualization
  6. SCADA Data: Real-time SCADA asset and tag data
  7. Runtime Statistics: Scenario execution metadata

Current Frontend Dashboard Fake Data​

The dashboard currently displays:

  • Hydraulic Parameters: Pressure, flow, tank levels, pump status, valve status
  • Water Quality: Chlorine levels, water age
  • Demand & Consumption: Current demand, peak demand, peak periods
  • Alerts & Events: Low pressure, tank overflow, pump failures
  • Simulation Results: Convergence status, run time, last run time

1. Auto-Update Configuration​

Recommended Update Intervals:

  • Real-time metrics (pressure, flow, tank levels, alerts): 30-60 seconds
  • Quality parameters: 2-5 minutes (less critical for real-time monitoring)
  • System summary/historical trends: 5-10 minutes

Implementation Strategy:

// Configurable update intervals
const UPDATE_INTERVALS = {
  realtime: 30000,    // 30 seconds
  quality: 180000,    // 3 minutes
  summary: 300000     // 5 minutes
};

2. API Endpoint Strategy​

Recommended Approach: Hybrid with Grouped Endpoints by Update Frequency

Grouped Endpoints​

  • /api/dashboard/realtime

    • Returns: pressure, flow, tank levels, pump/valve status, and alerts
    • Update interval: 30-60 seconds

  • /api/dashboard/quality

    • Returns: water quality metrics (e.g., chlorine, water age)
    • Update interval: 2-5 minutes

  • /api/dashboard/summary

    • Returns: demand/consumption trends, simulation results, historical stats
    • Update interval: 5-10 minutes

Example Controller Signatures:

[HttpGet("dashboard/realtime")]
public async Task<ActionResult<RealtimeMetrics>> GetRealtimeMetrics([FromQuery] int scenarioId, [FromQuery] long currentTime) { ... }

[HttpGet("dashboard/quality")]
public async Task<ActionResult<QualityMetrics>> GetQualityMetrics([FromQuery] int scenarioId, [FromQuery] long currentTime) { ... }

[HttpGet("dashboard/summary")]
public async Task<ActionResult<SystemSummary>> GetSystemSummary([FromQuery] int scenarioId) { ... }

Frontend Implementation: Each dashboard section subscribes to its own update interval and endpoint, ensuring efficient and targeted data refreshes.

3. Data Structure Design​

Example Response Structures​

Realtime Metrics (/api/dashboard/realtime):

{
  "timestamp": 1703123456,
  "scenarioId": 1,
  "pressure": { "average": 65.2, "min": 32.1, "max": 98.7, "lowPressureNodes": 2 },
  "flow": { "average": 450.3, "min": 120.0, "max": 900.0, "totalFlow": 14500.0 },
  "tankLevels": { "averagePercentFull": 76.5, "totalVolume": 2.1, "lowTanks": 0 },
  "pumpStatus": { "active": 5, "inactive": 2, "totalFlow": 600.0, "totalPower": 1200.0 },
  "valveStatus": { "open": 8, "closed": 3, "flowControl": 5 },
  "alerts": { "total": 3, "lowPressure": 2, "tankOverflow": 0, "pumpFailures": 1 }
}

Quality Metrics (/api/dashboard/quality):

{
  "timestamp": 1703123456,
  "scenarioId": 1,
  "chlorine": { "average": 0.7, "min": 0.2, "max": 1.2, "lowChlorineNodes": 1 },
  "waterAge": { "average": 18.5, "max": 24.0 }
}

System Summary (/api/dashboard/summary):

{
  "timestamp": 1703123456,
  "scenarioId": 1,
  "demandTrends": { "current": 14500, "peak": 15000, "peakPeriods": ["6-8am", "5-7pm"] },
  "historicalMetrics": { "last24h": [...], "last7d": [...] },
  "systemStatus": { "convergence": "Yes", "lastRunTime": "12.3s", "lastRun": "2024-06-10 14:22", "scenarioStatus": "Running" }
}

4. Implementation Considerations​

Performance Optimizations​

  1. Database Indexing: Ensure proper indexes on SimUnixTime, ScenarioId, and entity IDs
  2. Caching Strategy:
    • Redis cache for frequently accessed data
    • Client-side caching for historical data
    • HTTP caching headers for static data
  3. Data Pagination: For large datasets, implement pagination
  4. Compression: Enable gzip compression for API responses

Error Handling​

  1. Graceful Degradation: Show cached data if API is unavailable
  2. Retry Logic: Implement exponential backoff for failed requests
  3. Fallback Data: Use last known good values during outages
  4. User Feedback: Clear indicators when data is stale or unavailable

Security Considerations​

  1. Authentication: Implement proper JWT token validation
  2. Rate Limiting: Prevent API abuse with rate limiting
  3. Data Validation: Validate all input parameters
  4. CORS Configuration: Proper CORS setup for frontend access

Monitoring & Observability​

  1. API Metrics: Track response times, error rates, and usage patterns
  2. Dashboard Performance: Monitor frontend update frequency and success rates
  3. Data Quality: Track data freshness and completeness
  4. User Experience: Monitor dashboard load times and responsiveness

5. Implementation Phases​

Phase 1: Basic Integration (Week 1-2)​

  • Create grouped dashboard endpoints (realtime, quality, summary)
  • Replace fake data with real API calls
  • Implement basic auto-refresh for each section (per recommended intervals)
  • Add error handling and loading states

Phase 2: Enhanced Features (Week 3-4)​

  • Implement configurable update intervals (user or admin adjustable)
  • Add data caching and optimization
  • Create alert derivation logic
  • Add historical trend data

Phase 3: Advanced Features (Week 5-6)​

  • Implement real-time WebSocket connections for critical data (optional)
  • Add user preferences for dashboard customization
  • Create advanced filtering and drill-down capabilities
  • Implement comprehensive monitoring and analytics

Summary: This hybrid grouped-endpoints design ensures each dashboard section updates at the most appropriate interval, minimizes unnecessary data transfer, and provides a scalable, maintainable, and user-friendly architecture for integrating your dashboard with the API.

Last updated on by Jake