Skip to main content

Aggregator System Enhancements

Feature: Enhanced Aggregation Capabilities Date: October 2025 Status: ✅ Implemented Components: API (Backend) + Frontend (React)

Table of Contents​

  1. Overview
  2. Backend Changes
  3. Frontend Changes
  4. Configuration Migration
  5. Testing

1. Overview​

1.1 Purpose​

Enhanced the aggregation system to support:

  • Quality metric aggregation
  • Multiple aggregators in single call
  • Entity type inference
  • Cross-table aggregation
  • Improved configuration management

1.2 Key Features​

✅ Quality Aggregation:

  • New /api/Aggregate/Qual endpoint
  • Supports all quality attributes
  • Works across all entity types

✅ Array Aggregators:

  • Request multiple aggregations at once
  • Single API call for avg, min, max, sum, count
  • Improved performance

✅ Configuration-Driven:

  • Migrated from hardcoded constants
  • Backend-driven metric definitions
  • Runtime flexibility

2. Backend Changes​

2.1 Quality Aggregation Endpoint​

File: dwd-api-aspnet/DwdApiAspNet/Controllers/AggregateController.cs

New Endpoint:

[HttpGet("Qual")]
public async Task<ActionResult> GetQualityAggregation(
    [FromQuery] string aggregator,      // avg,min,max,sum,count
    [FromQuery] string attribute,       // age,chlorine,tr1,conductivity,thm
    [FromQuery] string? entityType,     // junction,tank,reservoir,pipe,pump,valve
    [FromQuery] int scenarioId,
    [FromQuery] long? timestamp,
    [FromQuery] int decimalPrecision = 2)
{
    // Determine entity types that support this attribute
    var entityTypes = DetermineEntityTypes(attribute, entityType);

    // Aggregate across tables
    var results = await AggregateAcrossEntities(
        aggregator, attribute, entityTypes, scenarioId, timestamp);

    return Ok(new
    {
        values = results,
        unit = GetUnitForAttribute(attribute),
        metadata = new { aggregators, attribute, entityType, scenarioId }
    });
}

Supported:

  • All quality attributes
  • All entity types (automatic detection)
  • Multiple aggregators
  • Timestamp filtering

2.2 Entity Type Inference​

Automatic Detection:

private List<string> DetermineEntityTypes(string attribute, string? specifiedType)
{
    if (!string.IsNullOrEmpty(specifiedType))
        return new List<string> { specifiedType };

    // Infer from attribute
    return attribute.ToLower() switch
    {
        "age" => AllEntityTypes,
        "chlorine" => AllEntityTypes,
        "flow" => LinkEntityTypes,
        "pressure" => NodeEntityTypes,
        _ => AllEntityTypes
    };
}

2.3 Cross-Table Aggregation​

Implementation:

// Union queries across multiple tables
var junctionData = GetJunctionQualityData(attribute, scenarioId, timestamp);
var tankData = GetTankQualityData(attribute, scenarioId, timestamp);
var pipeData = GetPipeQualityData(attribute, scenarioId, timestamp);

// Combine and aggregate
var allData = junctionData.Concat(tankData).Concat(pipeData);
var result = CalculateAggregations(allData, aggregators);

3. Frontend Changes​

3.1 Configuration Context Integration​

Migrated from hardcoded constants to API-driven configuration:

Before:

import { METRICS, AGGREGATOR_MAP } from '../constants';

const availableAggregators = AGGREGATOR_MAP[metric]; // Hardcoded

After:

import { useConfiguration } from '../contexts/ConfigurationContext';

const { getAggregatorsForMetric } = useConfiguration();
const availableAggregators = getAggregatorsForMetric(metric); // From API

3.2 Quality Metrics Support​

Enhanced Components:

  • Quality/Hydraulics dropdowns now support configuration-driven metrics
  • Aggregator selection based on metric configuration
  • Dynamic metric availability

3.3 Chart Components​

Updated:

  • HydrQualChart.jsx - Uses configuration context
  • ScadaChart.jsx - Uses configuration context
  • GqcLineChart.jsx - Supports quality metrics

4. Configuration Migration​

4.1 Heatmap Configuration​

Backend Migration:

// visualization-config.json
{
  "visualizationConfigs": [
    {
      "id": "flow-heatmap-color-stops",
      "metricId": "Flow",
      "configType": "HeatmapColorStops",
      "config": {
        "colorStops": [0, 0.001, 0.3, 0.4, 1],
        "colors": ["#0000FF", "#00FFFF", "#FFFF00", "#FF8000", "#FF0000"]
      }
    }
  ]
}

Frontend Usage:

const { getHeatmapConfig } = useConfiguration();
const colorStops = getHeatmapConfig('Flow');

4.2 Aggregator Configuration​

Backend:

// metrics.json
{
  "metrics": [
    {
      "id": "Flow",
      "aggregators": ["avg", "min", "max"],
      "floatPrecision": 2
    }
  ]
}

Frontend:

const { config } = useConfiguration();
const aggregators = config.metrics
    .find(m => m.id === 'Flow')
    ?.aggregators || [];

5. Testing​

5.1 Test Cases​

Quality Aggregation:

# Test 1: Single aggregator
GET /api/Aggregate/Qual?aggregator=avg&attribute=chlorine&scenarioId=1
✅ Returns average chlorine

# Test 2: Multiple aggregators
GET /api/Aggregate/Qual?aggregator=avg,min,max&attribute=age&scenarioId=1
✅ Returns all three aggregations

# Test 3: Entity type filter
GET /api/Aggregate/Qual?aggregator=avg&attribute=chlorine&entityType=junction&scenarioId=1
✅ Returns only junction data

# Test 4: Timestamp filter
GET /api/Aggregate/Qual?aggregator=avg&attribute=chlorine&scenarioId=1&timestamp=3600
✅ Returns data at specific time

Configuration Loading:

# Test 5: Get all configuration
GET /api/Configuration/all
✅ Returns complete configuration

# Test 6: Get metrics for entity
GET /api/Metrics/for-entity/Tank
✅ Returns tank-compatible metrics

5.2 Frontend Integration Tests​

✅ Configuration loads on app start ✅ Dropdowns populate from configuration ✅ Charts display quality metrics ✅ Aggregators available per metric ✅ Fallback to cached data on offline

Summary​

✅ Aggregator Enhancements Complete:

  • Quality aggregation endpoint
  • Array aggregator support
  • Entity type inference
  • Cross-table aggregation
  • Configuration-driven metrics

✅ Frontend Integration Complete:

  • Configuration context
  • Dynamic metric/aggregator loading
  • Chart component updates
  • Backward compatibility maintained

✅ Migration Benefits:

  • No hardcoded constants
  • Runtime flexibility
  • Single source of truth
  • Easy to extend
  • Better maintainability

Related Documentation:

  • Configuration Migration: FRONTEND_CONFIG_MIGRATION.md
  • API Reference: API_Endpoints_Reference.md
  • API Enhancements: This document

Status: ✅ Complete and Production Ready Last Updated: October 2025

Last updated on by Jake, Cursor