Device AI Predictions - Architecture Comparison

This document provides a side-by-side comparison of the existing AI prediction implementation (for hydraulic model entities) and the proposed implementation for Device data.

High-Level Comparison

Component	Model Entities (Current)	Device Entities (Proposed)
Entity Types	Tank, Reservoir, Junction, Pipe, Pump, Valve	Device (Water Meter)
Data Source	Hydraulic simulation results	Physical meter readings
Data Granularity	Sub-hourly to hourly	Monthly to yearly
Data Volume	High (1000s of points)	Low (10s of points)
Prediction Model	TSAI InceptionTimePlus	TSAI InceptionTimePlus (same)
Fallback Model	Scikit-learn Linear Regression	Scikit-learn Linear Regression (same)
Frontend Component	ForecastChart, HydrQualChart	DeviceChart

Architecture Comparison

Current: Model Entity Predictions

┌──────────────────────────────────────────────────────────────┐
│                    Frontend (React)                          │
│  ┌─────────────────┐  ┌─────────────────┐                   │
│  │ EsriCustomPopup │  │ ForecastChart   │                   │
│  │ HydrQualChart   │  │                 │                   │
│  └────────┬────────┘  └────────┬────────┘                   │
│           │                    │                             │
│           └─────────┬──────────┘                             │
└─────────────────────┼────────────────────────────────────────┘
                      │
                      ▼
  GET /{node_type}/{node_id}/graph-data
      ?base_url={url}
      &scenario_id={id}
      &hydraulics_metric={metric}
      &quality_species={species}
      &include_forecast=true
      &forecast_percentage=1.0
                      │
┌─────────────────────┼────────────────────────────────────────┐
│              FastAPI Service                                 │
│  ┌──────────────────────────────────────────────────────┐   │
│  │ get_node_graph_data()                                │   │
│  │   ├─ get_node_hydraulics_generic()                   │   │
│  │   ├─ get_node_quality_generic()                      │   │
│  │   └─ add_forecast_data()                             │   │
│  │       └─ predict_with_tsai()                         │   │
│  └──────────────────────────────────────────────────────┘   │
└─────────────────────┼────────────────────────────────────────┘
                      │
                      ▼
  GET /api/{NodeType}/{nodeId}/Hydraulics?scenarioId={id}
  GET /api/{NodeType}/{nodeId}/Quality?scenarioId={id}&species={name}
                      │
┌─────────────────────┼────────────────────────────────────────┐
│              ASP.NET API                                     │
│  ┌──────────────────────────────────────────────────────┐   │
│  │ TankController, ReservoirController, etc.            │   │
│  │   ├─ GetHydraulics()                                 │   │
│  │   └─ GetQuality()                                    │   │
│  └──────────────────────────────────────────────────────┘   │
└─────────────────────┼────────────────────────────────────────┘
                      │
                      ▼
        NodeHydraulicsDatum, NodeQualityDatum tables

Proposed: Device Entity Predictions

┌──────────────────────────────────────────────────────────────┐
│                    Frontend (React)                          │
│  ┌─────────────────┐  ┌─────────────────┐                   │
│  │ EsriCustomPopup │  │ DeviceChart     │                   │
│  │ (with Device    │  │ (enhanced)      │                   │
│  │  support)       │  │                 │                   │
│  └────────┬────────┘  └────────┬────────┘                   │
│           │                    │                             │
│           └─────────┬──────────┘                             │
└─────────────────────┼────────────────────────────────────────┘
                      │
                      ▼
  GET /device/{device_id}/tag/{tag_id}/graph-data
      ?base_url={url}
      &include_forecast=true
      &forecast_percentage=1.0
      &startTime={unix}
      &endTime={unix}
                      │
┌─────────────────────┼────────────────────────────────────────┐
│              FastAPI Service (NEW)                           │
│  ┌──────────────────────────────────────────────────────┐   │
│  │ get_device_tag_graph_data()  [NEW]                   │   │
│  │   ├─ Fetch from ASP.NET Device endpoint              │   │
│  │   ├─ Transform data format                           │   │
│  │   └─ add_forecast_data()  [REUSED]                   │   │
│  │       └─ predict_with_tsai()  [REUSED]               │   │
│  └──────────────────────────────────────────────────────┘   │
└─────────────────────┼────────────────────────────────────────┘
                      │
                      ▼
  GET /api/Device/{deviceId}/tag/{tagId}/data
      ?startTime={unix}
      &endTime={unix}
                      │
┌─────────────────────┼────────────────────────────────────────┐
│              ASP.NET API (NEW)                               │
│  ┌──────────────────────────────────────────────────────┐   │
│  │ DeviceController  [ENHANCED]                         │   │
│  │   └─ GetDeviceTagData()  [NEW]                       │   │
│  └──────────────────────────────────────────────────────┘   │
└─────────────────────┼────────────────────────────────────────┘
                      │
                      ▼
               DeviceTagDatum table

Data Model Comparison

Model Entity Data

NodeHydraulicsDatum:

{
  Id: string
  SimUnixTime: long
  NodeId: string
  ScenarioId: int
  Head: double?
  Demand: double?
  Pressure: double?
  Flow: double?
  // ... multiple metrics in one record
}

NodeQualityDatum:

{
  Id: string
  SimUnixTime: long
  NodeId: string
  ScenarioId: int
  Species: string
  Quality: double
}

Device Entity Data

DeviceTagDatum:

{
  DeviceTagId: int  (PK)
  UnixTime: long    (PK)
  Value: double
  // Simple: just one value per timestamp
}

DeviceTag (metadata):

{
  Id: int
  Name: string        // "Consumption", "Usage"
  TagType: string     // "monthly_consumption"
  Unit: string        // "cubic feet"
  DeviceId: int
}

Key Differences:

Model data: Multiple metrics per record, scenario-based
Device data: Single value per record, tag-based
Model data: Tied to simulation scenarios
Device data: Real-world measurements, no scenarios

API Endpoint Comparison

Model Entities (Current)

FastAPI

GET /{node_type}/{node_id}/graph-data

Parameters:
  - node_type: "tank" | "reservoir" | "junction" | "pipe" | "pump" | "valve"
  - node_id: string
  - base_url: string
  - scenario_id: int
  - hydraulics_metric: string (optional)
  - quality_species: string (optional)
  - include_forecast: bool
  - forecast_percentage: float
  - start_time: int (optional)
  - end_time: int (optional)

Response:
{
  "node_type": "tank",
  "node_id": "T-1",
  "scenario_id": 1,
  "hydraulics_metric": "Head",
  "quality_species": "Chlorine",
  "hydraulics_data": [
    { "simUnixTime": 1234, "head": 123.45, ... }
  ],
  "quality_data": [
    { "simUnixTime": 1234, "species": "Chlorine", "quality": 0.8 }
  ],
  "forecast_info": { ... }
}

ASP.NET

GET /api/{NodeType}/{nodeId}/Hydraulics

Parameters:
  - scenarioId: int
  - startTime: long (optional)
  - endTime: long (optional)
  - decimalPrecision: int

Response:
[
  { "id": "...", "simUnixTime": 1234, "head": 123.45, "demand": 50.2, ... }
]

Device Entities (Proposed)

FastAPI (NEW)

GET /device/{device_id}/tag/{tag_id}/graph-data

Parameters:
  - device_id: int
  - tag_id: int
  - base_url: string
  - include_forecast: bool
  - forecast_percentage: float
  - start_time: int (optional)
  - end_time: int (optional)

Response:
{
  "device_id": 123,
  "tag_id": 456,
  "include_forecast": true,
  "data": [
    { "deviceTagId": 456, "unixTime": 1234, "value": 1234.56, "isForecast": false },
    { "deviceTagId": 456, "unixTime": 2345, "value": 1250.30, "isForecast": true }
  ],
  "forecast_info": { ... }
}

ASP.NET (NEW)

GET /api/Device/{deviceId}/tag/{tagId}/data

Parameters:
  - startTime: long (optional)
  - endTime: long (optional)

Response:
[
  { "deviceTagId": 456, "unixTime": 1234, "value": 1234.56 },
  { "deviceTagId": 456, "unixTime": 2345, "value": 1150.23 }
]

Key Differences:

Model: Separate endpoints for hydraulics and quality, combined response
Device: Single endpoint for tag data, simpler structure
Model: Scenario-based filtering
Device: No scenario concept (real-world data)
Model: Multiple metrics per timestamp
Device: Single value per timestamp

Frontend Component Comparison

Model Entities

ForecastChart.jsx (Dedicated prediction chart):

<ForecastChart
  id="T-1"
  type="Tank"
  species="Chlorine"
  hydraulicsMetric="Head"
/>

Always fetches from FastAPI
Always includes predictions
Dedicated to showing forecasts

HydrQualChart.jsx (Standard chart):

<HydrQualChart
  id="T-1"
  type="Tank"
  species="Chlorine"
  hydraulicsMetric="Head"
/>

Fetches from ASP.NET API
No predictions
Standard data visualization

Device Entities

DeviceChart.jsx (Hybrid chart):

<DeviceChart
  deviceId={123}
  leftTag={{ id: 456, name: "Consumption", unit: "cf" }}
  rightTag={{ id: 789, name: "Usage", unit: "cf" }}
  showPredictions={true}  // Toggle predictions on/off
/>

Conditionally fetches from FastAPI or ASP.NET
Supports dual-axis (left/right tags)
Single component handles both modes

Key Differences:

Model: Separate components for predictions vs. standard
Device: Single component with toggle
Model: Single metric or quality per chart
Device: Can show two tags simultaneously (dual-axis)

Prediction Pipeline Comparison

Shared Components (Used by Both)

Both Model and Device predictions use the same core prediction logic:

# Shared by both entity types
def add_forecast_data(data: List[dict], forecast_percentage: float) -> List[dict]:
  """Add forecast data using time series predictions."""
  # 1. Calculate forecast points
  # 2. Identify numeric fields
  # 3. Generate predictions for each field
  # 4. Create forecast records with isForecast flag
  # 5. Return combined data

def predict_with_tsai(data: List[dict], target_field: str,
                     forecast_points: int) -> List[float]:
  """Use TSAI InceptionTimePlus for predictions."""
  # 1. Prepare time series data
  # 2. Train TSAI model
  # 3. Generate predictions
  # 4. Fallback to scikit-learn if needed

def predict_with_scikit_learn(data: List[dict], target_field: str,
                              forecast_points: int) -> List[float]:
  """Fallback using Linear Regression."""
  # 1. Prepare features
  # 2. Train linear model
  # 3. Generate predictions

Data Transformation Differences

Model Entities:

# Input from ASP.NET API
{
  "id": "...",
  "simUnixTime": 1234567890,
  "head": 123.45,
  "demand": 50.2,
  "pressure": 85.3
}

# Already in correct format for prediction pipeline
# Multiple fields predicted simultaneously

Device Entities:

# Input from ASP.NET API
{
  "deviceTagId": 456,
  "unixTime": 1234567890,
  "value": 1234.56
}

# Transform to prediction format
{
  "simUnixTime": 1234567890,  # Rename unixTime -> simUnixTime
  "value": 1234.56            # Keep value field
}

# Single field (value) predicted

Configuration Comparison

Model Entities

# config.py
PREDICTION_CONFIG = {
  "min_data_points": 5,
  "window_size_ratio": 0.2,
  "min_window_size": 10,
  "max_window_size": 30,
  "batch_sizes": 32,
  "training_epochs": 3,
  "learning_rate": 0.001
}

FALLBACK_CONFIG = {
  "min_data_points": 3,
  "context_window_size": 10,
  "default_interval_seconds": 3600  # 1 hour
}

NODE_TYPE_MAPPING = {
  "tank": "Tank",
  "reservoir": "Reservoir",
  "junction": "Junction",
  "pipe": "Pipe",
  "pump": "Pump",
  "valve": "Valve"
}

Device Entities (Proposed Addition)

# config.py (NEW)
DEVICE_CONFIG = {
  "supported_tag_types": [
    "monthly_consumption",
    "quarterly_consumption",
    "yearly_consumption",
    "daily_flow",
    "hourly_flow"
  ],
  "min_data_points": 3,  # Lower threshold for sparse data
  "default_forecast_percentage": 0.5,
  "default_interval_seconds": 2592000  # 30 days (monthly)
}

# Can override PREDICTION_CONFIG for devices if needed
DEVICE_PREDICTION_CONFIG = {
  "min_data_points": 3,     # Lower for monthly data
  "window_size_ratio": 0.3,  # Larger window for seasonal patterns
  "min_window_size": 3,
  "max_window_size": 12,     # ~1 year for monthly data
  "training_epochs": 2       # Fewer epochs for smaller datasets
}

User Experience Comparison

Model Entities

User clicks on Tank/Pipe/etc in map
Popup opens with entity details
User clicks "Graph" tab
Dropdowns appear: "Hydraulics Metric" and "Quality Species"
AI Prediction button appears (green button with brain icon)
Chart shows standard data initially
User clicks "AI Predictions" button
Button turns orange, text changes to "Hide Predictions"
Chart updates with forecast data (may take 1-2 seconds)
Predictions shown as continuation of line with annotation

Device Entities (Proposed)

User clicks on Device (water meter) in map
Popup opens with device details
User clicks "Graph" tab
Dropdowns appear: "Left Axis Tag" and "Right Axis Tag"
AI Prediction button appears (green button with brain icon) ✓ Same
Chart shows standard data initially
User clicks "AI Predictions" button
Button turns orange, text changes to "Hide Predictions" ✓ Same
Chart updates with forecast data (may take 1-2 seconds) ✓ Same
Predictions shown with dashed lines, 70% opacity

Key Similarities:

Same prediction button component
Same toggle behavior
Same visual feedback (orange when active)
Same loading states

Key Differences:

Device: Two tags simultaneously (dual-axis)
Device: Predictions styled differently (dashed vs. solid)
Device: No quality/hydraulics distinction (just tags)

Testing Comparison

Model Entities Tests

Unit Tests:

Test prediction pipeline with synthetic data
Test fallback mechanisms
Test time series window generation

Integration Tests:

Test FastAPI → ASP.NET → Database flow
Test with different node types
Test with various scenario IDs
Test with quality and hydraulics data

UI Tests:

Test prediction button toggle
Test chart rendering with predictions
Test annotation display
Test multiple node types

Device Entities Tests (Proposed)

Unit Tests:

Test prediction pipeline with sparse data ✓ Same algorithm
Test with different tag types (monthly/quarterly/yearly)
Test data transformation (unixTime ↔ simUnixTime)

Integration Tests:

Test FastAPI → ASP.NET → Database flow ✓ Similar pattern
Test with different device IDs
Test with different tag types
Test with date range filters

UI Tests:

Test prediction button toggle ✓ Same button
Test dual-axis chart with predictions
Test dashed line styling for forecasts
Test with one vs. two tags

Performance Comparison

Model Entities

Typical Dataset:

1000-5000 data points per metric
Sub-hourly granularity
Multiple metrics per request

Performance Characteristics:

Training time: 2-5 seconds
Memory usage: Moderate (32MB+)
Prediction time: < 1 second

Device Entities (Expected)

Typical Dataset:

12-48 data points per tag (monthly/quarterly)
Monthly to yearly granularity
Single value per request

Expected Performance:

Training time: < 1 second (much less data)
Memory usage: Low (< 10MB)
Prediction time: < 0.5 seconds
Likely to use fallback model more often due to data sparsity

Optimization Opportunities:

Device predictions could be cached longer (data changes infrequently)
Batch predictions for multiple devices
Pre-compute predictions for common date ranges

Summary

Aspect	Model Entities	Device Entities	Reusable?
Prediction Algorithm	TSAI + Scikit-learn	TSAI + Scikit-learn	✅ 100%
API Pattern	Generic node endpoint	Device-specific endpoint	✅ Pattern only
Data Format	Multi-metric records	Single-value records	⚠️ Needs transform
Frontend Button	PredictionButton	PredictionButton	✅ 100%
Chart Component	ForecastChart/HydrQualChart	DeviceChart	⚠️ Needs enhancement
Configuration	PREDICTION_CONFIG	DEVICE_CONFIG	⚠️ Additional config
UI/UX Flow	Click → Toggle → Predict	Click → Toggle → Predict	✅ 100%

Reusability Score: ~70-80%

New Code Required: ~20-30%

Implementation Effort Comparison

Model Entities (Original Implementation)

Effort: ~40-60 hours
Scope: Full TSAI pipeline, multiple node types, frontend integration
Complexity: High (new feature)

Device Entities (Proposed Addition)

Effort: 10-14 hours (see design doc)
Scope: Extend existing pipeline, single entity type, enhance one component
Complexity: Low to Medium (extension of existing)

Efficiency Gain: ~75% reduction in effort due to reusable components

Conclusion

The proposed Device AI predictions implementation leverages ~70-80% of the existing infrastructure:

✅ Fully Reusable:

Prediction algorithms (TSAI, scikit-learn)
Prediction button component
UI/UX patterns
Configuration patterns
Error handling

⚠️ Partially Reusable:

API endpoint structure (same pattern, different route)
Chart components (enhance existing)
Data transformation (simple mapping)

❌ New Components:

ASP.NET Device tag endpoint
FastAPI Device prediction endpoint
Device-specific configuration

This high degree of reusability makes the implementation efficient and maintainable, with an estimated effort of only 10-14 hours vs. 40-60 hours for a from-scratch implementation.

Device AI Predictions - Architecture Comparison

High-Level Comparison​

Architecture Comparison​

Current: Model Entity Predictions​

Proposed: Device Entity Predictions​

Data Model Comparison​

Model Entity Data​

Device Entity Data​

API Endpoint Comparison​

Model Entities (Current)​

FastAPI​

ASP.NET​

Device Entities (Proposed)​

FastAPI (NEW)​

ASP.NET (NEW)​

Frontend Component Comparison​

Model Entities​

Device Entities​

Prediction Pipeline Comparison​

Shared Components (Used by Both)​

Data Transformation Differences​

Configuration Comparison​

Model Entities​

Device Entities (Proposed Addition)​

User Experience Comparison​

Model Entities​

Device Entities (Proposed)​

Testing Comparison​

Model Entities Tests​

Device Entities Tests (Proposed)​

Performance Comparison​

Model Entities​

Device Entities (Expected)​

Summary​

Implementation Effort Comparison​

Model Entities (Original Implementation)​

Device Entities (Proposed Addition)​

Conclusion​

High-Level Comparison

Architecture Comparison

Current: Model Entity Predictions

Proposed: Device Entity Predictions

Data Model Comparison

Model Entity Data

Device Entity Data

API Endpoint Comparison

Model Entities (Current)

FastAPI

ASP.NET

Device Entities (Proposed)

FastAPI (NEW)

ASP.NET (NEW)

Frontend Component Comparison

Model Entities

Device Entities

Prediction Pipeline Comparison

Shared Components (Used by Both)

Data Transformation Differences

Configuration Comparison

Model Entities

Device Entities (Proposed Addition)

User Experience Comparison

Model Entities

Device Entities (Proposed)

Testing Comparison

Model Entities Tests

Device Entities Tests (Proposed)

Performance Comparison

Model Entities

Device Entities (Expected)

Summary

Implementation Effort Comparison

Model Entities (Original Implementation)

Device Entities (Proposed Addition)

Conclusion