Skip to main content

Database → EPANET Synchronization

Feature: Real-time database-to-EPANET synchronization Date: October 16-17, 2025 Status: ✅ Implemented and Complete Purpose: Apply database changes to EPANET model before simulations

Table of Contents​

  1. Overview
  2. Strategy
  3. Implementation
  4. Service Architecture
  5. Testing
  6. Limitations

1. Overview​

1.1 Purpose​

Enable database changes to flow directly to EPANET before running hydraulic and quality simulations, eliminating the need for manual INP file regeneration.

1.2 Problem Solved​

Before:

  • Network loaded from .inp files only
  • Database modifications not reflected in simulations
  • Manual INP file export required after every change
  • Simulation results didn't reflect database state

After:

  • Database changes automatically applied to EPANET model
  • Simulations use current database values
  • No manual export required
  • Database is source of truth

1.3 Solution Approach​

Chosen: Option 1 - Update EPANET In-Memory Model

Apply database changes to EPANET's in-memory model after loading from INP file, before running simulation.

┌─────────────┐
│  Load from  │
│  INP file   │
└──────┬──────┘
       │
       ▼
┌─────────────┐
│  Apply DB   │ ← New synchronization step
│  Changes    │
└──────┬──────┘
       │
       ▼
┌─────────────┐
│  Run Sim    │
└─────────────┘

Why this approach:

  • ✅ Minimal code changes
  • ✅ Leverages existing EPANET setter functions
  • ✅ INP file remains as a baseline/template
  • ✅ Database becomes source of truth for changes
  • ✅ No binary format dependencies

2. Strategy​

2.1 Data Categories​

Global Network Data (All Records Synced):

  • Junctions, Tanks, Reservoirs (all nodes)
  • Pipes, Valves, Pumps (all links via LinkProperty table)
  • Patterns (all patterns)
  • Curves (all curves)
  • Node and Link properties

Scenario-Specific Data (Needs Special Handling):

  • Options (demand model, solver settings)
  • Time Parameters (duration, timesteps)
  • Controls (simple and rule-based)

2.2 Update Flow​

2.3 Property Mapping​

Nodes: | Database | EPANET Property Code | Entity | Sync Method | |----------|---------------------|--------|-------------| | Elevation | EN_ELEVATION (0) | All Nodes | SyncJunctionsAsync | | BaseDemand | EN_BASEDEMAND (1) | Junctions | SyncJunctionsAsync | | Pattern | EN_PATTERN (2) | Junctions/Reservoirs | SyncJunctionsAsync/ReservoirsAsync | | InitLevel | EN_INITLEVEL (21) | Tanks | SyncTanksAsync | | MinLevel | EN_MINLEVEL (20) | Tanks | SyncTanksAsync | | MaxLevel | EN_MAXLEVEL (22) | Tanks | SyncTanksAsync | | Diameter | EN_TANKDIAM (23) | Tanks | SyncTanksAsync | | VolCurve | EN_VOLCURVE (26) | Tanks | SyncTanksAsync |

Links: | Database | EPANET Property Code | Entity | Sync Method | |----------|---------------------|--------|-------------| | Diameter | EN_DIAMETER (0) | Pipes/Valves | SyncPipesAsync/ValvesAsync | | Length | EN_LENGTH (1) | Pipes | SyncPipesAsync | | Roughness | EN_ROUGHNESS (2) | Pipes | SyncPipesAsync | | MinorLoss | EN_MINORLOSS (3) | Pipes/Valves | SyncPipesAsync/ValvesAsync | | InitSetting | EN_INITSETTING (4) | Valves/Pumps | SyncValvesAsync/LinkPropertiesAsync | | InitStatus | EN_INITSTATUS (11) | All Links | SyncLinkPropertiesAsync | | KWall | EN_KWALL (23) | Pipes | SyncLinkPropertiesAsync | | PumpCurve | EN_PUMP_HCURVE (14) | Pumps | SyncLinkPropertiesAsync |

3. Implementation​

3.1 Core Infrastructure ✅​

New Service: NetworkSynchronizationService.cs

Location: epanetconsolecore/EpanetConsoleCore/Services/

Main Interface:

public class NetworkSynchronizationService
{
    private readonly EpanetWrapper _epanetWrapper;
    private readonly EN2PostgresContext _context;
    private readonly HTLogger _logger;

    public NetworkSynchronizationService(
        EpanetWrapper epanetWrapper,
        EN2PostgresContext context,
        HTLogger logger)
    {
        _epanetWrapper = epanetWrapper;
        _context = context;
        _logger = logger;
    }

    // Main synchronization method
    public async Task SyncAllFromDatabaseAsync()
    {
        await SyncNodesAsync();
        await SyncLinksAsync();
        await SyncPatternsAsync();
        await SyncCurvesAsync();
        await SyncOptionsAsync();
        await SyncTimeParametersAsync();
    }
}

3.2 Synchronization Methods​

Node Synchronization​

// Junctions: Elevation, BaseDemand, Pattern
private async Task SyncJunctionsAsync()
{
    var junctions = await _context.Junctions.ToListAsync();
    foreach (var junction in junctions)
    {
        int index = _epanetWrapper.EN_getnodeindex(junction.Id);
        if (index <= 0) continue;

        _epanetWrapper.EN_setnodevalue(index, EN_ELEVATION, junction.Elevation);
        _epanetWrapper.EN_setnodevalue(index, EN_BASEDEMAND, junction.BaseDemand);

        if (!string.IsNullOrEmpty(junction.Pattern))
        {
            int patIndex = _epanetWrapper.EN_getpatternindex(junction.Pattern);
            if (patIndex > 0)
                _epanetWrapper.EN_setnodevalue(index, EN_PATTERN, patIndex);
        }
    }
}
// Uses LinkProperty table for flexible property updates
private async Task SyncLinkPropertiesAsync()
{
    var linkProperties = await _context.LinkProperties.ToListAsync();
    foreach (var prop in linkProperties)
    {
        int index = _epanetWrapper.EN_getlinkindex(prop.LinkId);
        if (index <= 0) continue;

        _epanetWrapper.EN_setlinkvalue(index, prop.LinkPropertyTypeId, prop.LinkValue);
    }
}

Pattern & Curve Bulk Updates​

// Efficient bulk pattern update
public int EN_setpattern(int index, double[] values, int len)
{
    return EpanetImport.EN_setpattern(_project, index, values, len);
}

// Efficient bulk curve update
public int EN_setcurve(int index, double[] xValues, double[] yValues, int nPoints)
{
    return EpanetImport.EN_setcurve(_project, index, xValues, yValues, nPoints);
}

3.3 Integration Point​

File: epanetconsolecore/EpanetConsoleCore/Program.cs

Integrated at line 251 (after startup, before simulation):

// Load network from INP file
bool startupSuccess = epanetWrapper.Startup(doQuality, msxFilename);
if (!startupSuccess) { /* error handling */ }

// ✨ Synchronize network from database
logger.LogInformation("Synchronizing network from database...");
scenario.CurrentStep = "Synchronizing network from database";
scenario.ProgressPercentage = 7;
await enContext.SaveChangesAsync();

var syncService = new NetworkSynchronizationService(epanetWrapper, enContext, logger);
try
{
    await syncService.SyncAllFromDatabaseAsync();
    logger.LogInformation("Network synchronization completed");
}
catch (Exception syncEx)
{
    logger.LogError($"Network synchronization failed: {syncEx.Message}");
    // Continue with INP values (partial sync better than crash)
}

// Continue with simulation...

4. Service Architecture​

4.1 Synchronization Categories​

Implemented Methods:

  1. Node Sync Methods:

    • SyncJunctionsAsync() - Elevation, BaseDemand, Pattern
    • SyncTanksAsync() - Elevation, Diameter, Levels, VolCurve
    • SyncReservoirsAsync() - InitialHead, Pattern
    • SyncNodePropertiesAsync() - InitQual, Emitter, TankBulkCoeff
    • SyncTankMixingAsync() - Mixing model and fraction

  2. Link Sync Methods:

    • SyncPipesAsync() - Diameter, Length, Roughness, MinorLoss
    • SyncValvesAsync() - Diameter, ValveType, InitSetting, MinorLoss
    • SyncLinkPropertiesAsync() - Pump properties, KWall, InitStatus, etc.

  3. Pattern & Curve Methods:

    • SyncPatternsAsync() - Updates existing patterns (bulk operation)
    • SyncCurvesAsync() - Updates existing curves (bulk operation)

  4. Configuration Methods:

    • SyncOptionsAsync() - Trials, Accuracy, QualityTolerance
    • SyncTimeParametersAsync() - Duration, timesteps (partial)

4.2 Scenario-Specific Handling​

Options:

  • Base sync uses default options
  • SaveInpFileMode applies scenario-specific options via SyncOptionsFromObject()
  • Regular runs apply demand model via WriteDemandModel()

Controls:

  • Simple controls: Written via existing WriteControls() method
  • Rule-based controls: Written via existing WriteControls() method
  • Already scenario-aware

4.3 Audit Results ✅​

Global Data Synchronization:

  • ✅ All nodes synced (Junctions, Tanks, Reservoirs)
  • ✅ All links synced (Pipes, Valves, Pumps)
  • ✅ All patterns synced
  • ✅ All curves synced
  • ✅ All properties synced

Scenario-Specific Handling:

  • ✅ Options handled correctly (SaveInpFileMode fixed)
  • ✅ Controls already scenario-aware
  • ⚠️ TimeParameters partially implemented (string parsing pending)

5. Testing​

5.1 Test Scenarios​

Test Case 1: Pipe Property Changes

-- Change pipe diameter in database
UPDATE "Pipes" SET "Diameter" = 12 WHERE "Id" = 'P-1';

-- Run simulation
-- Expected: EPANET uses diameter=12, not INP file value

Test Case 2: Tank Level Changes

-- Change tank initial level
UPDATE "Tanks" SET "InitLevel" = 50 WHERE "Id" = 'T-1';

-- Run simulation
-- Expected: Tank starts at level 50

Test Case 3: Pattern Changes

-- Update pattern multipliers
UPDATE "DataPattern" SET "Pattern" = 1.5 WHERE "Pattern_Id" = 'PAT1' AND "SeqNum" = 1;

-- Run simulation
-- Expected: Pattern uses new multipliers

5.2 Verification Points​

✅ Verify synchronization happens:

  • Check logs for "Synchronizing network from database..."
  • Check logs for "Network synchronization completed"
  • Progress bar updates to 7% during sync

✅ Verify values applied:

  • Compare EPANET results with expected values
  • Check that database values override INP values
  • Verify scenario-specific options are used

✅ Verify error handling:

  • Sync failures logged but don't crash simulation
  • Partial sync better than no sync
  • Clear error messages

6. Limitations​

6.1 Current Limitations​

Cannot Add New Elements:

  • Cannot add new junctions, pipes, etc. via database
  • Can only update existing elements from INP file
  • INP file defines network topology

String Parsing Pending:

  • Time parameters stored as strings ("24 HOURS")
  • Need parsing logic to convert to seconds
  • Currently skipped in sync

Pattern/Curve Creation:

  • Cannot add new patterns or curves
  • Can only update existing ones
  • Pattern/Curve IDs must exist in INP

6.2 Design Rationale​

Why INP as baseline:

  • Network topology is complex (connectivity graph)
  • EPANET requires valid network structure
  • Easier to load valid baseline then update values
  • Avoids topology validation complexity

Why not sync new elements:

  • Adding nodes/links requires topology validation
  • Must ensure network connectivity
  • Graph algorithms complex and error-prone
  • Future enhancement if needed

7. Future Enhancements​

7.1 Time Parameter Parsing​

Status: Planned Effort: Low (1-2 hours)

Parse time strings and apply to EPANET:

// Parse "24 HOURS" → 86400 seconds
// Parse "5 DAYS" → 432000 seconds
// Parse "30 MINUTES" → 1800 seconds

7.2 Topology Updates​

Status: Future Consideration Effort: High (weeks)

Support adding/removing network elements:

  • Add new junctions/pipes via database
  • Validate network connectivity
  • Update topology in EPANET
  • Complex graph algorithms required

7.3 Differential Sync​

Status: Optimization Idea Effort: Medium (days)

Only sync changed records:

  • Track modified timestamp on entities
  • Sync only records changed since last sync
  • Performance improvement for large networks

Summary​

✅ Implemented:

  • NetworkSynchronizationService with comprehensive sync methods
  • Automatic sync before every simulation
  • Scenario-specific options handling
  • Error handling and logging
  • Audit of all sync methods

✅ Benefits:

  • Database is source of truth
  • Real-time updates reflected in simulations
  • No manual INP export required
  • Improved workflow efficiency

✅ Production Ready:

  • Error handling prevents crashes
  • Logging for troubleshooting
  • Backward compatible (INP still works)
  • Tested with multiple scenarios

⏸️ Future Work:

  • Time parameter string parsing
  • Consider topology updates if needed
  • Performance optimization for large networks

Related Documentation:

  • Implementation Plan: Available in archive (detailed step-by-step)
  • EPANET API Reference: See EpanetWrapper documentation
  • Database Schema: See architecture documentation

Status: ✅ Complete and Production Ready Last Updated: October 17, 2025

Last updated on by Jake, Cursor