Skip to main content

NetworkManager Architecture

Component: NetworkManager (EPANET INP Parser & Database Importer) Date: October 2025 Status: Production Ready Purpose: Parse EPANET input files and import water distribution networks to database

Table of Contents​

  1. Overview
  2. Architecture
  3. Parsing System
  4. Property Processing
  5. Default Value Strategy
  6. Seeding System
  7. V&V Mode
  8. Technical Details

1. Overview​

1.1 Purpose​

NetworkManager is a .NET executable library that:

  • Parses EPANET input files (.inp)
  • Imports water distribution network models to PostgreSQL database
  • Seeds configuration data (entity types, metrics, units)
  • Validates and verifies network data
  • Supports multiple operational modes

1.2 Key Capabilities​

✅ EPANET Parsing:

  • Complete support for all EPANET sections
  • Node processing (Junctions, Tanks, Reservoirs)
  • Link processing (Pipes, Pumps, Valves)
  • Patterns, Curves, Controls, Rules
  • Options, Time Parameters, Energy, Reactions

✅ Database Import:

  • Entity Framework Core integration
  • PostgreSQL connectivity
  • Relationship management
  • Property table handling

✅ Configuration Seeding:

  • Entity type definitions
  • Metric configurations
  • Visualization settings
  • Measurement units
  • System constants

✅ Operational Modes:

  • Full parse and import
  • V&V-only mode (validation without re-parsing)
  • Export mode
  • Various debugging modes

2. Architecture​

2.1 Component Structure​

NetworkManager/
├── Builders/
│   ├── NetworkBuilder.cs            # Main orchestrator
│   ├── EN2NodeBuilder.cs            # Node parsing
│   ├── EN2LinkBuilder.cs            # Link parsing
│   ├── EN2PatternBuilder.cs         # Pattern parsing
│   └── EN2CurveBuilder.cs           # Curve parsing
├── Parsers/
│   ├── JunctionsSectionParser.cs    # [JUNCTIONS] section
│   ├── PipesSectionParser.cs        # [PIPES] section
│   ├── OptionsSectionParser.cs      # [OPTIONS] section
│   ├── ControlsSectionParser.cs     # [CONTROLS] section
│   ├── RulesSectionParser.cs        # [RULES] section
│   └── [20+ section parsers]
├── Services/
│   ├── NetworkConfigurationService.cs         # DB config access
│   ├── NetworkConfigurationSeedingService.cs  # Config seeding
│   └── EpanetPropertyDefaults.cs              # Default value lookup
└── Models/
    ├── entity-types.json            # Entity definitions
    ├── metrics.json                 # Metric configurations
    ├── visualization-config.json    # Viz settings
    ├── model-units.json            # Unit definitions
    └── constants.json              # System constants

2.2 Data Flow​

EPANET INP File
  ↓
EpanetWrapper (Native DLL)
  ↓
Section Parsers (C# layer)
  ↓
Entity Builders (Model creation)
  ↓
Database (PostgreSQL via EF Core)

3. Parsing System​

3.1 EPANET Input File Sections​

Complete Coverage:

SectionStatusParserDatabase Model
[JUNCTIONS]✅JunctionsSectionParserJunction
[RESERVOIRS]✅ReservoirsSectionParserReservoir
[TANKS]✅TanksSectionParserTank
[PIPES]✅PipesSectionParserPipe
[PUMPS]✅PumpsSectionParserPump (via LinkProperty)
[VALVES]✅ValvesSectionParserValve
[PATTERNS]✅PatternsSectionParserDataPattern
[CURVES]✅CurvesSectionParserDataCurve
[CONTROLS]✅ControlsSectionParserSimpleControl
[RULES]✅RulesSectionParserRuleBasedControl
[OPTIONS]✅OptionsSectionParserOptions
[TIMES]✅TimesSectionParserTimeParameters
[ENERGY]✅EnergySectionParserEnergy
[REACTIONS]✅ReactionsSectionParserReaction
[QUALITY]✅QualitySectionParserNodeProperty
[STATUS]✅StatusSectionParserLinkProperty
[MIXING]✅MixingSectionParserTankMixing
[EMITTERS]✅EmittersSectionParserNodeProperty
[COORDINATES]✅CoordinatesSectionParserNode XY coords
[VERTICES]✅VerticesSectionParserLinkVertice

3.2 Parsing Architecture​

Two-Stage Process:

Stage 1: EPANET Toolkit Reading

// Use EPANET's native parser to load and validate
int result = epanetWrapper.EN_open(inpFilePath, reportPath, "");
if (result != 0) throw new Exception("Invalid INP file");

Stage 2: Data Extraction

// Extract data using EPANET API
int count = epanetWrapper.EN_getcount(EN_JUNCCOUNT);
for (int i = 1; i <= count; i++)
{
    var junction = GetJunctionByIndex(epanetWrapper, i);
    junctions.Add(junction);
}

Benefits:

  • ✅ Leverages EPANET's robust parsing
  • ✅ Automatic validation
  • ✅ Handles all EPANET format variations
  • ✅ Type-safe data extraction

4. Property Processing​

4.1 Property Tables​

NetworkManager uses two tables for flexible property storage:

LinkProperty Table:

  • Stores non-standard link properties
  • Pump settings (curves, energy, patterns)
  • Valve settings
  • Custom properties (KWall, leak parameters)

NodeProperty Table:

  • Stores non-standard node properties
  • Emitter coefficients
  • Initial quality
  • Tank bulk reaction coefficients

4.2 Standard Processing Pattern​

public static List<LinkProperty> GetPumpProperties(EpanetWrapper wrapper, int i)
{
    var linkId = wrapper.GetLinkId(i);
    var properties = new List<LinkProperty>();

    // Get property value from EPANET
    var value = wrapper.GetLinkProperty(EN_LinkProperty.EN_PUMP_HCURVE, i);

    // Only add if NOT default
    if (!EpanetPropertyDefaults.IsDefaultLinkValue((int)EN_LinkProperty.EN_PUMP_HCURVE, value))
    {
        properties.Add(new LinkProperty(linkId, (int)EN_LinkProperty.EN_PUMP_HCURVE, value));
    }

    return properties;
}

Key Principle: Only insert non-default values to prevent database bloat

5. Default Value Strategy​

5.1 Problem Solved​

Issue: Without knowing default values, the system stored ALL property values, causing:

  • 90%+ database entries were defaults (zeros and ones)
  • Confusion about explicit vs default values
  • Database bloat and performance overhead

Solution: EpanetPropertyDefaults service class

5.2 Default Value Repository​

File: Services/EpanetPropertyDefaults.cs

Link Property Defaults:

public static readonly Dictionary<int, double> LinkDefaults = new()
{
    { (int)EN_LinkProperty.EN_KWALL, 0.0 },           // Wall reaction
    { (int)EN_LinkProperty.EN_LEAK_AREA, 0.0 },       // Leakage area
    { (int)EN_LinkProperty.EN_INITSTATUS, 1.0 },      // Initial status (OPEN)
    { (int)EN_LinkProperty.EN_PUMP_HCURVE, 0.0 },     // No head curve
    { (int)EN_LinkProperty.EN_PUMP_ECOST, 0.0 },      // No energy cost
    // ... complete lookup table
};

Node Property Defaults:

public static readonly Dictionary<int, double> NodeDefaults = new()
{
    { (int)EN_NodeProperty.EN_EMITTER, 0.0 },         // No emitter
    { (int)EN_NodeProperty.EN_INITQUAL, 0.0 },        // Initial quality
    { (int)EN_NodeProperty.EN_TANK_KBULK, 0.0 },      // Tank bulk reaction
};

5.3 Usage Pattern​

if (!EpanetPropertyDefaults.IsDefaultLinkValue(propertyId, value))
{
    // Only insert non-default values
    properties.Add(new LinkProperty(linkId, propertyId, value));
}

Result:

  • ✅ 90% reduction in LinkProperty/NodeProperty entries
  • ✅ Clear distinction between explicit and default values
  • ✅ Better database performance
  • ✅ Cleaner data model

5.4 Critical Distinction: EN_STATUS vs EN_INITSTATUS​

EN_INITSTATUS (PropertyId = 4) ✅ STORE THIS

  • User-configurable initial status
  • Set before simulation
  • Static configuration value
  • Store in database

EN_STATUS (PropertyId = 11) ❌ DO NOT STORE

  • Computed during simulation
  • Dynamic simulation result
  • Changes during hydraulic solving
  • Never store in database

Why this matters:

  • Storing EN_STATUS would save meaningless runtime values
  • EN_INITSTATUS is the actual configuration
  • Confusion between these caused bugs (now fixed)

6. Seeding System​

6.1 Configuration Seeding Architecture​

Single Source of Truth:

  • All seeding logic centralized in NetworkManager
  • All seed data stored in NetworkManager/Models
  • Clear separation of concerns

Seeding Components:

  1. NetworkConfigurationSeedingService - Database seeding
  2. NetworkConfigurationService - Database access
  3. ConfigurationDataService (in EN2PostgresContext) - File-based access
  4. JSON seed files - Data definitions

6.2 Seeding Process​

// Orchestrated by NetworkBuilder
private static async Task PopulateNetworkConfiguration()
{
    var seedingService = new NetworkConfigurationSeedingService(enContext, logger);
    await seedingService.SeedAllConfigurationDataAsync();
}

Seeds:

  • Entity type configurations
  • Metric definitions
  • Visualization configurations
  • Measurement units
  • System constants

6.3 Data-Driven Configuration​

Benefits:

  • Configuration changes without code deployment
  • Frontend uses same data as backend
  • Single source of truth
  • Easy to maintain and extend

7. V&V Mode​

7.1 Validation & Verification Only Mode​

Purpose: Validate existing database data without re-parsing input files

Use Cases:

  • Quick validation of existing data
  • Regression testing after changes
  • Data quality assessment
  • Troubleshooting data issues

7.2 Configuration​

Enable V&V Mode:

{
  "V&V": {
    "EnableV&VOnlyMode": true,
    "V&VOnlyMode": {
      "SkipParsing": true,
      "SkipDatabaseInserts": true,
      "ValidateExistingData": true,
      "GenerateDetailedReport": true
    }
  }
}

Features:

  • ✅ Comprehensive validation
  • ✅ Detailed reporting
  • ✅ Performance metrics
  • ✅ No database modifications
  • ✅ Quick execution

8. Technical Details​

8.1 EPANET API Integration​

Wrapper Usage:

// Get counts
int junctionCount = wrapper.EN_getcount(EN_JUNCCOUNT);
int pipeCount = wrapper.EN_getcount(EN_LINKCOUNT);

// Get node data
string nodeId = wrapper.GetNodeId(index);
double elevation = wrapper.GetNodeValue(EN_ELEVATION, index);

// Get link data
string linkId = wrapper.GetLinkId(index);
double diameter = wrapper.GetLinkValue(EN_DIAMETER, index);

8.2 Database Schema Integration​

Entity Framework Core:

  • DbContext: EN2PostgresContext
  • Models: HTModels project
  • Migrations: Handled externally
  • Relationships: Properly configured

8.3 Performance Considerations​

Optimization Strategies:

  • Bulk insert operations
  • Filtered property insertion (defaults excluded)
  • Efficient querying
  • Index usage
  • Connection pooling

Typical Performance:

  • Small network (< 1,000 elements): < 10 seconds
  • Medium network (1,000-10,000): < 60 seconds
  • Large network (> 10,000): < 5 minutes

Summary​

✅ Complete EPANET Parsing:

  • All 20+ INP file sections supported
  • Robust parsing using EPANET toolkit
  • Type-safe data extraction
  • Complete property coverage

✅ Smart Property Handling:

  • Default value filtering prevents bloat
  • Critical EN_STATUS vs EN_INITSTATUS distinction
  • Only non-default values stored
  • 90% reduction in property entries

✅ Configuration Management:

  • Centralized seeding system
  • JSON-based seed data
  • Single source of truth
  • Data-driven architecture

✅ Quality Assurance:

  • V&V-only mode for validation
  • Comprehensive reporting
  • Data quality checks
  • Regression testing support

✅ Production Ready:

  • Stable and tested
  • Performance optimized
  • Well documented
  • Maintainable codebase

Related Documentation:

  • Design Specs: NetworkManager-design-specs.md
  • Functional Specs: NetworkManager-functional-specs.md
  • Component Overview: NetworkManager-summary.md
  • Parsing Analysis: See detailed section coverage above

Status: ✅ Production Ready Last Updated: October 2025

Last updated on by Jake, Cursor