Skip to main content

NetworkManager Fixes - History

Component: NetworkManager Period: October 2025 Status: All Fixes Implemented and Verified Purpose: Document critical fixes and improvements

Table of Contents​

  1. Database Upsert Fixes
  2. Default Value Filtering
  3. Property Processing Fixes
  4. Code Quality Improvements

1. Database Upsert Fixes​

1.1 Node Coordinates Update (CRITICAL) 🔴​

Severity: CRITICAL - Data Corruption Status: ✅ FIXED

Problem: Update() methods for Junction, Tank, and Reservoir were missing updates to XCoord and YCoord properties. These properties are defined in the Node base class, but update methods only updated entity-specific properties. Node coordinates would NEVER update after initial import!

Impact:

  • ❌ Visualization showed wrong locations
  • ❌ Geometry calculations incorrect
  • ❌ Spatial queries failed
  • ❌ Re-import didn't fix coordinates

Solution:

// Added to Junction.Update(), Tank.Update(), Reservoir.Update()
public void Update(Junction other)
{
    // Base class properties
    this.XCoord = other.XCoord;    // ✅ ADDED
    this.YCoord = other.YCoord;    // ✅ ADDED

    // Entity-specific properties
    this.Elevation = other.Elevation;
    this.BaseDemand = other.BaseDemand;
    // ...
}

Files Modified:

  • efcorelibraries/HTModels/Models/EN2Models/Junction.cs
  • efcorelibraries/HTModels/Models/EN2Models/Tank.cs
  • efcorelibraries/HTModels/Models/EN2Models/Reservoir.cs

Result: ✅ Node coordinates now properly sync on every import

1.2 LinkVertex Composite Key Issue (CRITICAL) 🔴​

Severity: CRITICAL - Orphaned Data Status: ✅ FIXED

Problem: Vertices have composite primary key (LinkId, Sequence) but no auto-increment Id. GenericEntityProcessor couldn't properly find existing vertices, and removed vertices from INP file would persist in database.

Impact:

  • ❌ Deleted vertices remained in database
  • ❌ Modified vertices duplicated
  • ❌ Geometry became corrupted over time

Solution: Implemented delete-and-reinsert strategy per link:

// Delete all vertices for this link
var existingVertices = await context.LinkVertices
    .Where(v => v.LinkId == linkId)
    .ToListAsync();
context.LinkVertices.RemoveRange(existingVertices);

// Insert fresh vertices from INP
foreach (var vertex in newVertices)
{
    context.LinkVertices.Add(vertex);
}

Files Modified:

  • epanetconsolecore/NetworkManager/Parsers/LinkSectionParser.cs

Result: ✅ Vertices now properly sync with INP file on every run

1.3 DataCurve Array Update​

Severity: MEDIUM Status: ✅ FIXED

Problem: DataCurve stores X/Y values in PostgreSQL array columns (float8[]). Array replacement might not be properly detected by EF Core's change tracking.

Solution:

private void UpdateEntity(DataCurve existingCurve, DataCurve newCurve)
{
    // Explicitly replace arrays
    existingCurve.XValues = newCurve.XValues;
    existingCurve.YValues = newCurve.YValues;
    existingCurve.CurveType = newCurve.CurveType;
    existingCurve.ModifiedAt = DateTime.UtcNow;
}

Result: ✅ Curves update correctly when data points change

1.4 DataPattern Array Update​

Severity: MEDIUM Status: ✅ FIXED

Problem: Pattern stores multipliers in PostgreSQL array column. Same array tracking issue as DataCurve.

Solution:

private void UpdateEntity(DataPattern existingPattern, DataPattern newPattern)
{
    existingPattern.Multipliers = newPattern.Multipliers;
    existingPattern.Description = newPattern.Description;
    existingPattern.ModifiedAt = DateTime.UtcNow;
}

Result: ✅ Patterns update correctly when multipliers change

2. Default Value Filtering​

2.1 Database Bloat Problem​

Severity: MEDIUM - Performance Status: ✅ FIXED

Problem: LinkProperty and NodeProperty tables filled with default values (zeros and ones):

  • 90%+ entries were defaults
  • Database bloat and performance overhead
  • Confusion about explicit vs default values

Solution: Created EpanetPropertyDefaults service class:

public static class EpanetPropertyDefaults
{
    public static readonly Dictionary<int, double> LinkDefaults = new()
    {
        { (int)EN_LinkProperty.EN_KWALL, 0.0 },
        { (int)EN_LinkProperty.EN_LEAK_AREA, 0.0 },
        { (int)EN_LinkProperty.EN_INITSTATUS, 1.0 },  // OPEN
        { (int)EN_LinkProperty.EN_PUMP_HCURVE, 0.0 },
        // ... complete lookup
    };

    public static bool IsDefaultLinkValue(int propertyId, double value)
    {
        return LinkDefaults.TryGetValue(propertyId, out var defaultVal)
               && Math.Abs(value - defaultVal) < 1e-10;
    }
}

Usage Pattern:

// Only insert if NOT default
if (!EpanetPropertyDefaults.IsDefaultLinkValue(propertyId, value))
{
    linkProperties.Add(new LinkProperty(linkId, propertyId, value));
}

Files Modified:

  • Services/EpanetPropertyDefaults.cs (NEW)
  • Builders/EN2LinkBuilder.cs
  • Builders/EN2NodeBuilder.cs
  • All section parsers

Result: ✅ 90% reduction in LinkProperty/NodeProperty entries ✅ Database significantly leaner ✅ Clear distinction between explicit and default values

3. Property Processing Fixes​

3.1 Section Parser Default Filtering​

Severity: MEDIUM Status: ✅ FIXED

Problem: Section parsers inserted ALL values, even defaults:

  • QualitySectionParser - inserted INITQUAL=0 for ALL nodes
  • StatusSectionParser - inserted INITSTATUS=1 (OPEN) for ALL links

Solution: Added default filtering to both parsers:

QualitySectionParser (AFTER):

var initialQuality = epanetWrapper.GetNodeInitQual(i);

// Only create if NOT default (0)
if (!EpanetPropertyDefaults.IsDefaultNodeValue((int)EN_NodeProperty.EN_INITQUAL, initialQuality))
{
    var qualityProperty = new NodeProperty(nodeId, (int)EN_NodeProperty.EN_INITQUAL, initialQuality);
    nodeProperties.Add(qualityProperty);
}

StatusSectionParser (AFTER):

var initStatus = epanetWrapper.GetInitStatus(i);
var statusValue = ConvertStatusToNumeric(initStatus);

// Only create if NOT default (1=OPEN)
if (!EpanetPropertyDefaults.IsDefaultLinkValue((int)EN_LinkProperty.EN_INITSTATUS, statusValue))
{
    var statusProperty = new LinkProperty(linkId, (int)EN_LinkProperty.EN_INITSTATUS, statusValue);
    linkProperties.Add(statusProperty);
}

Result: ✅ Only non-default status and quality values stored

3.2 EN_STATUS vs EN_INITSTATUS (CRITICAL)​

Severity: CRITICAL - Wrong Data Stored Status: ✅ FIXED

Problem: StatusSectionParser used EN_STATUS (11) instead of EN_INITSTATUS (4).

Critical Distinction:

  • EN_INITSTATUS (4): User configuration - Initial status before simulation ✅ Store this
  • EN_STATUS (11): Simulation result - Current status during/after simulation ❌ Don't store this

Analogy:

  • EN_INITSTATUS = "thermostat setting" (configuration)
  • EN_STATUS = "current temperature" (runtime measurement)

Solution:

// BEFORE (WRONG):
new LinkProperty(linkId, (int)EN_LinkProperty.EN_STATUS, statusValue);  // ❌

// AFTER (CORRECT):
new LinkProperty(linkId, (int)EN_LinkProperty.EN_INITSTATUS, statusValue);  // ✅

Result: ✅ Correct property stored in database

3.3 Parser Entity Processor Fix​

Severity: MEDIUM Status: ✅ FIXED

Problem: GenericEntityProcessor had issues with:

  • Composite primary keys
  • Array property updates
  • Entity relationship handling

Solution:

  • Enhanced UpdateEntity methods for array properties
  • Implemented delete-reinsert for composite keys
  • Improved change tracking

Result: ✅ All entity types update correctly

3.4 Duplicate Metric Resolution​

Severity: LOW Status: ✅ FIXED

Problem: Metric seeding created duplicate entries due to lack of unique constraints.

Solution:

  • Added unique index on metric name
  • Check for existing before insert
  • Proper error handling

Result: ✅ No duplicate metrics in database

4. Code Quality Improvements​

4.1 Architecture Refactoring​

Before: Monolithic parsing in main method After: Section-based architecture with individual parsers

Benefits:

  • ✅ Easier to maintain
  • ✅ Easier to extend
  • ✅ Better separation of concerns
  • ✅ Reusable components

4.2 Standardization​

Default Value Handling:

  • ✅ Single source of truth (EpanetPropertyDefaults)
  • ✅ Consistent across all parsers
  • ✅ Well-documented defaults

Error Handling:

  • ✅ Consistent logging
  • ✅ Proper exception management
  • ✅ User-friendly error messages

Summary​

Total Fixes: 10+ critical and medium severity issues

Categories:

  • 🔴 Critical Fixes: 3 (Node coordinates, LinkVertex, EN_STATUS)
  • 🟡 Medium Fixes: 5 (Array updates, default filtering)
  • 🟢 Improvements: Multiple (code quality, architecture)

Impact:

  • ✅ Data integrity restored
  • ✅ Coordinates sync correctly
  • ✅ Vertices update properly
  • ✅ Database bloat eliminated
  • ✅ Correct properties stored
  • ✅ Better code maintainability

Result: Production-ready NetworkManager with reliable parsing, proper data handling, and maintainable architecture.

Related Documentation:

  • Architecture: NETWORKMANAGER_ARCHITECTURE.md
  • Reference: NetworkManager-Reference.md
  • Design Specs: NetworkManager-design-specs.md

Status: ✅ All Fixes Implemented Last Updated: October 2025

Last updated on by Jake, Cursor