Skip to main content

NetworkManager Data Validation & Verification (V&V) Plan

Overview​

This document outlines the comprehensive Data Validation & Verification (V&V) strategy for the NetworkManager EPANET input file parsing system. The V&V framework ensures data integrity, completeness, and correctness of parsed EPANET data.

V&V Objectives​

  1. Data Completeness: Verify all expected data is parsed and stored
  2. Data Accuracy: Ensure parsed values match EPANET input file values
  3. Data Consistency: Validate business rules and logical relationships
  4. Performance Monitoring: Track parsing performance and identify bottlenecks
  5. Regression Prevention: Detect changes that break existing functionality

V&V Framework Components​

1. Count Validation (Immediate Priority)​

Purpose​

Verify that all expected entities are parsed and stored in the database.

Implementation​

  • Add count validation to NetworkBuilder after parsing completion
  • Compare expected vs actual record counts
  • Log discrepancies for investigation

Validation Rules​

// Expected relationships
Nodes.Count == Quality.Count  // Every node should have quality record
Links.Count == Status.Count   // Every link should have status record
Junctions.Count >= Demands.Count  // Junctions may have demands
Tanks.Count >= TankMixing.Count   // Tanks may have mixing models
Pumps.Count >= Energy.Count       // Pumps may have energy settings

2. Data Range & Type Validation​

Purpose​

Ensure parsed values are within reasonable ranges and have correct data types.

Validation Categories​

A. Elevation Validation​
  • Range: -1000 to 10000 meters
  • Type: Numeric, not null
  • Business Rule: Reservoirs typically have higher elevations than tanks
B. Diameter Validation​
  • Range: 0.1 to 100 inches/meters
  • Type: Numeric, positive
  • Business Rule: Pipes must have diameter > 0
C. Emitter Coefficient Validation​
  • Range: 0 to 1000
  • Type: Numeric, non-negative
  • Business Rule: Only applies to junction nodes
D. Quality Values Validation​
  • Range: Any real number (can be negative for some parameters)
  • Type: Numeric, not NaN or Infinity
  • Business Rule: Values should be reasonable for water quality parameters

3. Business Logic Validation​

Purpose​

Validate EPANET-specific business rules and logical relationships.

Validation Rules​

A. Node Type Validation​
-- Tanks should have mixing models if specified
SELECT t."NodeId"
FROM "Nodes" t
JOIN "TankMixing" tm ON t."NodeId" = tm."TankId"
WHERE t."NodeType" != 'TANK';

-- Junctions should have demands if specified
SELECT j."NodeId"
FROM "Nodes" j
JOIN "Demands" d ON j."NodeId" = d."JunctionId"
WHERE j."NodeType" != 'JUNCTION';
-- Pumps should have energy settings if specified
SELECT p."LinkId"
FROM "Links" p
JOIN "Energy" e ON p."LinkId" = e."PumpId"
WHERE p."LinkType" != 'PUMP';

-- Pipes should have leakage if specified
SELECT l."LinkId"
FROM "Links" l
JOIN "Leakage" leak ON l."LinkId" = leak."LinkId"
WHERE l."LinkType" NOT IN ('PIPE', 'CVPIPE');
C. Status Value Validation​
  • Valid Values: "OPEN", "CLOSED", or numeric settings
  • Business Rule: Valves should have OPEN/CLOSED, pumps should have numeric settings
D. Pattern Validation​
  • Uniqueness: Pattern IDs should be unique
  • Completeness: All referenced patterns should exist

4. Data Quality Metrics​

Purpose​

Provide quantitative measures of data quality and parsing completeness.

Metrics​

A. Completeness Metrics​
-- Calculate percentage of complete records
SELECT
    'Node Elevation Completeness' as metric,
    ROUND((COUNT(CASE WHEN "Elevation" IS NOT NULL THEN 1 END) * 100.0 / COUNT(*)), 2) as percentage
FROM "Nodes";
B. Consistency Metrics​
  • Node-Link Connectivity: Verify all links reference existing nodes
  • Pattern References: Verify all pattern references exist
  • Curve References: Verify all curve references exist
C. Performance Metrics​
  • Parsing Time: Track time per section
  • Memory Usage: Monitor memory consumption
  • Database Performance: Track insert/update times

5. Regression Testing​

Purpose​

Detect changes that break existing functionality or data quality.

Implementation Strategy​

A. Baseline Comparison​
  • Store known good values for test files
  • Compare current results against baseline
  • Flag significant deviations
B. Automated Test Suite​
public class RegressionTestSuite
{
    public async Task<TestResult> RunRegressionTests(EN2PostgresContext.EN2PostgresContext context)
    {
        var results = new TestResult();

        // Test against known baseline
        await TestNodeCounts(context, results);
        await TestLinkCounts(context, results);
        await TestDataRanges(context, results);

        return results;
    }
}

6. Configuration-Driven Validation​

Purpose​

Make validation rules configurable and maintainable.

Configuration Structure​

{
  "Validation": {
    "EnableDataValidation": true,
    "EnablePerformanceMonitoring": true,
    "GenerateReport": true,
    "ValidationRules": {
      "CheckDataRanges": true,
      "CheckBusinessRules": true,
      "CheckCompleteness": true
    },
    "DataRanges": {
      "Elevation": { "Min": -1000, "Max": 10000 },
      "Diameter": { "Min": 0.1, "Max": 100 },
      "EmitterCoefficient": { "Min": 0, "Max": 1000 }
    },
    "PerformanceThresholds": {
      "MaxParseTimeMs": 5000,
      "MaxMemoryUsageMB": 500
    }
  }
}

Implementation Phases​

Phase 1: Immediate (Count Validation)​

  • Duration: 1-2 days
  • Scope: Basic count validation in NetworkBuilder
  • Deliverables:
    • Count validation logic
    • Basic logging and reporting
    • Integration with existing NetworkBuilder

Phase 2: Short-term (Data Range Validation)​

  • Duration: 1 week
  • Scope: Data range and type validation
  • Deliverables:
    • Range validation framework
    • Business rule validation
    • Enhanced error reporting

Phase 3: Medium-term (Comprehensive V&V)​

  • Duration: 2-3 weeks
  • Scope: Full V&V test suite
  • Deliverables:
    • Automated test framework
    • Performance monitoring
    • V&V report generation

Phase 4: Long-term (Regression Testing)​

  • Duration: 1-2 weeks
  • Scope: Regression testing framework
  • Deliverables:
    • Baseline comparison system
    • Automated regression tests
    • Continuous integration support

V&V Report Structure​

Report Sections​

  1. Executive Summary: Overall validation status
  2. Count Validation: Record count verification
  3. Data Quality: Range and type validation results
  4. Business Logic: Rule validation results
  5. Performance Metrics: Parsing performance data
  6. Recommendations: Suggested improvements

Sample Report Output​

=== NetworkManager Data V&V Report ===
Generated: 2024-01-15 10:30:00
Input File: example.inp
Validation Status: PASSED
Error Count: 0
Warning Count: 2

=== Count Validation ===
✅ Nodes: 150 (Expected: 150)
✅ Links: 200 (Expected: 200)
✅ Patterns: 5 (Expected: 5)
⚠️  Emitters: 3 (Expected: 5) - Missing 2 emitters

=== Data Quality ===
✅ Elevation Range: -50 to 500 (Valid)
✅ Diameter Range: 0.5 to 48 (Valid)
⚠️  Emitter Coefficients: 0 to 50 (Some values may be low)

=== Performance Metrics ===
Total Parse Time: 2.5 seconds
Average Section Time: 0.15 seconds
Memory Usage: 45 MB

Success Criteria​

Phase 1 Success Criteria​

  • Count validation implemented and working
  • Basic error logging functional
  • Integration with NetworkBuilder complete
  • No performance degradation

Phase 2 Success Criteria​

  • Data range validation working
  • Business rule validation functional
  • Enhanced error reporting
  • Configuration-driven validation

Phase 3 Success Criteria​

  • Comprehensive V&V test suite
  • Performance monitoring active
  • Automated report generation
  • 95% test coverage

Phase 4 Success Criteria​

  • Regression testing framework
  • Baseline comparison system
  • Continuous integration support
  • Zero regression issues

Risk Mitigation​

Technical Risks​

  • Performance Impact: Monitor parsing performance, optimize validation code
  • False Positives: Tune validation rules based on real data
  • Maintenance Overhead: Keep validation rules simple and well-documented

Process Risks​

  • Over-Validation: Focus on critical validations first
  • User Adoption: Provide clear error messages and guidance
  • Testing Coverage: Ensure comprehensive test coverage

Conclusion​

This V&V plan provides a structured approach to ensuring data quality and reliability in the NetworkManager system. By implementing validation in phases, we can achieve immediate benefits while building toward a comprehensive validation framework.

The focus on count validation as the immediate priority provides quick wins while establishing the foundation for more sophisticated validation capabilities.

Last updated on by Jake, Cursor