Skip to main content

Azure Blob Storage - Configuration Guide

Version: 1.0 Date: October 20-21, 2025 Status: Production Ready Feature: EPANET Input File Cloud Storage Configuration

Table of Contents​

  1. Quick Reference
  2. Prerequisites
  3. NuGet Packages
  4. Configuration Steps
  5. Command-Line Arguments
  6. Security
  7. Troubleshooting

1. Quick Reference​

🎯 Key Principle​

EpanetConsoleCore NEVER uses appsettings.json for Azure configuration.

All Azure Blob Storage configuration MUST be passed via command-line arguments to prevent configuration conflicts in the shared executable environment.

✅ Quick Setup Checklist​

  1. Install NuGet packages
  2. Generate SAS tokens (2 tokens: read+write for API, read-only for Console)
  3. Configure API appsettings.json
  4. API automatically passes configuration to EpanetConsoleCore
  5. Test with a scenario

2. Prerequisites​

Azure Resources Required​

  1. Azure Storage Account

    • General Purpose v2 (GPv2) recommended
    • Standard or Premium tier
    • LRS or GRS replication

  2. Blob Container

    • One container per network/client recommended
    • Private access level (no anonymous access)
    • Container name: lowercase, alphanumeric, hyphens allowed

Network Requirements​

  • HTTPS access to *.blob.core.windows.net
  • Outbound port 443 open
  • No proxy restrictions (or configure proxy)

3. NuGet Packages​

3.1 API Project (dwd-api-aspnet/DwdApiAspNet)​

Required Packages:

<ItemGroup>
  <!-- Azure Blob Storage -->
  <PackageReference Include="Azure.Storage.Blobs" Version="12.19.1" />

  <!-- Azure Identity (optional, for Managed Identity) -->
  <PackageReference Include="Azure.Identity" Version="1.10.4" />
</ItemGroup>

Install via CLI:

cd dwd-api-aspnet/DwdApiAspNet
dotnet add package Azure.Storage.Blobs --version 12.19.1
dotnet add package Azure.Identity --version 1.10.4

3.2 EpanetConsoleCore Project​

Required Packages:

<ItemGroup>
  <!-- Azure Blob Storage -->
  <PackageReference Include="Azure.Storage.Blobs" Version="12.19.1" />
</ItemGroup>

Install via CLI:

cd epanetconsolecore/EpanetConsoleCore
dotnet add package Azure.Storage.Blobs --version 12.19.1

3.3 Package Details​

Azure.Storage.Blobs (v12.19.1)

  • Purpose: Azure Blob Storage client library for .NET
  • Features Used:
    • BlobServiceClient - Connect to storage account
    • BlobContainerClient - Manage containers
    • BlobClient - Upload/download blobs
    • BlobSasBuilder - Generate SAS tokens
    • StorageSharedKeyCredential - Account key authentication

Documentation: https://docs.microsoft.com/en-us/dotnet/api/azure.storage.blobs

4. Configuration Steps​

Step 1: Generate Container-Scoped SAS Tokens​

You need TWO separate SAS tokens with different permissions:

API Token (Read + Write)​

  • Permissions: Read, Write, Delete, List, Add, Create (rwdlac)
  • Expiry: 1-2 years (or use Azure Key Vault rotation)
  • Protocol: HTTPS only
  • Use Case: Upload, manage, and delete files

Console Core Token (Read-Only)​

  • Permissions: Read, List (rl)
  • Expiry: 1-2 years
  • Protocol: HTTPS only
  • Use Case: Download files for simulations

Generate via Azure Portal:

  1. Navigate to your Storage Account → Containers
  2. Select your container (e.g., dwd-input-files-mtw)
  3. Click "Shared access tokens" or "Generate SAS"
  4. For API Token:
    • Permissions: Select Read, Write, Delete, List, Add, Create
    • Start/End time: Set appropriate expiry
    • Protocol: HTTPS only
  5. For Console Token:
    • Permissions: Select Read, List only
    • Start/End time: Set appropriate expiry
    • Protocol: HTTPS only
  6. Click "Generate SAS token and URL"
  7. Copy the token (the part after ? in the URL)
  8. Remove leading ? if present

Example Token Format:

sv=2023-01-03&ss=b&srt=c&sp=rwdlac&se=2026-12-31T23:59:59Z&st=2025-01-01T00:00:00Z&spr=https&sig=SIGNATURE_HERE

Step 2: Configure API appsettings.json​

Location: dwd-api-aspnet/DwdApiAspNet/appsettings.json

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "AllowedHosts": "*",
  "InputFileStorage": {
    "ContainerName": "dwd-input-files-mtw",
    "AzureBlob": {
      "ServiceUrl": "https://yourstorageaccount.blob.core.windows.net",
      "ContainerSasToken": "sv=2023-01-03&ss=b&srt=c&sp=rwdlac&se=2026-12-31T23:59:59Z&st=2025-01-01T00:00:00Z&spr=https&sig=YOUR_API_TOKEN_HERE"
    }
  },
  "ConnectionStrings": {
    "PSQLDB": "Host=localhost;Port=5432;Database=your_database;Username=postgres;Password=your_password"
  },
  "EpanetConsoleCore": {
    "Path": "C:\\path\\to\\EpanetConsoleCore.exe"
  }
}

Important:

  • Remove leading ? from SAS token if present
  • Use the read+write token here (sp=rwdlac)
  • Replace yourstorageaccount with your actual storage account name
  • Replace placeholders with your actual values

Step 3: Console Core Configuration (Automatic)​

⚠️ CRITICAL: EpanetConsoleCore does NOT use appsettings.json for Azure configuration.

Why?

  • EpanetConsoleCore is a shared published executable used by multiple networks
  • Each network has its own API instance with its own configuration
  • Using appsettings.json would cause silent failures with wrong network's data
  • Command-line arguments ensure each API passes its own configuration explicitly

How it Works:

When the API launches EpanetConsoleCore for a scenario, it automatically passes Azure configuration via command-line arguments:

EpanetConsoleCore.exe "Host=localhost;Port=5432;..." 1 \
  --azure-url "https://account.blob.core.windows.net" \
  --azure-sas "sv=2023...&sp=rl&..." \
  --container "dwd-input-files-mtw"

You don't need to configure anything manually - the API handles this!

Step 4: Alternative - Local File System (Development)​

For development or testing without Azure, use local file system:

API appsettings.json:

{
  "InputFileStorage": {
    "Provider": "LocalFileSystem",
    "LocalFileSystem": {
      "BasePath": "C:\\HydroTrek\\InputFiles"
    }
  }
}

Note: This mode doesn't require Azure configuration or SAS tokens.

5. Command-Line Arguments Reference​

API Launches EpanetConsoleCore With:​

EpanetConsoleCore.exe <connection-string> <scenario-id> [Azure options]

Arguments Table​

ArgumentRequiredDescriptionExample
Positional 1✅ YesDatabase connection string"Host=localhost;Port=5432;Database=mtw;..."
Positional 2✅ YesScenario ID1
--azure-url⚠️ If AzureAzure Blob Storage service URL"https://account.blob.core.windows.net"
--azure-sas⚠️ If AzureContainer-scoped SAS token"sv=2023...&sp=rl&..."
--container⚠️ If AzureContainer name"dwd-input-files-mtw"
--save-inp❌ NoExport INP file path (no simulation)"C:\output\scenario_1.inp"

"If Azure" means: Required only if the scenario's InputFile.StorageProvider = AzureBlob

Example Command​

Full command the API generates:

C:\path\to\EpanetConsoleCore.exe \
  "Host=localhost;Port=5432;Database=mtw_db;Username=postgres;Password=pass;" \
  42 \
  --azure-url "https://mystorageaccount.blob.core.windows.net" \
  --azure-sas "sv=2023-01-03&ss=b&srt=c&sp=rl&se=2026-12-31T23:59:59Z&st=2025-01-01T00:00:00Z&spr=https&sig=SIGNATURE" \
  --container "dwd-input-files-mtw"

This command:

  1. Connects to database using connection string
  2. Loads scenario ID 42
  3. Uses Azure Blob Storage at the specified URL
  4. Authenticates with the SAS token
  5. Downloads input file from the container
  6. Runs simulation
  7. Stores results back to database

6. Security​

6.1 Token Security Best Practices​

DO:

  • ✅ Use container-scoped tokens (not account-level)
  • ✅ Set expiration dates on all tokens
  • ✅ Use read-only tokens for EpanetConsoleCore
  • ✅ Use HTTPS-only protocol
  • ✅ Store tokens in Azure Key Vault for production
  • ✅ Rotate tokens regularly
  • ✅ Monitor token usage and access logs

DON'T:

  • ❌ Share account keys with EpanetConsoleCore
  • ❌ Use account-level permissions
  • ❌ Store tokens in source control
  • ❌ Use tokens without expiration
  • ❌ Allow HTTP protocol
  • ❌ Give write permissions to simulation engine

6.2 Security Model​

API Tier:

  • Uses read+write SAS token
  • Full CRUD access to blob storage
  • Manages file uploads and deletions
  • Generates temporary download URLs

EpanetConsoleCore:

  • Uses read-only SAS token
  • Can only download files
  • Cannot upload, modify, or delete
  • Principle of least privilege

6.3 Container Isolation​

  • Each network/client gets dedicated container
  • Scoped access prevents cross-container access
  • SAS tokens limit permissions and duration
  • Network A cannot access Network B's files

6.4 Production Recommendations​

  1. Use Azure Key Vault

    {
      "InputFileStorage": {
        "AzureBlob": {
          "ServiceUrl": "@Microsoft.KeyVault(SecretUri=https://vault.vault.azure.net/secrets/BlobServiceUrl/)",
          "ContainerSasToken": "@Microsoft.KeyVault(SecretUri=https://vault.vault.azure.net/secrets/BlobSasToken/)"
        }
      }
    }

  2. Enable Managed Identity (removes need for tokens)

  3. Enable Blob Storage Logging

  4. Set up Monitoring and Alerts

  5. Implement Token Rotation

7. Troubleshooting​

Problem 1: "Azure Blob Storage Configuration Missing"​

Error:

❌ AZURE BLOB STORAGE CONFIGURATION MISSING

This scenario uses Azure Blob Storage (StorageProvider=AzureBlob),
but required command-line arguments are missing:
  • --azure-url
  • --azure-sas
  • --container

Solution:

  • Check API appsettings.json has correct Azure configuration
  • Verify API is passing command-line arguments
  • Check scenario's InputFile has StorageProvider = AzureBlob
  • Ensure EpanetConsoleCore is launched by the API, not manually

Problem 2: "Authentication Failed"​

Error:

Server failed to authenticate the request

Solution:

  • Verify SAS token is not expired
  • Check token has correct permissions (rl for Console, rwdlac for API)
  • Ensure token doesn't have leading ?
  • Verify token is for correct container
  • Check HTTPS-only protocol is enforced

Problem 3: "Container Not Found"​

Error:

The specified container does not exist

Solution:

  • Verify container name matches configuration exactly
  • Container names are case-sensitive
  • Check container exists in Azure Portal
  • Verify ServiceUrl points to correct storage account

Problem 4: "File Not Found in Blob Storage"​

Error:

Blob not found

Solution:

  • Check InputFile.BlobName is correct
  • Verify file was uploaded successfully
  • Check InputFile.StorageProvider is AzureBlob
  • Verify container name matches

Problem 5: "Permission Denied"​

Error:

This request is not authorized to perform this operation using this permission

Solution:

  • Check SAS token permissions
  • API needs rwdlac, Console needs rl
  • Verify token hasn't expired
  • Ensure correct token is being used (API vs Console)

Testing Checklist​

✅ Test Azure Connection:

  1. Upload a file via frontend
  2. Verify it appears in Azure Portal
  3. Download file via frontend
  4. Run a scenario using the uploaded file
  5. Check simulation completes successfully
  6. Verify temporary files are cleaned up

✅ Test Local Fallback:

  1. Configure Provider: "LocalFileSystem"
  2. Place files in configured BasePath
  3. Run scenarios with local files
  4. Verify simulations work

Configuration Summary​

Minimum Required Configuration​

API appsettings.json:

{
  "InputFileStorage": {
    "ContainerName": "your-container-name",
    "AzureBlob": {
      "ServiceUrl": "https://youraccount.blob.core.windows.net",
      "ContainerSasToken": "your-read-write-sas-token"
    }
  },
  "EpanetConsoleCore": {
    "Path": "path-to-EpanetConsoleCore.exe"
  }
}

NuGet Packages:

  • Azure.Storage.Blobs v12.19.1 (both projects)

Azure Resources:

  • Storage Account
  • Blob Container
  • 2 SAS Tokens (read+write for API, read-only for Console)

That's it! The API handles everything else automatically.

  • Design Document: design.md
  • API Reference: Available in Swagger UI
  • Frontend Guide: See design.md Section 5
  • Database Schema: See design.md Section 2.2

Status: ✅ Production Ready Last Updated: October 21, 2025 Version: 1.0

Last updated on by Jake, Cursor