Skip to main content

Azure Blob Storage Configuration Guide

This guide explains how to configure the DWD system to use Azure Blob Storage for EPANET input files.

Overview​

The DWD system uses a shared published instance of EpanetConsoleCore.exe that runs scenarios for multiple networks. Since it's a single executable used by all networks, configuration cannot rely on appsettings.json files. Instead, all Azure Blob Storage configuration is passed via command-line arguments.

Architecture​

  • API (DwdApiAspNet): Reads Azure configuration from its appsettings.json and passes it to EpanetConsoleCore via command-line arguments
  • Console Core (EpanetConsoleCore): Receives Azure configuration via command-line arguments, which override any appsettings.json present

Configuration Steps​

1. Generate Container-Scoped SAS Tokens​

You need two separate SAS tokens with different permissions:

API Token (Read + Write)​

  • Permissions: Read, Write, Delete, List, Add, Create (rwdlac)
  • Expiry: Set far in the future (e.g., 1-2 years)
  • Protocol: HTTPS only

Console Core Token (Read-Only)​

  • Permissions: Read, List (rl)
  • Expiry: Set far in the future (e.g., 1-2 years)
  • Protocol: HTTPS only

Generate via Azure Portal:

  1. Navigate to your Storage Account → Container
  2. Select "Shared access tokens" (or use "Generate SAS" in the container menu)
  3. Set permissions, expiry, and protocol
  4. Click "Generate SAS token and URL"
  5. Copy the token (remove leading ? if present)

2. Configure API appsettings.json​

Location: dwd-api-aspnet/DwdApiAspNet/appsettings.json

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "AllowedHosts": "*",
  "InputFileStorage": {
    "ContainerName": "dwd-input-files-mtw",
    "AzureBlob": {
      "ServiceUrl": "https://yourstorageaccount.blob.core.windows.net",
      "ContainerSasToken": "sv=2023-01-03&ss=b&srt=c&sp=rwdlac&se=2026-12-31T23:59:59Z&st=2025-01-01T00:00:00Z&spr=https&sig=YOURSIGNATUREHERE"
    }
  },
  "ConnectionStrings": {
    "PSQLDB": "Host=localhost;Port=5432;Database=mtw_10_7_2025;Username=postgres;Password=gq010102"
  },
  "EpanetConsoleCore": {
    "Path": "C:\\git\\gqc\\hydrotrek-dwd-suite\\epanetconsolecore\\EpanetConsoleCore\\bin\\x64\\Release\\net8.0\\publish\\EpanetConsoleCore.exe"
  }
}

Important Notes:

  • Remove leading ? from SAS token if present
  • Use read + write permissions (sp=rwdlac)
  • Replace yourstorageaccount with your actual storage account name
  • Replace YOURSIGNATUREHERE with your actual SAS token signature

3. Console Core Configuration​

⚠️ IMPORTANT: EpanetConsoleCore does NOT use appsettings.json for Azure configuration.

All Azure Blob Storage configuration MUST be passed via command-line arguments when launching EpanetConsoleCore.

Why?

  • EpanetConsoleCore is a shared published executable used by multiple networks
  • Using appsettings.json would cause silent failures where the wrong network's configuration is used
  • Command-line arguments ensure each network's API instance passes its own configuration explicitly

If you forget to pass command-line arguments, EpanetConsoleCore will fail loudly with a clear error message explaining what's missing.

How It Works​

Command-Line Argument Passing​

When the API launches EpanetConsoleCore, it passes Azure configuration via command-line arguments:

EpanetConsoleCore.exe "Host=localhost;..." 1 --azure-url "https://account.blob.core.windows.net" --azure-sas "sv=2023..." --container "dwd-input-files-mtw"

Arguments:

  • Positional 1: Database connection string (required)
  • Positional 2: Scenario ID (required)
  • --azure-url: Azure Blob Storage service URL
  • --azure-sas: Container-scoped SAS token (without leading ?)
  • --container: Container name
  • --save-inp: (Optional) Export INP file without running simulation

Configuration Enforcement​

EpanetConsoleCore ONLY uses command-line arguments for Azure Blob Storage configuration.

There is NO fallback to appsettings.json. This is intentional to prevent:

  • Silent failures where the wrong network's configuration is used
  • Confusion about which configuration is being applied
  • Accidentally running scenarios with incorrect Azure storage

If Azure configuration is missing when needed, EpanetConsoleCore will fail immediately with a clear error message:

❌ AZURE BLOB STORAGE CONFIGURATION MISSING

This scenario uses Azure Blob Storage (StorageProvider=AzureBlob), but required command-line arguments are missing:

  • --azure-url
  • --azure-sas
  • --container

EpanetConsoleCore must be launched with Azure configuration:

  EpanetConsoleCore.exe "<connection-string>" <scenario-id> \
    --azure-url "https://account.blob.core.windows.net" \
    --azure-sas "sv=2023..." \
    --container "dwd-input-files-mtw"

💡 Note: This is a shared executable serving multiple networks.
   Configuration MUST be passed via command-line arguments, not appsettings.json.

Testing​

Test API Upload​

  1. Navigate to the Input Files page in the frontend
  2. Click "Upload File"
  3. Select an .inp file and upload
  4. Verify in Azure Portal that the blob appears in your container

Test Console Download​

  1. Create a scenario using a blob-stored input file
  2. Click "Start Scenario"
  3. Monitor logs - should see:
    Using blob-stored input file: your-file-name
    Container: dwd-input-files-mtw, Blob: your-file-20251023-123456.inp
    Downloaded <blob> to <temp-path>

Verify in Azure Portal​

  1. Navigate to your storage account
  2. Open the container (dwd-input-files-mtw)
  3. Verify files appear with naming format: {name}-{timestamp}.inp

Troubleshooting​

"No SAS token configured for container"​

Error: InvalidOperationException: No SAS token configured for container: dwd-input-files-mtw

Solutions:

  1. Verify InputFileStorage:AzureBlob:ContainerSasToken is present in API's appsettings.json
  2. Check for typos in configuration key names
  3. Ensure SAS token doesn't have leading ?

"The remote server returned an error: (403) Forbidden"​

Error: RequestFailedException: The remote server returned an error: (403) Forbidden

Solutions:

  1. Verify SAS token has correct permissions (rwdlac for API, rl for Console)
  2. Check SAS token hasn't expired
  3. Ensure container name matches exactly (case-sensitive)
  4. Verify protocol is HTTPS only

"Container must already exist when using SAS token"​

Error: InvalidOperationException: Container must already exist when using SAS token

Solutions:

  1. Manually create the container in Azure Portal before first use
  2. Ensure container name in config matches actual container name

Files not downloading/uploading​

Solutions:

  1. Check API logs for detailed error messages
  2. Verify ServiceUrl is correct format: https://accountname.blob.core.windows.net
  3. Test SAS token manually using Azure Storage Explorer
  4. Ensure firewall rules allow your IP address

SAS Token Security​

Best Practices​

  1. Use container-scoped tokens, not account-scoped
  2. Set reasonable expiry dates (1-2 years, not indefinite)
  3. Use HTTPS only (spr=https)
  4. Limit permissions:
    • API: Only grant what's needed (rwdlac)
    • Console: Read-only (rl)
  5. Rotate tokens periodically (e.g., annually)
  6. Never commit tokens to source control

Token Rotation​

When rotating SAS tokens:

  1. Generate new SAS tokens in Azure Portal
  2. Update API's appsettings.json
  3. Restart API service
  4. Verify operations work with new token
  5. Old tokens will expire automatically

No need to update published EpanetConsoleCore - it receives tokens from the API!

Multi-Instance Setup​

For hosting multiple networks, each API instance should have its own:

  1. Container: dwd-input-files-{network-name}
  2. SAS Token: Separate token per container
  3. appsettings.json: Configured for that specific network

Example for two networks:

MTW Instance (appsettings.json):

{
  "InputFileStorage": {
    "ContainerName": "dwd-input-files-mtw",
    "AzureBlob": {
      "ServiceUrl": "https://storage.blob.core.windows.net",
      "ContainerSasToken": "sv=2023...&sig=MTW_TOKEN_HERE"
    }
  }
}

XYZ Instance (appsettings.json):

{
  "InputFileStorage": {
    "ContainerName": "dwd-input-files-xyz",
    "AzureBlob": {
      "ServiceUrl": "https://storage.blob.core.windows.net",
      "ContainerSasToken": "sv=2023...&sig=XYZ_TOKEN_HERE"
    }
  }
}

Both instances can use the same published EpanetConsoleCore.exe!

Backward Compatibility​

The system still supports local file storage for migration scenarios:

  • StorageProvider=AzureBlob (1): Uses Azure Blob Storage (requires command-line args)
  • StorageProvider=LocalFileSystem (2): Uses local file system (no Azure config needed)

Important:

  • Scenarios using StorageProvider=AzureBlob will fail loudly if Azure configuration is not passed via command-line arguments
  • Scenarios using StorageProvider=LocalFileSystem will continue to work without any Azure configuration
  • There is NO silent fallback to appsettings.json - this is intentional to prevent configuration mistakes

References​

Last updated on by Jake, Cursor