Skip to main content

Diagrams

One-shot Geometry Creation

One-shot History Insertion

This is for the 1 ten year file and 1 two year file. This took about 16 hrs. to finish on Deven's Laptop.

Hourly Data upsertion on a daily basis

We are using Paramiko a python implementation of SSHv2

One-shot Daily Table Calculation/Insertion

Creation of sql dump

Note: The sql dump is created after the insert into daily_riverflows is successful.

Restore on VM

Note: The size of the entire DB is around 58GBs. The SQL dump is 10GBs in size. Make sure there is sufficient storage available on the VM before starting the restore. Takes about 4 hrs to finish on the Azure VM

After a brief discussion with Jake (proposed refactor)

One-shot Geometry Creation
One-shot History Insertion
- This is for the 1 ten year file and 1 two year file. This took about 16 hrs. to finish on Deven's Laptop.
Hourly Data upsertion on a daily basis
- We are using Paramiko a python implementation of SSHv2
One-shot Daily Table Calculation/Insertion
Creation of sql dump
- Note: The sql dump is created after the insert into daily_riverflows is successful.
Restore on VM
- Note: The size of the entire DB is around 58GBs. The SQL dump is 10GBs in size. Make sure there is sufficient storage available on the VM before starting the restore. Takes about 4 hrs to finish on the Azure VM
After a brief discussion with Jake (proposed refactor)