Work Flow for CCTV
Intro
CCTV is complicated it has a lot of moving parts. This workflow is designed to help get started on CCTV and run things successfully.
Basic AI concept of 3-1-1: Takes all data and divides it into 5ths 3 of which is used to train the model. 1 of which is used to validate the training and 1 of it is to test the model.
I do not know where this spliting happens.
Running CCTV Projects
CCTV is Broken into 4 parts.
- Notebooks - fills in the database.
- Compute MSI Part 1: Training fastai_multi_label_v3_generic - Trains and does validates the AI model
- Compute MSI Part 2: Predictions fastai_multi_label_predict - Tests the AI model on clean dataset that has not been used as part of the training or validaiation.
- Streamlit Application - Views the Data
Virtual Environments
There are three Virtual Environments that are used on the MSI server for CCTV:
- gqc-utility-notebooks
- compute-msi
- cctv-apps
Repositories
- gqc-utility-notebooks
- compute-msi
- cctv-apps-streamlit
Running The Notebooks
For the Notebooks section your repository is gqc-utility-notebooks and you should use the virtual environment gqc-utility-notebooks
Notebooks are .ipynb files and are located in the file
If you are plaining on changing code in the notebooks you do not want to change the .py file you want to change the ipynb file and then run nbdev_export
Notes about Setting Up COV from CSV's
These notes are about the frames table.
If I use the CSVs then there is no data in the following columns.
When I look at the old database the one that we were refrencing in Streamlit.
TODO This needs to be moved to COV workFlow
cov_combined_92videofile_insepction_id_cross_reference_for_image_labelling videofile,pipe_id,inspection_info,cross_reference_id,video_name,video_type,sewer_use 000019-FJBYYX-D-2023-01-05.MP4,FJBYYX,FJBYYX-D-2023-01-05,5731,000019-FJBYYX-D-2023-01-05,COV-video_type_1,SW
cov_combined_92videofile_insepction_id_cross_reference videofile,pipe_id,inspection_info,Inspection ID 000019-FJBYYX-D-2023-01-05.MP4,FJBYYX,__FJBYYX-D-2023-01-05,5731.0
cov_combined_92videos_PACP_details_for_image_labelling cross_reference_id,Distance,Continuous,Code,Value_Percent 25197,0.0,,AMH,
cov_combined_92videos_PACP_details Condition ID,Source ID,Inspection ID,Distance,Counter,Defect Code,Continuous,Value of First Dimension,Value of Second Dimension,Rehab. Merge Width,Value Percent,Joint,Clock From,Clock To,Remarks,VCR Time,Active