Steps to train CCTV defect classification models

danger

These are instructions shared by Vannary on 05/2023 and are obsolete as of 01/26/2024.
Check CCTV Workflow document for up-to-date instructions.

Create Data

1. Create a shortcut folder of new uploaded videos folder and place the folder in CCTV/<Utitlity>/Data/Uploaded_Data of the gdrive.

2. Run 0-zip-videos notebook to zip the videos into the group of 10 and store them in CCTV/<Utitlity>/Data/Videos and create a video list of each group and save to CCTV/<Utility>/Data/Video_Lists/

3. If you have an access to compute canada,

   1. Note: You cannot have more than 500,000 files in your project space. It is better to zip multiples into one zip file.
   2. Use rclone to copy those zip files and video lists csv files to the CCTV/<Utitlity>/Data/Videos/ and CCTV/<Utitlity>/Data/Video_Lists/ directory in Compute Canada
   3. Run sbatch shellrunner_extract_frames.sh -s vs_cctv_video_frame_extraction -c <video_group_name> -u <Utility> to extract every 30th frames from videos and save them as zip file to CCTV/<Utitlity>/Data/Extracted_Frames/. The run will also output zip file containing list of frames csv files and save to CCTV/<Utitlity>/Data/Extracted_CSV/.
      1. The processing time depends on the video size and video resolution. It can range from a few minutes to 5 hours per video group on Cedar cluster.
   4. Run sbatch shellrunner_Azure_OCR_6_stacked_full_image.sh -s colab_cctv_full_frame_stitch_and_azure_ocr -c <video_group_name> -u <Utility> to get JSON file of texts detected in the image and save to CCTV/<Utitlity>/Data/Azure_JSON/.
      1. The processing time depends on the image size and image resolution. The processing time can range from 1 to 10 hours per video group on Cedar cluster.
      2. You should not submit jobs for more than 30 video groups at once. You can get a disconnect error from Azure if you do too many multiple calls at once.
   5. Run sbatch shellrunner_blur.sh -s vs_cctv_images_save_blurred_frames -c <video_group_name> -u <Utility> to create blurred images of the extracted frames and save the outputs to CCTV/<Utitlity>/Data/Blurred_Frames/.
      1. The processing time depends on the image size and image resolution. It ranges from 10 min to 2 hours per video group on Cedar cluster.
   6. Extract middle frames and each video using sbatch shellrunner_mid_frames.sh -s mid_frame_extractor_from_videos -c <video_group_name> -u <Utility> save them as zip file to CCTV/<Utitlity>/Data/.
      1. Export the zip file to local compute and use streamlit app to classify video type.
      2. Upload the video type csv to gdrive
   7. Transfer outputs from steps 2 to 5 to gdrive using rclone.

4. If you run notebooks on colab,

   1. Note:
      1. We might need to modify these notebooks to read file from a zip file and to save the outputs as a zip file
      2. You can only run up to 5 notebooks at a time.
   2. Run CCTV_video_frame_extraction to extract every 30th frames from videos and save them as zip file to CCTV/<Utitlity>/Data/Extracted_Frames/. The run will also output zip file containing list of frames csv files and save the outputs to CCTV/<Utitlity>/Data/Extracted_CSV/
      1. The processing time depends on the video size and video resolution.
   3. Run colab-cctv-full-frame-stitch-and-Azure-OCR notebook to get JSON file of texts detected in the image and save the outputs to CCTV/<Utitlity>/Data/Azure_JSON/.
      1. This step has the largest processing time. The processing time depends on the image size and image resolution.
   4. Run CCTV_Images_Save_blurred_frames notebook to create blurred images from the extracted frames and save the outputs to CCTV/<Utitlity>/Data/Blurred_Frames/
   5. Run mid_frame_extractor_from_videos notebook to extract middle frame of each video.
   6. Use streamlit app to classify video type.

5. Run CCTV_Images_extract_labels notebook to create a zip file containing extracted labels csv file of each video.

   1. The processing time is < 1 hour per video group.

6. Create a metadata csv file from DBs provided by Utility. Refer to this page for more detail.

7. Add cross_reference_id to list of video csv files using this notebook. You might need to modify the notebook to work for different utilities.

8. Run create_image_labels notebook to annotate images.

   1. The processing time is < 1 hour per video group.

9. Run generate_data_for_k_fold_cross_val notebook to split data into train, validation, and test sets.

   1. Note: I will edit this notebook so it is easier for you to use.
   2. The processing time is < 1 hour per video group.