Questions

The questions before the tracked timeline here are available in ai3-hackathon repository under docs/judges.

10/19/2023 - 10/20/2023

What payload formats would the MQTT messages contain? A MQTT message payload can be in any format, including a JSON string and a CSV, as shown in the following table:

Topic: weather_data (JSON)	Topic: weather_data (CSV)
{"dateutc": 1678488300000, "tempf": null, "humidity": 15, "windspeedmph": null,, ...... }	dateutc, tempf, humidity, windspeedmph, .....
{"dateutc": 1678488300010, "tempf": 35, "humidity": null, "windspeedmph": null,, ...... }	1678488300000, null, 65, 15, ....

The IoTData_EmergencyScenarios/fire contains payload in JSON format, whereas the PSIAP/EQ_data contains payload in a CSV format.

ans The MQTT message you show in the figure is correct. We will have one sensor for one topic. The message will be in JSON format with only timestamps and values.

What payload format(s) will be used at TEEX?
For NK MQTT topics, you mentioned in our last meeting that it would be a CSV payload. That would imply multiple rows of data are sent in a single MQTT message (as shown in the following image). Is this correct?

We have a dashboard on our laptop that updates in real-time. During the final round, would the laptop be connected to a projection system so that the judges can see the dashboard?
ans Yes, there will be a video system for showing the dashboard and the slides.
We are planning to show our labels for NK topics in our dashboard and we will be updating those labels in real-time. Would we be evaluated based on the final labels?
ans Yes, each team will be evaluated based on the final labels only.
We were told NK Topic ‘Values' will not be Boolean and will be numerical. Is there a probability of the values also being string characters? (E.g., 'normal', 'high')
ans You are right. The topics with categorical, Boolean, and string values will be only provided as background information. Only numerical data will be used for NK topics.
Our dashboard shows the received payloads in 2D and 3D GIS formats, for which we need geographical coordinates in the payloads. Can we expect such payloads?
ans Those will be provided as static metadata for the sensors. Those are not included in the payload.
Is there any estimate on the number of topics that will be provided for the final round, both known and NK?
ans We will reveal the exact number of topics on the data collection day.
Will the list of topics be given to us before the final round or only during the live demo?
ans The list will be provided on the data collection day (Nov 1st), one day before the live demo day.

09/28/2023 - Session with Jian Tao

We need the technical specifications of the proposed process for measuring accuracy now, so that we can implement it to benchmark our results.
- MQTT broker will stream Unknown data (NK) as individual topics, ranging about 10-15 topics. Each type will be a standalone unkown type, synthesized by resusing the numbers from known data.
- Sample on data stream is below:
- Each unknown data topic has data in the format given in the PSIAP github as follows. Jian said they will always be accompanied by timestamps or dates:
```
   Date;NK
   10/03/2004;1360
   10/03/2004;1292
   10/03/2004;1402
```
- The values of the unknown data are only numbers, not boolean and strings (need to confirm for strings, Jake understands this as no strings from what Jian said).
- Since all NK messages are in a topic, we can consider the Topic name as Topic ID.
- The data sent is most likely real data.
What is the Volume, Variety and Velocity of the messages used in the accuracy determination at TEEX?
- Assuming we have 10 streams, if we predict labels of 5 streams correctly, we get 50 points.
- Frequency between messages can be between 1- 5 seconds. And it's not gonna be the same for all topics.
“Quality of the processed unknown data for first responders” is based on ”usability”. How will the Judges determine the “usability”?
- Need to focus more on the Visualization. A dashboard of graphs could be useful.
- Find ways to bring it together with other datasets.
- Find how we can integrate unknown dataset that we've identified to existing datasets that are streaming.
- How will these be helpful to first responders? is something we need to come up with.
- VISUALIZATION WILL BE VERY IMPORTANT. Not every other team is focusing on that part. They're all really focusing on the models to identify and such. We also need to make it usable.

Additional Notes

They'll be anouncing the schedule soon.
They will be accomodating if you want to come earlier for a tour or anything.
They really want as many people from the teams joining as possible.
Do they cover travel expenses?

Questions - 09/07/2023

The questions are for Phase 3 with replies from Jian tao, on 09/07/2023.

Regarding the 'EQ_output.csv' (Earthquake) file provided for Phase2 Seismic data,
- For our testing, we have converted the CSV file to a JSON file (as shown in the following block), thus columns were converted to keys.
- Will the keys always be time and mag (or Mag)? The README provided defines "Mag" as "Ritcher scale reading of the earthquake (Please note the typo in Ritcher - which should be Richter).
```
   [
      {
         "time": "2023-03-10 00:54:48.811953408",
         "mag": 1.1026406882929414
      },
      {
         "time": "2023-03-10 02:41:46.728636672",
         "mag": 0.2346321366111436
      },
      {
         "time": "2023-03-10 02:43:26.880993280",
         "mag": 0.6309064255197545
      }
   ]
```
ans All the sensor data will have similar structure (time, value), even the unknown data will be streamed with time and value of the sensor data.
The Phase2 fire data provided as 'fire_alarm_data.json' has keys as device IDs and values as objects, which further have key : value pairs shown below.
```
   [{

      "time_stamp":1678406410187,

      "16":"{'smoke_sensor_status': 'normal', 'battery_state': 'high', 'muffling': False}",

      "22":"{'smoke_sensor_status': 'normal', 'battery_state': 'high', 'muffling': False}",

      "21":"{'smoke_sensor_status': 'normal', 'battery_state': 'high', 'muffling': False}"

   }]
```
- Should we be considering the Device IDs (16, 21, 22) also as keys?
  ans Each fire/smoke detector will be treated separately. The sensor ID will be treated as the metadata. The format you receive will be like Time, value
- Should the nested JSON structure also be considered for training and testing, or do we have a standard that will be followed in the live streaming data?
  ans For the smoke detector, only the smoke sensor status will be used. You will see something like the follow for each detector : 1678406410187, “normal”

The Phase2 weather data has NULL for many values (example is below) which might not provide great value for training. Should this be still included as part of the training?

   [{"dateutc": 1678502100000, "tempf": null, "humidity": null, "windspeedmph": null, "windgustmph": null, "maxdailygust": null, "winddir": null, "uv": null, "solarradiation": null, "hourlyrainin": null, "eventrainin": 258.012, "dailyrainin": null, "weeklyrainin": null, "monthlyrainin": null, "totalrainin": null, "battout": 1, "tempinf": 69.8, "humidityin": 53, "baromrelin": 30.197, "baromabsin": 29.672, "feelsLike": null, "dewPoint": null, "feelsLikein": 69.0, "dewPointin": 51.9, "lastRain": "2023-03-02T18:01:00.000Z"}]

ans We will control the percentage of missing values for the live event, but you should definitely expect missing values. We will split the message into multiple streams with one sensor data one message to better control of the quality of the data during the final event.

During the live streaming at the final event, What are the different message types we can expect to receive for a given scenario data?
- For example, given Earthquake data, Will this also include different JSON message types like Flood and weather data?
  ans For the live event, we will simulate an earthquake scenario. We will try to incorporate as many data sets as possible to provide a decent coverage of the datatypes that might be helpful for first responders and emergency management team. It is possible some relevant but not so useful streaming data will be shared to introduce some “noise”.
There are some typos in the README files provided for data in PSIAP repository. This can be an issue since we are using the key words to standardize from this README, which can affect the accuracy of the models.
- For e.g., Fahrenheit given as Farenheit, Richter as Ritcher, Synthetic Data as Synthetitc Data
  ans Thanks! I fixed those.
NK Keys:
- Data content:
  - What domain(s) can we expect the 'NK' keys to be from?
  - The 'Phase3 / unknown.csv' file contains the following data:
    Date; NK 10/03/2004;1360 10/03/2004;1292
  - We converted each row to a JSON message as follows:
    {"Date": "10/03/2004","NK": 1360} {"Date": "10/03/2004","NK": 1292}
  - Would a single JSON message contain only 1 NK key with other non-NK keys?
    ans Yes, a single JSON message contain only one NK key.
  - Would the a value corresponding to one NK key be 'NULL' in some cases?
    ans For missing data, NULL or an empty slot will be sent.
- Accuracy determination:
  - For our NK testing scenarios, we randomly select a key (with non-null value) and replace it with 'NK'. We then store the replaced key in 'changedKey' and append the key to the JSON object to be passed, in order to test the accuracy. A sample object is given below. With this test data, we are able to predict the NK keys with accuracy rate close to 100%.
    - Original:
      {"dateutc": 1678488300000, "dewPoint": null, "dewPointin": 55.4, "feelsLikein": 76.2, "lastRain": "2023-03-02T18:01:00.000Z"}
    - Modified:
      {"dateutc": 1678488300000, "dewPoint": null, "feelsLikein": 76.2, "lastRain": "2023-03-02T18:01:00.000Z", "NK": 55.4, "changedKey": "dewPointin"}
  - Given above data, how can we calculate the accuracy during the final round with live streaming data, especially when we do not possess the actual key values associated with the 'NK' keys?
    ans Natural language processing is inadequate in such scenarios. Some machine learning model will be needed to study the patterns of the data stream for a good matching.
  - [SK] Follow-up: [Logic explained on handling NK] + Our question was related to the mechanism that you will be using to measure the accuracy of our key classification approach. If you can share your proposed approach, we can use that to measure the accuracy and the quality of our approach.
    ans The accuracy will be measured by counting the correctly labeled unknown data sets out of the total number of unknown data streams. E.g., if you group successfully identify 5 out of 10 unknown streaming data, you will get 5 raw points, or something like that.
Should we explore expanding our training data outside phase 2 and phase 3 data provided?
ans Yes, the data sets were provided as samples. They are not sufficient for training a good ML model. External data sets will definitely be helpful.
Would there be more videos posted apart from the existing videos in the Google drive?
ans Yes, we will post more videos soon.

Questions

10/19/2023 - 10/20/2023​

09/28/2023 - Session with Jian Tao​

Additional Notes​

Questions - 09/07/2023​

10/19/2023 - 10/20/2023

09/28/2023 - Session with Jian Tao

Additional Notes

Questions - 09/07/2023