Grafana Data Visualization

Installing Grafana in Ubuntu

note

The installation instructions changed per Sudhir's email on 4/26/2024.

Download Grafana here

gqc@gqc-Aspire-5749:~/Downloads$ sudo dpkg -i grafana-enterprise_10.4.2_amd64.deb
Selecting previously unselected package grafana-enterprise.
(Reading database ... 196296 files and directories currently installed.)
Preparing to unpack grafana-enterprise_10.4.2_amd64.deb ...
Unpacking grafana-enterprise (10.4.2) ...
Setting up grafana-enterprise (10.4.2) ...
Adding system user `grafana' (UID 130) ...
Adding new user `grafana' (UID 130) with group `grafana' ...
Not creating home directory `/usr/share/grafana'.
### NOT starting on installation, please execute the following statements to configure grafana to start automatically using systemd
 sudo /bin/systemctl daemon-reload
 sudo /bin/systemctl enable grafana-server
### You can start grafana-server by executing
 sudo /bin/systemctl start grafana-server

Use Grafana with TDEngine

1. Start taosAdapter which is required for TDEngine connection.

   sudo systemctl start taosadapter

   sudo systemctl enable taosadapter

   

2. Install tdengine plugin for grafana.

   • Option 1: Install Using terminal

     TDENGINE_DS_NAME=TDengine
     TDENGINE_CLOUD_URL=http://localhost:6041
     TDENGINE_USER=root
     TDENGINE_PASSWORD=taosdata
     bash -c "$(curl -fsSL https://raw.githubusercontent.com/taosdata/grafanaplugin/master/install.sh)"

   • Option 2: Install using GUI. For more details see here.

     1. Open http://localhost:3000.
     2. From the left toolbar, goto Configuration>Plugins.
     3. Search for TDEngine.
     4. Make a connection with default TDEngine values (or the values you set) Eg:
        • URL: http://localhost:6041
        • username: root
        • password: taosdata

3. Import the dashboard

See here for the steps on exporting a grafana dashboard.

1. Open grafana (Eg: http://192.168.0.28:3000/ replace the ip with address of grafana server) and goto dashboard > import.

   

2. Import the json content by either uploading the file or pasting the json content. (Find the timeseries and FFT dashboard here)

3. After uploading the file or pasting the json, Options window will come up. Select any environment variables from the next window. For example, following screenshot shows where to select the TDEngine datasource in the new environment.

4. Add the dashboard and view it.

5. Sample snapshots

   1. Dummy data visualization

   2. Actual sensor data

Use Grafana to compare Collimator and Matlab outputs

1. In Grafana goto Configuration>Data Sources.

2. Click Add datasource. And browse for TDEngine. Then use the following settings:

   • Name: TDengine-Cloud
   • Host: https://gw.us-east-1.aws.cloud.tdengine.com
   • Token: eb75d3ce4fe27e9b4c37b56bce53c90cf604ef8e

3. Import the dashboard from this json to grafana.

4. If all good you'll get this dashboard. Make sure you use the correct time range to see the data.

Exporting grafana dashboard

1. Goto your dashboard and select the share icon

2. In the popup window, select Export. Tick Export for sharing externally and click Save to file.

3. The saved JSON file can be imported to any Grafana instance. It will have the option to select the Datasource during the import process.

Setting up Grafna Alerts

Add alert conditions

1. Goto your grafana dashboard (local or cloud). Menu items and icons can be slightly different based on the Grafana version you use. But the steps should be similar.
2. Select Alerts>Alert rules>Create Alert Rule
3. Give a name to the rule. For example,
4. Setup the SQL query to obtain the data to run the rule on. You should see data on the table after setting the marked fields. Change the sql based on the node mac id.

   select ts, amplitude_gx from sensor_mesh.node_0c_b8_15_81_1e_e8_accel_latest  where ts > $from and ts < $to;
   
5. Then compare the max amplitude of the value series with the threshould as following.
6. For Alert Evaluation Behavior, Select a folder (if this is the first time you are setting up an alert, create a folder to save them into. Give the folder a name like Demo Alerts. And similarly create a group and name it like DemoGroup) Set up the rest of the fields in this section as below
7. Add details as necessary. For now I'll leave them as blanks.
8. In Notifications section, select the appropriate key (or add new key if doing for the first time) and values
9. Click Save, and you'll see the new rule in the Alerts page. It will show you the state of the alert as of now.

Setup the contact points

Contact points are where we will be sending the alert emails. Grafana cloud doesn't require any additional configuration as it has it's own mail server to send email alerts. With Grafana local installation we have to add details of our own SMTP server for email alerts to be sent. Following topic describes how to setup SMTP server on local Grafana.

Setting up SMTP server

Only required for local Grafana installations. Not required for Grafana cloud. Reference: How to setup grafana email alerts

danger

Free Gmail SMTP service might be stopped soon as they are phasing out less secure app access. Currently the Gmail SMTP server access is obtained using a workaround using an app password. We need to switch to a future proof SMTP service

1. For the demo we have created a gmail account to use as our SMTP server.

   deepvibegqc@gmail.com
   Password: DeepVibe2@22
   App password: rearexoiiqktlkit

2. We have to have the following information for the SMTP sever. The data shown here are for the gmail account, deepvibegqc@gmail.com we created.

   host = smtp.gmail.com:465
   user = deepvibegqc@gmail.com
   password = rearexoiiqktlkit
   from_address = deepvibegqc@gmail.com
   from_name = Grafana

3. Open the grafana.ini configuration file.

   sudo nano /etc/grafana/grafana.ini

4. Search for [smtp] to find the section corresponding to SMTP settings. Replace the content with following. Replace the field values if you are using a different SMTP server.

   [smtp]
   enabled = true
   host = smtp.gmail.com:465
   user = deepvibegqc@gmail.com
   # If the password contains # or ; you have to wrap it with triple quotes. Ex """#password;"""
   password = """rearexoiiqktlkit"""
   ;cert_file =
   ;key_file =
   skip_verify = true
   from_address = deepvibegqc@gmail.com
   from_name = Grafana
   # EHLO identity in SMTP dialog (defaults to instance_name)
   ;ehlo_identity = df_sensor_alert@sealmaticincia.com
   # SMTP startTLS policy (defaults to 'OpportunisticStartTLS')
   ;startTLS_policy = NoStartTLS

   Not that ; and # are both used to indicate start of a comment in the ini syntax.

5. Restart the grafana service.

   sudo service grafana-server restart
   sudo service grafana-server status

Add contact points

1. Goto your Grafana portal. Select Alerting from left menu and then goto Contact Points. Click Add contact point

2. Provide details for the new contact point.

   1. Give a name to the contact point group.
   2. Add list of emails where the alerts should be sent in the Addresses field, separated by ;s.
   3. Click on Test and see if you are recieved a test email. Check spam or junk folders in your mailbox if the mail doesn't appear in the inbox.
   4. Finally, click Save contact point.

3. Now you'll see the newly created contact point listed in the Contact points page.

4. Finally, we have to configure Grafana to send alerts from our set of rules to the newly created Contact Point.

   1. Goto Notification policies, under Alerting.
   2. Click edit button in root policy panel.
   3. Change the default contact point to the newly created contact point.

Enabling embedded images in the alerts

Created following this article.

On AMD64 Ubuntu machine

1. When setting up the alert make sure to include dashboard ID and the panel ID

2. Install the grafana-image-rendere plugin

   sudo grafana-cli plugins install grafana-image-renderer

3. In Grafana config file, enable unified_alerting and enable screenshots.

   1. Open the grafana config file.

      sudo nano /etc/grafana/grafana.ini`
   2. Search for [unified_alerting] and uncomment enabled line. So it should look like,

      [unified_alerting]
      #Enable the Unified Alerting sub-system and interface. When enabled we'll migrate all of your alert rules and notification channels to the new system. New alert rules will be created and your notification channels will be converted into an Alertmanager configuration. Previous data is preserved to enable backwards compatibility but new data is removed.
      enabled = true
   3. Search for [unified_alerting.reserved_labels]. Add the following lines above that.

      [unified_alerting.screenshots]
      capture = true
   4. Save and close the grafana config file.

4. Restart the grafana-service

   sudo systemctl stop grafana-server
   sudo systemctl start grafana-server

5. The resulting alerts would look like this:

On ARM64 (Raspberry-Pi)

danger

Couldn't get the workaround to run. Leaving the notes here in-case we want to follow up in future.

1. Tried direct installation of the image-render plugin. Not supported for ARM64

2. Followed a workaround:

   1. Create a new directory in git/public/grafana-render

   2. Run the following commands.

      sudo su
      wget https://nodejs.org/dist/v16.18.0/node-v16.18.0-linux-arm64.tar.xz
      tar xf node-v16.18.0-linux-arm64.tar.xz
      cd node-v16.18.0-linux-arm64
      cp -R * /usr/local/
      git clone https://github.com/grafana/grafana-image-renderer
      npm install --global yarn
      apt install chromium-browser
      cd grafana-image-renderer
      /usr/local/bin/yarn install --pure-lockfile
      /usr/local/bin/yarn run build

   3. Run the grafana server

      node build/app.js server --port=8081

   4. Add the following lines to the [rendering] section of /etc/grafana/grafana.ini

      [rendering]
      server_url = http://localhost:8081/render
      callback_url = http://localhost:3000/

      

Resetting Grafana Password

In case the login password is forgotten, following method can be used to reset the password. Reference

cd ~/git/gqc/linux_socket_app/grafana_config
sudo grafana-cli admin reset-admin-password admin