What is more exciting than staring at graphs? Nothing right. If you just love some system metrics or want to display the temperature in your home over time, a time series database like InfluxDB combined with Grafana is the way to go!

This guide will show you an easy way to get both of these systems up in no time with the help of Traefik and Docker. If you already have a server with Traefik running then continue. Otherwise I recommend you read my previous post and you'll be back here soon.

Setup

Let's make a directory for your data to live in, as sudo if needed

sudo su
mkdir /opt/influx

Then we need an internal docker network for the services to communicate on

docker network create influx

docker-compose.yml

We'll put our services in a docker-compose file to be able to easily manage things. We'll put the basics in first.

nano /opt/influx/docker-compose.yml

version: "2"
services:
  influxdb:
    # We'll add stuff here
  grafana:
    # and here

networks:
  default:
    external:
      name: influx # The network we created
  web:
    external: true # For traefik

InfluxDB

There are a number of environment variables available for the image of which we'll use a few.

If you want to send data to Influx from the outside world you need to have a frontend in Traefik with the port and hostname, if not then remove the labels section. You can also remove - web under networks.

We also need a user with write access and a database to write to. For some nice system metrics I use telegraf so here we'll have the InfluxDB image create that database and user initially. If you want to use something else as input just change the values accordingly.

  influxdb:
    image: influxdb:1.5.4-alpine
    restart: unless-stopped
    ports:
      - "127.0.0.1:8086:8086"
    volumes:
      - ./influxdb:/var/lib/influxdb
    environment:
      - INFLUXDB_DB=telegraf 
      - INFLUXDB_USER=telegraf
      - INFLUXDB_USER_PASSWORD=secretpassword
    labels:
      - "traefik.enable=true"
      - "traefik.frontend.rule=Host:influxdb.example.com"
      - "traefik.port=8086"
      - "traefik.docker.network=web"
    networks:
      - web
      - default # The external "influx" network

☝️ Remember to replace example.com in traefik.frontend.rule.

Grafana

Now let's add the Grafana service so that we have something to display our future stats with. First let's add a volume for persistent storage.

docker volume create grafana-storage

Then add this to docker-compose.yml between influxdb and networks.

  grafana:
    image: grafana/grafana:4.4.1
    restart: unless-stopped
    ports:
      - "127.0.0.1:3000:3000"
    volumes:
     - grafana-storage:/var/lib/grafana
     - ./log/grafana:/var/log/grafana
    environment:
     - GF_SECURITY_ADMIN_PASSWORD=supersecret
     # Comment out if you have users that should sign up
     - GF_USERS_ALLOW_SIGN_UP=false 
     - GF_SERVER_ROOT_URL=https://grafana.example.com
    labels:
      - "traefik.enable=true"
      - "traefik.frontend.rule=Host:grafana.example.com"
      - "traefik.port=3000"
      - "traefik.docker.network=web"
    networks:
      - web
      - default

volumes:
  grafana-storage:
    external: true

☝️ Remember to replace grafana.example.com in the 2 places above.

Grafana UI

Now you should be able to run the services!

# To start
docker-compose up -d
# Check logs
docker-compose logs
# Check processes
docker-compose ps

If everything looks alright, open up your Grafana URL and you should be greeted with a login screen. Use admin and the password you set as GF_SECURITY_ADMIN_PASSWORD. After logging in click "Add data source" and fill out the fields like this.

grafana-settings

The hostname influx_influxdb_1 in the Url setting is from the docker network influx, since the services are in separate containers you can't use localhost here.

If you're getting error messages when saving/testing the data source and using Safari, try another browser.

Getting some data in there

If you already have a way to send data to Influx, then you're done! If not then previously mentioned telegraf is just as easy to set up.

mkdir /opt/telegraf
cd /opt/telegraf
nano docker-compose.yml

version: "2"
services:
  telegraf:
    image: telegraf:1.5.2
    restart: unless-stopped
    volumes:
      - ./telegraf.conf:/etc/telegraf/telegraf.conf:ro
      # For docker stats
      - /var/run/docker.sock:/var/run/docker.sock:ro
    labels:
      - "traefik.enable=false"
    networks:
      - default

networks:
  default:
    external:
      name: influx

nano telegraf.conf

Here's a very simple conf just to get up and running. Feel free to use the default one for more options.

# Configuration for telegraf agent
[agent]
  interval = "10s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = ""
  debug = false
  quiet = false
  logfile = ""
  hostname = "MYHOST"

###############################################################################
#                            OUTPUT PLUGINS                                   #
###############################################################################

# Configuration for influxdb server to send metrics to
[[outputs.influxdb]]
  urls = ["http://influxdb:8086"] # required
  database = "telegraf" # required
  username = "telegraf"
  password = "secretpassword"
  retention_policy = ""
  write_consistency = "any"
  timeout = "5s"

###############################################################################
#                            INPUT PLUGINS                                    #
###############################################################################

# Read metrics about cpu usage
[[inputs.cpu]]
  ## Whether to report per-cpu stats or not
  percpu = true
  ## Whether to report total system cpu stats or not
  totalcpu = true
  ## If true, collect raw CPU time metrics.
  collect_cpu_time = false

# Read metrics about disk usage by mount point
[[inputs.disk]]
  ## Ignore some mountpoints by filesystem type. For example (dev)tmpfs (usually
  ## present on /run, /var/run, /dev/shm or /dev).
  ignore_fs = ["tmpfs", "devtmpfs"]

# Read metrics about disk IO by device
[[inputs.diskio]]
  ## Setting devices will restrict the stats to the specified devices.
  # devices = ["sda", "sdb"]

# Get kernel statistics from /proc/stat
[[inputs.kernel]]
  # no configuration

# Read metrics about memory usage
[[inputs.mem]]
  # no configuration

# Get the number of processes and group them by status
[[inputs.processes]]
  # no configuration

# Read metrics about swap memory usage
[[inputs.swap]]
  # no configuration

# Read metrics about system load & uptime
[[inputs.system]]
  # no configuration

# # Read metrics about docker containers
[[inputs.docker]]
  endpoint = "unix:///var/run/docker.sock"
  timeout = "5s"

# # Read metrics about network interface usage
[[inputs.net]]
  interfaces = ["eth*"]

Start the service and you'll have some data coming in.

docker-compose up -d
docker-compose logs

Now go back to your Grafana instance and create some dashboards!

Grafana sys

☝️ Telegraf sysstats by broferek

The End

If you noticed any factual errors, spelling mistakes (yes), best practises I'm not aware of or run into any trouble please comment below or hit me up on Twitter 🙌