Deploying Grafana to Kubernetes

Introduction

Grafana is an open-source platform for metric analytics, monitoring, and visualization. In this article, we will explore the basics of Grafana and learn how to deploy it to Kubernetes. You will find specific coding examples and screenshots you can follow to deploy Grafana. 

The Basics of Grafana

Before we dive into our code, you should be familiar with what Grafana can do for you. Grafana pulls up metrics from different data sources. For each source, it has a specific syntax and query editor that extracts and interprets data. 

Grafana supports multiple data sources like Prometheus, Mysql, MSSQL, PostgreSQL, Influx DB, Graphite, and ElasticSearch. It can also load data from some cloud-managed services like Google Stackdriver, Azure Monitor, and AWS CloudWatch. With the right plugin, you can even extend Grafana and add other data stores and online sources. Time series data for infrastructure and applications (such as disk I/O utilization, CPU, and memory) is first loaded into the analysis tool, then analyzed. 

Furthermore, Grafana allows you to easily create and edit dashboards. If you're not interested in installing your own Grafana, you can use Grafana through MetricFire's free trial.

Creating the Kubernetes Cluster

To better understand how Grafana works, we are going to use it to monitor a Kubernetes cluster. You can use any managed cloud cluster or even Minikube. In this tutorial, we are going to use GKE (Google Kubernetes Engine), the managed Kubernetes service of Google Cloud. If you are using GKE, your Google Cloud project should be linked to a billing account. You should also enable the Kubernetes API and install Google Cloud SDK. 

If you are going to use another cloud provider, you can follow almost the same steps shown here, except for some commands specific to Google Cloud Platform (GCP). 

Once installations and configurations are done, you can proceed by using:

gcloud container clusters create mycluster --zone europe-west1-b

Change "mycluster" for a cluster name of your choice and make sure to use your preferred zone.

NAME      LOCATION        MASTER_VERSION   MASTER_IP     MACHINE_TYPE   NODE_VERSION   NUM_NODES  STATUS
mycluster europe-west1-b   1.13.11-gke.9   34.77.193.73  n1-standard-1  1.13.11-gke.9  3          RUNNING

Alternatively, if you are using a Minikube cluster, you should first install it by following the official instructions. Then, create a cluster using: 

minikube start 

Now that the cluster is up and running, we need to install the Kubernetes command-line tool, kubectl. The installation is quite simple. If you are using Debian/Ubuntu, you need to run these commands:

sudo apt-get update && sudo apt-get install -y apt-transport-https

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update

sudo apt-get install -y kubectl

Create a Grafana Service

We are going to use the official Grafana image (v5.4.3) from Docker Hub to create a deployment.

kubectl create deployment grafana --image=docker.io/grafana/grafana:5.4.3

You can check if the image was deployed by using:

kubectl get deployments

You should see an output similar to the following: 

NAME     READY     UP-TO-DATE     AVAILABLE     AGE

grafana   1/1            1                         1                      66s

In order to access the Grafana dashboard, its service should be reachable from outside the cluster. However, after creating the deployment, Grafana is only accessible from inside the cluster because it uses a ClusterIP. 

You can verify this using:

kubectl get svc

You will see an output to: 

NAME          TYPE          CLUSTER-IP          EXTERNAL-IP          PORT(S)          AGE

kubernetes   ClusterIP    10.27.240.1            <none>                      443/TCP          100m

Next, we are going to expose the Grafana service using the LoadBalancer service type:

kubectl expose deployment grafana --type=LoadBalancer --port=80 --target-port=3000 --protocol=TCP

3000 is the default HTTP port that Grafana listens to if you haven't set up a custom configuration. For the sake of simplicity, we are going to map this internal port to the external port 80 using TCP. The load balancer creation may take a few minutes. To verify that the deployment was exposed, type the following:

kubectl get service grafana

You should be able to see the external IP of the load balancer: 

NAME          TYPE                CLUSTER-IP          EXTERNAL-IP          PORT(S)           AGE

grafana        LoadBalancer   10.27.252.155         35.205.13.61            80:32471/TCP   92s

You can also execute other operations on the deployed service, like scaling:

kubectl scale deployment grafana --replicas=2

Now, if you visit the same IP on the default port (80), you will see Grafana’s authentication page. 

Testing Grafana

The default login/password is admin/admin. You will be asked to change these credentials after your first login. You can also login to the free trial on MetricFire and start directly from this point, without having to do any set up.

To test our Grafana instance, we can add a data source. The simplest is adding a Grafana TestData source, a fake data source for testing purposes that makes verifying Grafana features easier and faster.

Click ‘Create your first data source’ and choose ‘TestData DB’, then click ‘Save and test’.  

Now, click  ‘Create a new dashboard’.

To see a panel with test data visualization, you can choose “Graph” or “Heatmap”, among others. 

Extending Grafana

Using plugins, we can extend our default installation. There are many plugins installed by default, but you can add more depending on what you need. You can see more about building Grafana dashboards and plugins on our advanced Grafana tutorial.

There are 3 types of plugins: Panels, Data Source, and Apps. In order to install a plugin like Grafana Kubernetes App, you need to execute this command:

grafana-cli plugins install grafana-kubernetes-app

Then, restart the Grafana service using:

service grafana-server restart

Since we use containers, the command to execute in our context is: 

kubectl exec -it  grafana-96fd979c-kjfhr  grafana-cli plugins install grafana-kubernetes-app

You should, of course, change grafana-96fd979c-kjfhr for the name of your pod. Once the app is installed, instead of restarting the service, we can just scale the deployment to 0,  then to 1 or more replicas. 

kubectl scale deployment grafana --replicas=0

kubectl scale deployment grafana --replicas=1

Because storage in containers is ephemeral, once pods are scaled or killed, our installation will get lost. In the next section, we will show you how to avoid this problem by using persisting Grafana storage. 

Persisting Grafana Storage

Some of the Grafana data needs to persist in our cluster, even when we do operations like scaling, restarting, or killing pods. 

First, we need to create a Grafana deployment that uses a persistent volume:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: grafana
  name: grafana
spec:
  replicas: 1
  selector:
    matchLabels:
      app: grafana
  template:
    metadata:
      labels:
        app: grafana
    spec:
      containers:
      - image: grafana/grafana:5.4.3
        name: grafana
        ports:
        - containerPort: 3000
          name: http

        volumeMounts:
          - name: grafana-storage
            mountPath: /var/lib/grafana
      volumes:
        - name: grafana-storage
          persistentVolumeClaim:
            claimName: grafana-storage
      securityContext:
        runAsNonRoot: true
        runAsUser: 65534
        fsGroup: 472

The above deployment uses the volume /var/lib/grafana. 

Next, create the persistent volume claim:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: grafana-storage
spec:
  accessModes:
- ReadWriteOnce
  resources:
requests:
storage: 1Gi

You can adapt the different configurations to your needs. 

After creating the volume claim and the deployment, we need to create the load balancer service to expose the deployment:

apiVersion: v1
kind: Service
metadata:
  name: grafana
  labels:
    app: grafana
spec:
  type: LoadBalancer
  ports:
  - port: 3000
    protocol: TCP
    targetPort: http
  selector:
    app: grafana

Our Kubernetes manifests files are stored in grafana-deployment.yaml, grafana-pvc.yaml and grafana-service.yaml, respectively. 

The final step is creating our Kubernetes objects:

kubectl create -f grafana-deployment.yaml -f grafana-pvc.yaml  -f grafana-service.yaml 

At this step, the cluster is reachable on the load balancer IP on port 3000. You can repeat the Kubernetes app installation procedure to make sure that the persistence is working.

kubectl exec -it  grafana-96fd979c-kjfhr  grafana-cli plugins install grafana-kubernetes-app

Conclusion

Deploying Grafana to a Kubernetes cluster is very simple. In this post, we have created a step-by-step guide that helps make the process seamless for you. Grafana’s versatility and flexibility allows you to adapt it to your specific needs, making it an excellent choice if you need to monitor, analyze, or visualize data. In addition to Kubernetes, Grafana offers Cloud and Prometheus monitoring, all available to you in a modern platform that is both affordable and easy to use.

Try out our free trial and start making Grafana dashboards right away. Also, feel free to book a demo if you have questions about what Grafana can do for you.