Table of Contents
In this article, we are going to see how we can integrate Google Cloud with Grafana. We will integrate Google Cloud SQL with Grafana and plot the metrics on Grafana. We will also look at how we can use Google Stackdriver as the data source in Grafana to expose the metrics of Google Cloud VM’s and platforms.
To use Grafana immediately, we will be using Hosted Grafana by MetricFire. Hosted Grafana by MetricFire allows you to import your GCP metrics immediately into a Graphite back-end with Grafana dashboards. MetricFire is low-cost and is built for scaling - MetricFire is a great solution for expanding your metrics monitoring and getting more out of monitoring.
All you need to do to use Hosted Grafana by MetricFire is sign up for the free trial. Get into the product and try and out!
You don’t need to have fully functional cloud services running with Google Cloud to understand this article, but it does assume that you have basic familiarity with Google Cloud.
About Google Cloud Platform
Google Cloud is one of the most popular platforms for building and deploying cloud applications. Google Cloud provides all the major features of cloud services such as Compute Engine, Storage, Containers, Virtual Machines, Cloud Storage, Virtual Private Cloud, Load Balancing, Content Delivery Network (CDN), Domain Name Service (DNS), Big Data, Queue Messaging and much more. Launched in 2008, Google Cloud has multiple data centers around the world.
Getting Started with Google Cloud Platform
If you have not done so already, sign up for a new Google Cloud account at Google Cloud Console. Google Cloud offers 3 months free with $300 worth of free credits to try out its products.
Once signed up and logged in, we are presented with the following Google Cloud Console homepage showing the overview of all the services which we are using. In our case, it's mostly empty as we have not started using any of its services yet.
Getting Started with Grafana
We will now provision Grafana and import our Google Cloud metrics. To check out how to get up and running with Grafana, go to our Getting started with Grafana docs page.
Setting up Grafana with MetricFire is the easiest way to get Grafana running. All you have to do is head into the free trial, set up your data source, and you’re good to go! The MetricFire docs explains exactly how to use add-ons to make it easier to integrate with GCP.
Next, we will provision Google Cloud SQL to monitor its health on the Grafana dashboard.
Getting Started with Google Cloud SQL
Cloud SQL is a relational database service offered by Google Cloud. It offers seamless replication and scalability so that users don't have to worry about those aspects while creating applications.
To provision a Cloud SQL instance, let’s head over to the SQL section and provision a Cloud SQL instance. Make sure to provide an Instance ID and a strong password, and select the region which is the closest to your geography. We can keep the database version as MySQL 5.7.
Once the Cloud SQL is provisioned, we will be able to see both public and private IP allocated to this instance.
Setting up Monitoring for Cloud SQL
Now our Cloud SQL instance is provisioned. We will configure it to make sure it is able to collect health metrics which we will be pulling in Grafana.
For this, we need to run the statistics collection agent on MySQL. We will be using the open-source library my2Collector for this purpose. Simply connect to the MySQL instance provisioned above and run the following command to install the agent.
mysql --user=root -pXXX < my2.sql
Integrating Cloud SQL with Graphite and Grafana
Now our Cloud SQL instance is ready and storing health metrics. It's time to switch to Grafana and configure our visualizations. Browse to the Grafana Url which we provisioned earlier and add a new MySQL data source as shown below.
MySQL Host IP Address will be the IP Address of our Google Cloud SQL instance which we provisioned in the earlier step. Specify the username and password which were specified for the Cloud SQL instance and hit Save.
Now the data source is added and it is time to configure the visualizations. Instead of creating our own dashboard from scratch, we will rely on the Grafana dashboard library to import the MySQL dashboard and customize it according to our needs. Click on Download JSON link and import the dashboard into the Grafana.
Once imported, you should see the statistics and health metrics related to the Cloud SQL instance such as Threads & Errors, Space Usage, SQL Commands/sec/Network, Queries/sec, Connections/sec, Query Cache Hit and others.
You can also set alerts on these dashboards if the value of the observed metrics falls out of the allowed range. For more information on how to create such advanced dashboards, have a look at our articles on Grafana Dashboards from Basic to Advanced Dashboards and Our Favorite Grafana Dashboards.
Using Google Stackdriver Data Source in Grafana
Google Stackdriver is the monitoring engine for applications hosted in Google Cloud. It offers metrics collection, log management, application performance management and out-of-the-box dashboards and views to monitor the Google Cloud-hosted platform and applications.
However, Google Cloud Stackdriver’s visualization and customization are not as powerful as Grafana. That’s why many users prefer Grafana as their monitoring and visualization solution even though Stackdriver is built-in into Google cloud.
Grafana ships with out of the box support for Stackdriver as the data source. The process to integrate Stackdriver with Grafana is fairly straightforward. First, we need to create a service account for our Google Cloud project. Once the service account file is generated, we can go to Grafana and upload it under the Stackdriver Datasource configuration.
For more detailed instructions, have a look at the article “Introducing Stackdriver as a data source for Grafana” in Google Cloud. These instructions are also available in Grafana documentation.
In this article, we saw how we can integrate our applications hosted in Google Cloud to Grafana. We also saw how to push metrics from Google Cloud SQL instance to Grafana and plot them into the dashboard.
Sign up for the MetricFire free trial to set up Graphite and build Grafana dashboards now. You can also book a demo and talk to the MetricFire team about how you can best set up your monitoring stack.