Using the Graphite API for Beginners

Introduction

Application Programming Interface, or API, is an important element to consider when you design an online product or platform. Well-designed API is equivalent to a well-designed user interface for enterprise customers. When you implement an intuitive and easy-to-use interface, your product can retain and attract more users. Likewise, when you have an efficiently structured API, your B2B customers can more comfortably interact with your software product, and this can be a good value-added service for both parties.

   

Among many areas, in the monitoring domain, API works as an essential gateway for users to retrieve metrics data. This article discusses the benefits of using the API of the widely-used monitoring software - Graphite.

      

Key Takeaways

1. Well-designed APIs are crucial for online products, offering value-added services for enterprise and B2B customers.
2. Graphite is a widely-used open-source monitoring software with scalable real-time graphing capabilities.
3. The Graphite API allows easy programmable interactions, avoiding the need for SQL databases and supporting Python 2 and 3.
4. Use Graphite API to integrate third-party services and monitoring metrics into existing systems.

 

Why should you use the Graphite API?

To understand the advantages of using Graphite API, we first need to know the major functionalities of the monitoring software. When you become familiar with Graphite, you can have a good grasp of what you can achieve and what benefits you can gain by using the Graphite API.

     

What is Graphite?

Graphite is an open-source monitoring software with the aim of performing reliably on even affordable hardware or cloud infrastructure. Companies use Graphite to track the performance of their online resources such as websites, applications, network servers, and more. It is known to provide a scalable solution for real-time graphing. You can write an application that digests numeric time-series data and sends it to Graphite’s processing backend - Carbon, which stores the data in its database. Then, you can visualize the data on Graphite’s web interfaces.

       

Major Features

Graphite is known for the following major features.

• Simple architecture and great performance.
• Time-series metrics in intuitive graphs and charts.
• Relatively easier to learn compared to other monitoring tools.
• Big community support and is widely used.
• Suits diverse use cases such as analytics, DevOps, prediction, and more.

      

Reasons to use the Graphite API 

In addition to the major features of Graphite, it comes with the Graphite API for developers to programmatically interact with the monitoring software. Let’s find out what benefits users can have by using the Graphite API.

   

• The Graphite API application is stateless and is free from installing a SQL database on your server, which can significantly reduce maintenance efforts. It only needs to communicate to a time-series database.
• Python 2 and 3 are both compatible.
• The HTTP API can accept JSON data to provide flexibility to users for data requests and querystring parameters.
• The API application is straightforward to install and configure.
• The architecture has been extremely simplified and it consists of many fewer building blocks than the web-based Graphite.
• No need to integrate memcash - rendering is live.
• No support for the Pickle format for rendering.
• It has a plugin-based architecture to integrate with time-series databases or to add more analysis features.
• The codebase was updated with a focus on test coverage and code quality.

     

When do you want to use Graphite API over Graphite

One of the main reasons to use API is to integrate a third-party service into your existing IT stack. Let’s imagine that you have a dashboard tool and you want to visualize Google Analytics data to expose traffic metrics. To do so, you will have to retrieve traffic data via Google Analytics API and store the data in your database. Likewise, if you use a third-party dashboard interface (not the one from the web-based Graphite), Graphite API can be a good option to integrate Graphite monitoring metrics into your existing systems. If you already use the built-in Graphite composer, the API may not be suitable for you.

       

          

How To get started with the Graphite API

Graphite is open-source software, to start using it, users need to go through installation and configuration processes.

      

Installation

If your operating system is on Debian 8 or Ubuntu 14.04 LTS, you can use the available packages that are a self-contained build of graphite-api. You can download packages on the graphite-api release. When you finish installation, Graphite-api will run as a service and become available on port 8888. The packages you install include all the optional dependencies.

      

Python package

Graphite-api requires Python packages. The requirements include:

• Python 2.6 or above or Python 3.3 or above. On Debian or Ubuntu, you need to install python-dev.
• gcc. On Debian or Ubuntu, install build-essential.
• Cairo, including development files. On Debian or Ubuntu, install libcairo2-dev package.
• libffi with development files, libffi-dev on Debian or Ubuntu.
• Pip. On Debian or Ubuntu, install python-pip.

      

Global installation

If you want to install Graphite-API globally, run the following command as root:

$ pip install graphite-api

      

Python virtualenv installation

If you want to isolate graphite-api installation, you can use Python virtualenv to prevent effect on the system-wide environment.

$ virtualenv /usr/share/python/graphite
$ /usr/share/python/graphite/bin/pip install graphite-api

All the required dependencies for running a Graphite server that users Whisper as a storage backend are installed.

        

Configuration

Set up graphite-api configuration according to your environment.

     

/etc/graphite-api.yaml

The main configuration file for graphite-api is located at /etc/graphite-api.yaml. This file however is optional. If graphite-api cannot find the config file, default values are used. The default setting are:

search_index: /srv/graphite/index
finders:
  - graphite_api.finders.whisper.WhisperFinder
functions:
  - graphite_api.functions.SeriesFunctions
  - graphite_api.functions.PieFunctions
whisper:
  directories:
    - /srv/graphite/whisper
time_zone: <system timezone> or UTC

 

Check the configuration page if you want to find more details on config settings.

      

Deployment

After installation and configuration, you can start a service for graphite-api. There are several options depending on your environment. The available options are:

• Gunicorn + nginx
• Apache + mod_wsgi
• Docker
• Nginx + uWSGI

      

Among them, let’s have a look at Gunicorn + nginx. If you want to find the other deployment options, check the main deployment page of Graphite-API.

      

Gunicorn + nginx

Install Gunicorn using the pip install.

$ pip install gunicorn

            

If you used the Python virtualenv above, you can install Gunicorn in the same virtualenv.

$ /usr/share/python/graphite/bin/pip install gunicorn

 

After installation, you can start creating the script to run Graphite-API using your favorite process watcher.

     

Upstart

description "Graphite-API server"
start on runlevel [2345]
stop on runlevel [!2345]

respawn

exec gunicorn -w2 graphite_api.app:app -b 127.0.0.1:8888

        

Supervisor

[program:graphite-api]
command = gunicorn -w2 graphite_api.app:app -b 127.0.0.1:8888
autostart = true
autorestart = true

      

systemd

# This is /etc/systemd/system/graphite-api.socket
[Unit]
Description=graphite-api socket

[Socket]
ListenStream=/run/graphite-api.sock
ListenStream=127.0.0.1:8888

[Install]
WantedBy=sockets.target

         

# This is /etc/systemd/system/graphite-api.service
[Unit]
Description=Graphite-API service
Requires=graphite-api.socket

[Service]
ExecStart=/usr/bin/gunicorn -w2 graphite_api.app:app
Restart=on-failure
#User=graphite
#Group=graphite
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target

      

After you create a script, you can finally configure the nginx vhost as below.

# /etc/nginx/sites-available/graphite.conf

upstream graphite {
    server 127.0.0.1:8888 fail_timeout=0;
}

server {
    server_name graph;
    listen 80 default;
    root /srv/www/graphite;

    location / {
        try_files $uri @graphite;
    }

    location @graphite {
        proxy_pass http://graphite;
    }
}

      

When you successfully went through the prior steps and the graphite-api service is now running, you can start sending requests to the API. 

       

Hosted Graphite API vs. Graphite API

Although open-source Graphite API and Graphite are powerful monitoring tools, they need the efforts and the resources of your teams to finish the initial setup and to run full-fledged service for internal and external users. While following the steps above, you can see errors and you might be stuck at some stage. To remove the initial hurdles, maintenance inconveniences, and difficulties to implement the complete suite of Graphite API and Graphite, MetricFire provides hosted Graphite API and Graphite.

      

The hosted Graphite has the following benefits.

• Redundancy storage: Graphite’s default storage is file-based and antiquated. MetricFire offers 3 times the redundant storage for seamless scaling and better data protection.
• Control by API: The APIs that MetricFire provides let you control and automate the resources of Hosted Graphite.
• Tagged metrics: Hosted Graphite stores data using tags that enable viewing and organizing metrics with data views.

      

Hosted Graphite keeps all the benefits of open-source Graphite and further enhances the tool with the built-in agent, team accounts, granular dashboard permissions, and integrations to other major platforms and services such as AWS, Heroku, logging tools, and more.

    

There are more benefits for handling high cardinality data specifically.

• It can handle massive-scale data volume with cold and hot data separation. With hosted Graphite, you can ingest 5 billion metrics per minute and store up to 1.5 petabytes of time-series data.
• To retrieve insights from high cardinality data, it maintains low latency queries in the event of high throughput.
• It is highly available among a high volume of concurrent reads and writes as the time series data, by its nature, requires more frequent writing than reading.

       

Conclusion

We discussed using the Graphite API for beginners. In addition to the web-based Graphite, Graphite API provides a flexible way to integrate the monitoring metrics by Graphite into your or customers’ existing system. As you may have felt, it can be complicated to go through all the setup steps and normally launch the services. You can use MetricFire products with minimal configuration to gain in-depth and complete insight into your digital assets.

        

If you would like to learn more about these services, feel free to book a demo to speak with one of our experts or sign up for the free trial today.