Introduction

Prometheus is a free and open-source software for real-time systems and event monitoring and alerting. Originally developed at SoundCloud, Prometheus became a project of the Cloud Native Computing Foundation in 2016, alongside other popular frameworks such as Kubernetes.

To start using Prometheus, you’ll need a solid understanding of all of the tool’s functionality. The Prometheus project encompasses multiple interlocking components, including:

• The Prometheus server, which acts as the system’s “brain” by collecting various metrics and storing them in a time-series database. Unlike other systems monitoring tools that wait passively to receive information, Prometheus actively scrapes metrics from applications and services (e.g. Docker containers) at designated intervals using the Prometheus client libraries (see below).

• Prometheus client libraries help users integrate their services by sending metrics and data in a format that Prometheus can understand. Programming languages with official support include Go, Java, Scala, Python, and Ruby, plus unofficial third-party support for more than a dozen other languages.

• Prometheus Alertmanager, which sends alerts and notifications to designated users when the tool detects anomalies and performance issues.

• Prometheus visualization tools for displaying metrics and data in a human-readable format. In addition to native Prometheus visualization, many Prometheus users integrate the tool with Grafana, an open-source web application for analytics and data visualization.

But there’s another crucial piece of the puzzle that we haven’t mentioned yet: Prometheus Pushgateway. Below, we’ll answer the important question: “what is Prometheus Pushgateway?”.

What is Prometheus Pushgateway?

As discussed above, Prometheus scrapes data from your applications and services at regular intervals—but what if those intervals are still too slow to capture valuable information? Many short-lived and batch jobs complete so quickly that their lifespans are shorter than the Prometheus scrape interval itself, making it tricky or impossible to capture metrics from them at runtime.

Enter Prometheus Pushgateway. Prometheus Pushgateway is a component of Prometheus that helps collect metrics for extremely brief jobs. Instead of waiting until the next Prometheus scrape interval, which could massively slow down the system itself, you can instead choose to have these brief jobs push metrics to a Pushgateway. A Prometheus Pushgateway acts as a kind of intermediate service or “gateway” (hence the name), storing these pushed metrics until the next scrape interval.

Since Prometheus is an open-source project, you can view the Pushgateway source code on github.com. Below are some tips and tricks for using Pushgateway:

• ​The --persistence.file flag sets the file location where you will be pushing metrics.
• The Pushgateway exposes all pushed metrics via the /metrics endpoint. You can change this with the --web.telemetry-path flag.
• By default, Pushgateway listens to port 9091, with the path /metrics/job/<JOB_NAME>{/<LABEL_NAME>/<LABEL_VALUE>}. 
• The honor_labels setting should always be set to true in the Pushgateway configuration in order to retain job labels and instance labels.

If you're interested in trying it out for yourself, sign up for our free trial. You can also sign up for a demo and we can talk about the best solutions for you.

What are the Use Cases of Prometheus Pushgateway?

Prometheus Pushgateway sounds like a tremendously useful tool for capturing data from jobs that are difficult to scrape—so why not use Pushgateway in a wide variety of applications? Not so fast. The Prometheus website itself says: “We only recommend using the Pushgateway in certain limited cases.” Some of the downsides of using Prometheus Pushgateway include:

• Prometheus Pushgateway does not offer automatic instance health monitoring, which is accessible via the up metric after every Prometheus scrape.
• If a single Pushgateway is used to monitor multiple instances, then the Pushgateway risks becoming a bottleneck and a single point of failure for the system.
• Pushgateway data needs to be manually deleted using an API, or else it will be exposed to Prometheus indefinitely.

According to Prometheus: “Usually, the only valid use case for the Pushgateway is for capturing the outcome of a service-level batch job.” Here, the term “service-level” refers to a batch job that is not linked to a single machine or job instance.

Examples of non-service-level batch jobs include cron jobs for automatic security updates and configuration management client runs. In this case, Prometheus suggests using the textfile collector from the Prometheus Node Exporter, which explicitly says that it is intended for “metrics that are tied to a machine.”

In general, Prometheus Pushgateways do not aggregate the metrics that they receive from your jobs—by design, this data will be exposed to Prometheus “forever.” However, if you have a use case where you could benefit from aggregating Pushgateway data, you can use a third-party Prometheus add-on such as the Prometheus Aggregation Gateway.

This Pushgateway variant aggregates the data it stores, e.g., histograms and counters with matching labels are added up. Need more information on Prometheus Pushgateway, or want to see a complete tutorial on how to use Prometheus Pushgateway? Check out our article “Prometheus Pushgateways - Everything You Need To Know.

Recommended reading: Prometheus - Use Cases

Use the MetricFire free trial to start using Prometheus without any set up. Also, talk to us directly by booking a demo - we’re always happy to talk with you about your company’s monitoring needs.

How Can MetricFire Help?

MetricFire is a cloud infrastructure and application monitoring platform that makes it simple to understand your data at a glance. We offer a hosted Prometheus service, including Grafana dashboards, that handles all the challenges and complexities for you—scaling, data storage, support and maintenance, and more—so that you can spend more time on the metrics and less time on the technical details.

To learn more, check out the Prometheus posts on our blog, or get in touch with us today for a chat about your business needs and objectives.

Want to get started with Prometheus right away? The MetricFire free trial is an excellent way to explore what you can do with hosted Prometheus, including using Prometheus Pushgateways.