Table of Contents
Introduction
Monitoring your Heroku stack helps you understand the performance of your application and infrastructure. You can identify bottlenecks, slow-performing queries, or resource-intensive processes and optimize them. Monitoring also allows you to detect issues or anomalies in real time. By setting up alerts based on predefined thresholds, you can be notified as soon as something goes wrong, enabling you to address the issue before it affects users. Heroku provides a platform-as-a-service (PaaS) environment, but it's important to monitor resource usage. This includes tracking CPU usage, memory consumption, and other metrics to ensure your application has the necessary resources to handle user traffic.
In this article, we'll detail how to forward performance metrics from your Heroku dynos and select add-ons, to a Hosted Graphite backend.
Configuring the Heroku Hosted Graphite Add-On
Hosted Graphite by MetricFire is a robust data storage backend that allows you to build custom dashboards and alerts from your data. Hosted Graphite manages a service that parses through your Heroku log-drain output, converts the data to the Graphite format, and forwards them to your Hosted Graphite account. You can then use these Graphite metrics to build custom dashboards and alerts. Additionally, you can store more than just Heroku metrics which can give you visibility into multiple layers of your infrastructure under a single pane of glass.
Heroku has an add-on for Hosted Graphite that can easily be enabled through the Heroku UI, or from a similar command from within the Heroku CLI:
heroku addons:add hostedgraphite -a <app-name>
Once the Hosted Graphite add-on is provisioned (default plan is Intro), you can navigate to your account from within the Heroku UI:
Once Heroku dyno metrics are flowing into your Hosted Graphite account, a default Heroku dashboard will be automatically installed. This dashboard offers a great starting point for your monitoring and can be customized further:
If you are not receiving web metrics, you probably have to enable Log Runtime Metrics from within your Heroku CLI:
heroku labs:enable log-runtime-metrics -a <app-name>
heroku restart -a <app-name>
Monitoring the Heroku Postgres Add-On
If you use Heroku's Postgres Add-On at the standard or premium tier, 30 performance metrics per DB instance will be forwarded to your Hosted Graphite account. They are prefixed with: heroku.<app-name>.heroku-postgres.<instance-name>.<type> and these are the metrics as they appear in the Graphite (dot-notation) format:
heroku.<app-name>.heroku-postgres.<instance-name>.database.active-connections
heroku.<app-name>.heroku-postgres.<instance-name>.database.current_transaction
heroku.<app-name>.heroku-postgres.<instance-name>.database.db_size
heroku.<app-name>.heroku-postgres.<instance-name>.database.index-cache-hit-rate
heroku.<app-name>.heroku-postgres.<instance-name>.database.table-cache-hit-rate
heroku.<app-name>.heroku-postgres.<instance-name>.database.tables
heroku.<app-name>.heroku-postgres.<instance-name>.database.waiting-connections
heroku.<app-name>.heroku-postgres.<instance-name>.server.load-avg-15m
heroku.<app-name>.heroku-postgres.<instance-name>.server.load-avg-1m
heroku.<app-name>.heroku-postgres.<instance-name>.server.load-avg-5m
heroku.<app-name>.heroku-postgres.<instance-name>.server.memory-cached
heroku.<app-name>.heroku-postgres.<instance-name>.server.memory-free
heroku.<app-name>.heroku-postgres.<instance-name>.server.memory-postgres
heroku.<app-name>.heroku-postgres.<instance-name>.server.memory-total
heroku.<app-name>.heroku-postgres.<instance-name>.server.read-iops
heroku.<app-name>.heroku-postgres.<instance-name>.server.tmp-disk-available
heroku.<app-name>.heroku-postgres.<instance-name>.server.tmp-disk-used
heroku.<app-name>.heroku-postgres.<instance-name>.server.wal-percentage-used
heroku.<app-name>.heroku-postgres.<instance-name>.server.write-iops
Hosted Graphite also provides a pre-configured dashboard for these metrics that can be installed from their Dashboard Library:
Monitoring the Heroku Redis Add-On
heroku.<app-name>.heroku-redis.<instance-name>.database.active-connections
heroku.<app-name>.heroku-redis.<instance-name>.redis.evicted-keys
heroku.<app-name>.heroku-redis.<instance-name>.redis.hit-rate
heroku.<app-name>.heroku-redis.<instance-name>.redis.memory-redis
heroku.<app-name>.heroku-redis.<instance-name>.server.load-avg-15m
heroku.<app-name>.heroku-redis.<instance-name>.server.load-avg-1m
heroku.<app-name>.heroku-redis.<instance-name>.server.load-avg-5m
heroku.<app-name>.heroku-redis.<instance-name>.server.memory-cached
heroku.<app-name>.heroku-redis.<instance-name>.server.memory-free
heroku.<app-name>.heroku-redis.<instance-name>.server.memory-total
heroku.<app-name>.heroku-redis.<instance-name>.server.read-iops
heroku.<app-name>.heroku-redis.<instance-name>.server.write-iops
Hosted Graphite also has a pre-configured Heroku Redis dashboard available in their Dashboard Library:
Monitoring the Heroku Kafka Add-On
If you use the Heroku Kafka add-on at the Standard tier or above, 10 metrics per cluster node will automatically be forwarded to your Hosted Graphite account. They are prefixed with: heroku.<app-name>.heroku-kafka.<node>.<instance-name>.<type> and this is what they look like in the Graphite format:
heroku.<app-name>.heroku-kafka.<node>.<instance-name>.kafka.bytes-in-per-second
heroku.<app-name>.heroku-kafka.<node>.<instance-name>.kafka.bytes-out-per-second
heroku.<app-name>.heroku-kafka.<node>.<instance-name>.server.load-avg-15m
heroku.<app-name>.heroku-kafka.<node>.<instance-name>.server.load-avg-1m
heroku.<app-name>.heroku-kafka.<node>.<instance-name>.server.load-avg-5m
heroku.<app-name>.heroku-kafka.<node>.<instance-name>.server.memory-cached
heroku.<app-name>.heroku-kafka.<node>.<instance-name>.server.memory-free
heroku.<app-name>.heroku-kafka.<node>.<instance-name>.server.memory-total
heroku.<app-name>.heroku-kafka.<node>.<instance-name>.server.read-iops
heroku.<app-name>.heroku-kafka.<node>.<instance-name>.server.write-iops
Hosted Graphite also has a pre-configured Heroku Kafka dashboard that can be generated from their Dashboard Library:
Create Custom Alerts using Your Graphite Data
In the Hosted Graphite UI, navigate to Alerts => Graphite Alerts to create a new alert. Name the alert, add a query to the alerting metric field, and add a description of what this alert is (NOTE: you can use wild card (*) searching to grab all metrics from a specified index):
Then select the Alert Criteria tab to set a threshold, and select a notification channel. The default notification channel will be the email you used to sign up for the Hosted Graphite account, but you can easily configure channels for Slack, PagerDuty, Microsoft Teams, OpsGenie, custom webhooks and more. See the Hosted Graphite docs for more details on notification channels:
Conclusion
Monitoring your Heroku infrastructure and add-ons is crucial for optimizing performance, ensuring resource efficiency, and proactively addressing issues. It allows you to detect and resolve bottlenecks, anomalies, and security incidents in real time, contributing to a better user experience. Additionally, effective monitoring aids in cost optimization by identifying unused resources and ensuring compliance with industry standards and regulations. Hosted Graphite integrates seamlessly with Heroku, and offers a monitoring environment for much less than other platforms, like Data Dog.
Tools like dashboards and alerts will complement your data by providing real-time visualization, proactive identification of issues, historical trend analysis, and facilitating informed decision-making, all of which are essential for maintaining a robust and efficient infrastructure.
Sign up for the free trial, and experiment with monitoring your Heroku stack today. You can also book a demo and talk to the MetricFire team directly about your monitoring needs.
