+

Integrate Amazon CloudWatch and Amazon Streams

Integrating Amazon CloudWatch and Amazon Streams with your monitoring system

To integrate Amazon CloudWatch and Amazon Streams with your monitoring system, reach out to MetricFire. Book a demo with the MetricFire team to discuss integrating Amazon CloudWatch and Amazon Streams and how that can support your monitoring system.

What is Amazon CloudWatch?

Amazon CloudWatch is a management and monitoring service designed for AWS and other infrastructure resources or on-premises applications. It is the official metrics monitoring tool for Amazon Web Services. Using CloudWatch, you can access all your performance and operational metrics in a single platform, helping you overcome the challenge of monitoring multiple systems. CloudWatch helps you monitor your entire stack — including applications, infrastructure, and services — thus freeing up valuable resources to allow you to focus on building applications.

You can use CloudWatch Container Insights to monitor and troubleshoot your applications and microservices. CloudWatch collects, aggregates, and summarizes computer utilization information; like CPU and memory usage, network data history, and also monitoring diagnostic information. Container Insights provides you with details about container management services, such as: Amazon ECS for Kubernetes (EKS), Amazon's Elastic Container Service (ECS), etc.

 

The brilliant thing about Amazon Cloudwatch is that it is your gatekeeper to data and metrics for all your Amazon applications and services. However, monitoring more than the standard set of metrics can become very expensive with CloudWatch. CloudWatch custom metrics are very expensive and they should be used sparingly. For example, if a company is monitoring their AWS systems with the standard CloudWatch dashboards, it might cost around 1000 USD a month. However, if you’re monitoring hundreds of thousands of metrics related to a new launch, AWS CloudWatch could quickly rack up to 50,000 USD a month.

That's why it's such a vital integration point for MetricFire. MetricFire treats all metrics the same, so if you’re monitoring thousands of specialized metrics, you’ll still pay the same basic rate for those metrics. CloudWatch can be integrated with MetricFire, so you can pull your AWS metrics into the MetricFire platform. Then, you can get low-cost metrics scaling, while still being able to monitor your AWS metrics all in a single pane of glass. MetricFire's advanced filtering lets you choose only the data views you want to see and discard the rest. You can also set up simple rules to discard data you no longer need to keep, plus receive alerts via email or Slack.

What is Amazon Streams?

Amazon Kinesis Data Streams (KDS) is a data streaming service that is both scalable and durable and operates in real-time. It can capture gigabytes of data-per-second from hundreds of thousands of sources, including websites, databases, social media feeds, IT logs, financial transactions, and location-tracking events. Real-time analytics tools such as dashboards, anomaly detection, and dynamic pricing are updated in milliseconds.

KDS is based around a producer/consumer model. A producer loads data records into Amazon KDS. A consumer then processes the data records from the stream. For example, a producer could load web server log data to a Kinesis data stream, where a consumer can process and analyze the log data.

You can use Kinesis Data Streams to collect log and event data from servers, desktops, and mobile devices. You can then build Kinesis Applications to process your data continuously, generate metrics, populate dashboards, and aggregate data into stores such as Amazon S3.

You can create Kinesis Applications that run real-time analytics on high-frequency event data events. For example, you could write an application to analyze sensor data collected by Kinesis Data Streams to gain insights into your sensor data.

If you collect gaming data, Kinesis Data Streams can collect data about player-game interactions and feed the data back into your gaming platform. This strategy can help you design a game that provides engaging and dynamic experiences based on players' past actions and behaviors.

With Metricfire, you can turbocharge your data stream monitoring services. By integrating Amazon CloudWatch with the Metricfire platform, you can display your KDS metrics on aesthetically pleasing dashboards. MetricFire's advanced filtering lets you choose only the data views you want to see and discard the rest. You can also set up simple rules to discard data you no longer keep, plus receive alerts via email or Slack when your data streams are doing something they shouldn't.

How to integrate Amazon CloudWatch and Amazon Streams with your monitoring system

To integrate Amazon CloudWatch and Amazon Streams with your monitoring system, sign up for a free trial with MetricFire. Talk with the MetricFire team about how to integrate Amazon CloudWatch and Amazon Streams and get Amazon CloudWatch and Amazon Streams interacting with your MetricFire dashboards directly.

What is MetricFire?

MetricFire is a full-scale platform that offers infrastructure, system, and application monitoring using a suite of open-source monitoring tools. The platform allows you to use either Prometheus- or Graphite-as-a-Service and have your metrics displayed on aesthetically-pleasing Grafana dashboards. 

     

MetricFire offers its users a complete ecosystem of end-to-end infrastructure monitoring, which is made up of three of the most popular open-source monitoring software services: Prometheus, Graphite, and Grafana. As well, plugins for a number of other open-source projects are preconfigured, such as StatsD, collectd, and Kubernetes. You get all these within a hosted environment as a single product. Not only does MetricFire fit well into the infrastructure monitoring use-case, such as network monitoring and server monitoring, we also do application monitoring and business intelligence.

        

Through this hosted environment, MetricFire works to boost the unique features of the open source projects to give you more functionality than the original products. Below are some of the MetricFire features at a glance: 

Cloud and on Premise Monitoring

Hosted Prometheus

Hosted Graphite

Grafana Dashboards

        

Whether to use Prometheus or Graphite is a great question. We’ve looked deeper into this decision in our blog article, Prometheus or Graphite. While Prometheus is a newer software that has a multidimensional data structure, Graphite is robust and sufficient. The key thing to remember is that Hosted Graphite by MetricFire is more than just Graphite. Our Hosted Graphite product actually adds data dimensionality and better data storage.

       

The benefits of MetricFire are:

    

No vendor lock-ins
MetricFire allows you to request for a full export at any time, because you will always own the data you input. You get all the benefits of an open-source tool with the stability and security of a SaaS tool. 
      

Easy Budgeting
A structured pricing model allows you to save time and work within your budget. The predictability and transparent pricing allows you to keep your costs in check and plan for the future.
       

Transparency
MetricFire works transparently on all aspects of their operations of SaaS system monitoring. You can see their own internal system metrics at their public status page.
      

Robust Support
Technical support is provided by engineers for engineers, so you can expect detailed and relevant answers to your queries. 

The author for this testimonial, Jim Davies, Head of DevOps, MoneySuperMarket.com.
“As MetricFire scales effortlessly, we can push and store more metrics than we really need today but might need tomorrow. This increases our depth of understanding of the systems that we run and heads off any future problems.”
Jim Davies, Head of DevOps, MoneySuperMarket.com.

Don't just take our word for it

Why thousands of engineers choose us:

fingerprint icon Own your data
dollar icon Major savings
eye icon Nothing hidden
face icon Engineering support

Vendor lock-in's not our thing. We’re believers in you still owning your data, so you can request a full export at any time. That means you get all the benefits of an open-source tool, but with the security and stability of a SaaS tool.

“We now have over ten times the amount of metrics we started with, and on different accounts. One of the great things about MetricFire is that scaling to support this increase has been hassle-free, requiring no additional work on our side.”

Maxime Audet, Cloud-Ops Team Lead, Coveo

Read customer story Customer photo of Maxime

Save valuable engineering time and stick within your budget. We're known for our predictable and transparent pricing so it's easy to keeps costs under control.

“Building and managing an on-premise installation at this scale would require a lot of engineer time, especially in the first year...we use this engineering time to work on initiatives closer to our core business”

Jim Davies, Head of DevOps, MoneySuperMarket.com

Read customer story Customer photo of Jim

Transparency is at the root of how we operate: from pricing to postmortems we're open about how we do things. We even publish our internal system metrics to our public status page.

“There’s complete transparency with everything MetricFire do which means we can accurately predict what we’ll be spending and comfortably keep within our budget.”

Itai Yaffe, Big Data Developer, Nielsen

Read customer story Customer photo of Itai

Our first line, technical support comes from engineers, for engineers. So you get highly detailed, expert support when you need it.

"Every time I have a question, I get an answer from support after just a couple of hours. Their technical knowledge is excellent.”

Shahar Kobrinsky, VP of Architecture and Scale, Eyeview

Read customer story Customer photo of Shahar

Questions?

Don’t see the integration you’re looking for? Need help setting up your monitoring? Get in touch and one of our engineers will help you out.

Contact us
Want to read more?

Related articles

Do more with MetricFire