Monitoring GitHub with Amazon SQS

To integrate GitHub and Amazon SQS with your monitoring system, please reach out to MetricFire. Book a demo with the MetricFire team to discuss integrating GitHub and Amazon SQS and how that can support your monitoring system.

GitHub is a service for hosting code and doing version control for code. The foundation of GitHub is "Git" - which is an open-source version control system. 

Git allows multiple coders to contribute to one project while remembering which is the most up-to-date version. Git also remembers who contributed and what they contributed to the project. All modifications and revisions are stored in a central repository, and contributions are made through a pull request. 

Pull requests can be made by anybody, but they must be approved by the repository admin. In some projects, pull requests can be contributed without approval. However, most projects have some quality assurance processes. 

When doing application performance monitoring for a project built in GitHub, you’ll want to export data about your GitHub setup into your regular monitoring dashboards. You want to monitor when changes happen, and the effect they have on your system. 

For example, if you make a change to your code, and suddenly the latency of requests to your server increases - you’ll want to know exactly which pull request caused that change. If you didn’t know that there were any changes in your code, then you wouldn’t be able to correlate the change in your code with the changes in how your system is behaving. This will slow down your debugging process. 

With MetricFire, you can get all of your monitoring dashboards automatically annotated with the changes you’ve made to your code in GitHub. If there are any changes in behavior, a small note will be present in the graph to tell you what’s happening in your environment. 

To get GitHub commits flagged in Hosted Graphite follow these instructions:
Go to the Addons page, and in the Annotation Add-Ons section click on the GitHub card.
Copy the webhook URL, including your API key.

Browse to your GitHub repo, visit the ‘settings’ section, then ‘webhooks’

This enables you to automatically monitor the correlation between GitHub changes and how your system is functioning.

Amazon Simple Queue Service (SQS) is a message queuing service that you use to decouple and scale microservices, distributed systems, and serverless applications. Amazon SQS supports programmatic sending of messages via web service applications as a way to communicate over the Internet. It is fully managed and eliminates the complexity and overhead associated with operating message-oriented middleware. Amazon SQS is used by companies such as Netflix, Dropbox, NASA, and more.

With SQS, you can send, store, and receive messages between software components without the need for another service. You can get started with SQS in minutes using the AWS console, Command-Line Interface, or SDK of your choice.

SQS offers two types of message queues:

• Standard queues provide maximum throughput, best-effort ordering, and at least once delivery. 
• SQS FIFO queues ensure messages are processed exactly once, in the exact order you send them.

You can use Amazon SQS to add an extra layer of security to your messaging services. You can exchange sensitive data between applications using server-side encryption (SSE) to encrypt each message. Plus, because Amazon SQS scales elastically with your applications, it is a highly cost-effective messaging solution.

Amazon SQS is fully integrated with Amazon CloudWatch, allowing you to view and analyze your Amazon SQS queue metrics, which is where MetricFire comes into play.

With Metricfire, you can turbocharge your Amazon SQS monitoring services. By integrating Amazon CloudWatch with the Metricfire platform, you can display your metrics on aesthetically pleasing dashboards. MetricFire's advanced filtering lets you choose only the data views you want to see and discard the rest. You can also set up simple rules to discard data you no longer keep, plus receive alerts via email or Slack when your message queues are doing something they shouldn't.

To get started, first, create a policy that we will later attach to the user.

1.  In the IAM Dashboard, click on Policies (on the left) or Customer Managed Policies under IAM Resources, then click on Create Policy.
2. Switch to the JSON tab, replace the existing text with the policy provided below, and click ‘Review policy‘.
3. Give the policy a name (Eg: HG_policy) and a description (optional).
4. Save the policy by clicking ‘Create Policy‘.

Next, we’ll create a user to attach the policy. We’ll use the Access Key/Secret Key tokens to permit Hosted Graphite to import CloudWatch metric data.

1.  In the IAM Console, click on ‘Users‘ on the left or under IAM Resources. Then select ‘Add User‘.
2. Give the user a name (Eg: HG_addon) and ‘Programmatic access‘ for the Access type.
3. Click ‘Next: Permissions‘ and click the ‘Attach existing policies directly‘ button at the top.
4. Search for the policy we just created and attach it by clicking the corresponding checkbox, then press ‘Next: Review‘.
5. Click on ‘Create User‘ and copy the Access Key and Secret Key into the relevant fields on the add-on set-up page here. Optionally, you can download a copy of this account information, but keep it safe as this information can be sensitive.

To integrate GitHub and Amazon SQS with your monitoring system, sign up for a free trial with MetricFire. Talk with the MetricFire team about how to integrate GitHub and Amazon SQS and get GitHub and Amazon SQS interacting with your MetricFire dashboards directly.

MetricFire is a full-scale platform that provides infrastructure, system, and application monitoring using a suite of open-source monitoring tools. We enable you to use Hosted Graphite and aesthetic custom dashboards to visualize your metrics so you can understand what is happening.

MetricFire offers a complete ecosystem of end-to-end infrastructure monitoring, comprised of open-source Graphite and popular dashboards. Plugins for many other open-source projects are preconfigured, such as StatsD, collectd, and more. Not only does MetricFire fit well into many monitoring use cases, such as server monitoring, but we also offer application and cloud infrastructure monitoring.

Through this hosted environment, MetricFire boosts the unique features of open-source projects to give you more functionality than the original products. Below are some of the MetricFire features at a glance: 

• Cloud Monitoring
  
  
• Hosted Graphite
  
  
• Dashboards

The key thing to remember is that Hosted Graphite by MetricFire is more than just Graphite. Our Hosted Graphite product actually adds data dimensionality and better data storage.

Benefits of Using MetricFire:

• Simple, low-cost pricing
  A structured pricing model based on unique time series metrics allows you to work within your budget. The predictability and transparent pricing allow you to keep your costs in check and plan for the future. 1 metric is 1 metric.
  
  
  
• Easy-to-use dashboards
  Easily share your dashboards with clients for free.
  
  
  
• Responsive alerting
  Alert notification integrations PagerDuty, Slack, email, and webhooks.
  
  
  
• Freedom of customization
  Custom metrics through your code.
  
  
  
• Fantastic customer support
  Highly available support is provided by engineers for engineers to get you set up quickly.
  
  
  
• Enterprise-ready
  Dedicated clusters for users that need their own environment.