Sumo Logic is a cloud-based log management and analytics service that leverages machine-generated big data to deliver real-time IT insights. In this blog post, we discuss how you can integrate Catchpoint with Sumo Logic. The integration involves pushing data from Catchpoint to Sumo Logic using Webhooks and using this data to write queries and create visualizations.
Webhooks are HTTP callbacks that are triggered by events specified in the code by the application developer; the event data is then recorded and pushed to an end-point URL. For example, when you subscribe to the newsletter feature on a website it triggers a webhook defined by the website developer. This webhook sends the user data to the third-party mailing service that is integrated with the website. The mailing service accepts the new user data and triggers a response email that is sent to the subscriber immediately. Webhooks are a powerful way of exchanging data.
Getting the Data
Step 1 – Setting up Sumo Logic
1: Login to your Sumo Logic account.
2: Click on Manage Data -> Collection -> Setup Wizard
3: Click on Set Up Streaming Data
4: Select Your custom app
5: Select HTTP source
6: Enter the details like Source Category and click continue, it should start creating an endpoint URL for you.
7: Get the endpoint URL, this is where Catchpoint webhook will send data to.
Step 2 – Setting up the webhook to post data to endpoint URL of Sumo Logic.
1: Select a test for which you want to enable webhook. Go to advanced options in the test setup page and enable test data webhook for that test.
2: Now go to Settings->API , copy paste the Sumo Logic endpoint URL to the Catchpoint portal and click on Save.
Step 3 – Verification (Sumo Logic), Go to search->search and you should see Catchpoint test data
Now that the integration is complete, we have set up a collector endpoint in Sumo Logic to which data is sent from Catchpoint. You can access the data by writing a query as follows (depends on use case).
_collector=”HTTP” or _sourceCategory=/opt/logs or _source=”Catchpoint"
In this case, all the queries will show similar output, since the endpoint URL is common to all three of them.
Figure 1 : Querying data
You can also go to Manage Data -> Collection and click the Open in Log Search Button (marked in red in Figure 1) to get the data of that particular HTTP source.
Figure 2 : Understanding source , source category, collector
After button click you should be redirected to search page and automatically _source would be set to Catchpoint (or whatever name you choose) . By default the name of HTTP source would be Custom App, you can change it by clicking Edit in the Figure 2.
Now let’s visualize DNS time for every second:
_collector=HTTP | json field=_raw "Summary.Timing.Dns" as DNS | timeslice 1s | avg(DNS) by _timeslice
The above query will plot DNS times at 1 sec intervals. Let’s dissect the query and understand how we can use it for similar metrics. We already know that DNS time can be retrieved from the JSON by using the expressions such as Summary.Timing.Dns (See Figure 3 for the JSON structure) . Once we have this figured out, everything falls into place as we can extract the DNS times using the expression:
json field=_raw "Summary.Timing.Dns" as DNS
Also, we set time slice to 1 second. By definition, time slice segments data by time periods or bucketed over time range.
Figure 3: JSON as received by Sumo Logic
Finally, we average the DNS value by time slice and we get a DNS visualization as shown in Figure 4 .
You can add this to dashboard by clicking on Add to Dashboard button or Update Dashboard (only if you have already created a dashboard)
Next, let’s visualize locations where the tests results are coming from.
(_collector=HTTP ) | _sourceHost as sourceIP | lookup latitude,longitude from geo://default on ip=sourceIP | count by latitude,longitude | sort _count
We know that we can get IP addresses from the following:
Hence, we look up for latitude and longitude based on IP address from the Sumo Logic databases and display the number of results coming from each location on the map. Make sure to click on map icon in the Aggregates tab.
Catchpoint integrates with major third-party services to provide detailed performance analysis and gives you control over how you use your data. As described in this post, the integration is simple; it allows you to pull data from Catchpoint into Sumo Logic and you can use the data to create powerful visualizations that present a 360 degree evaluation of your application’s performance. You can download a copy of the Sumo Logic Integration Guide here.
Interested in Catchpoint’s other integrations? View them here.