A comprehensive set of turnkey infrastructure integrations

Including dozens of AWS and Azure services, web, database, network, containers, orchestrations like Docker and Kubernetes, and more.

START FREE TRIAL

Complete visibility into the health and performance of applications and their underlying infrastructure

Quickly pinpoint the root cause of performance issues across the stack, down to a poor-performing line of code

START FREE TRIAL

Custom metrics and analytics

Analyze custom infrastructure, application, and business metrics

View Custom Metrics Monitoring Info

Powerful API that makes it easy to collect and create any custom metric

Achieve ultimate visibility and enhanced troubleshooting with synthetic and real user monitoring

START FREE TRIAL

Free APM Software

Catch bugs early on, and gain full visibility and insights into the applications you’re developing

View Product Info

Free, full-function APM tool for testing and troubleshooting application performance before moving into production

Dev Edition includes five traces per minute, 100 metrics, three hosts, and six containers

GET FREE TOOL

Log Management and Analytics powered by SolarWinds Loggly

Integrated, cost-effective, hosted, and scalable full-stack, multi-source log management

View Log Management and Analytics Info

Collect, search, and analyze log data in addition to your metrics and traces to quickly pinpoint application performance problems

Reduce mean time to resolution (MTTR) by quickly jumping from a trace or host view into the relevant logs to accelerate troubleshooting

START FRE TRAIL

Digital Experience Monitoring Powered by SolarWinds Pingdom

Make your websites faster and more reliable with easy-to-use web performance and digital experience monitoring

View Digital Experience Monitoring Info

Add client-side web application performance monitoring. Provide maximum observability by adding the user’s perspective.

Achieve ultimate visibility and enhanced troubleshooting with synthetic and real user monitoring

START FREE TRIAL

Every day more and more third-party service providers are publishing APIs for their customers to consume, with many of them competing against each other. For instance, with payment APIs, you can use Stripe, Recurly, Chargify, or several others.

Criteria for Choosing a Third-Party API

There are many criteria to help you choose the correct service for your product. Below are some examples.

  • Documentation – Is the service fully documented? Can you easily find answers to your questions? Are there numerous examples and are those examples in your target language (both programming and spoken language)?
  • Language support– Is there a wrapper or SDK for your language? How easy will it be to create calls to their service? Are there code examples?
  • Features – Does the service have all the features you need? Knowing which features are required can help you filter out unsuitable options.
  • Support – Is there someone available to answer your questions when there are issues? There are sometimes issues only the service provider can help you with.
  • Familiarity – Have you used the product successfully in the past? Was it easy to integrate with your former project?

Most of those criteria (we’ll call exploratory criteria) can be answered by a surface scanning of their documentation and marketing sites. Before committing financially to a company, you may want to know how it performs under real load. Is the service highly available? Is the performance better or on-par with similar service providers?

You may already be using a service provider, but aren’t sold on the performance, availability, or error count you encounter from the current service provider. You realize things could be worse, but it would be nice to benchmark the two against each other before making the switch.

To help with some additional criteria, you can use a tool like SolarWinds® AppOptics to help you gather information and see if the third-party meets the criteria.

  • Performance – How long does each third-party service take to respond? This might be the most important of all criteria. We can gather this information from AppOptics auto-instrumentation.
  • Reliability – How often are there unexpected errors? How often do you need to retry requests? Does this cause interruption for your customers? We can track this by using the AppOptics SDK to designate transaction names.
  • Resource size – REST APIs may be sending you more information than you need. AWS and other providers charge money based on network activity, so if there will be large amounts of packets exchanged, even small differences could wind up costing more than you’re comfortable with. We can track this using AppOptics custom metrics.

Using an APM Tool to Collect Third-Party Performance Criteria

Some of the previously stated criteria (we’ll call them performance criteria) will be collected automatically by AppOptics, where others will require taking advantage of the SDK to add custom information. I’m going to use a Rails example, but the basic idea can be implemented in any of our supported programming languages.

We can start with GET calls. Let’s say we want to add translation functionality to our application. Using the exploratory criteria, we’ve narrowed the list to two services, Service A and Service B (naming is hard).

Note: From this point on, I’ll assume you ‘ve gone through the AppOptics installation process, and traces have successfully been transmitted. It’s a two-step install and should take no longer than five minutes to start seeing traces from your application. You can start a two-week trial or sign up for the free AppOptics Dev Edition.

 

I’ll start with a sample controller for translating text and add a random choice to call Service A or Service B and return the response as JSON data.

Each service will be instrumented automagically by AppOptics and the response time will be broken down in detail in the Service > Remote Services section of the UI.

Here you can see Service B is taking about twice as long as Service A on average.

On to reliability. AppOptics does a great job of capturing errors and exceptions but in this case, we need a way to distinguish the metrics when the different remote services are used. We can use the SDK to change the name of the transaction to gain insight into the reliability of each remote service. We can set transaction names and change the if block in our code.

Now all the errors/exceptions can be filtered by transaction and we can compare them to see which service is more reliable and stable. This also gives us a chance to review if we’re sending the requests improperly.

To monitor resource size, we need to get creative and use a bit of ruby magic to track the information we need. We’re going to prepend a method to the Net::HTTP request method to gather metrics for each of the services.

We can add the following code file where the controller lives. By default, the only tag sent with the measurement is the ‘service‘ tag (which is the name of the AppOptics Service), but since we set the transaction name, we can add the ‘transaction‘ tag to separate out the different services. Also, using the ‘transaction‘ tag helps keep the tags in sync with the other metrics.

Now we’re able to plot the response and request size to compare the two. You can see Service A is about 20% higher for response size.

Decision Time

How the criteria will affect your decision is subjective. For instance, if you only use these services once during a user signup process, the impact of the request and response size is likely to be minimal.

I pulled all the information together into one dashboard so everything could be seen together. I have the yaml configuration file for the below dashboard if you want to import it into your account. The file has curly braces around the values which you’ll need to change to fit your situation.

This gives you a quick sample on how you can compare service performance criteria using AppOptics but there’s much more to explore. For instance, this works well for read-only services, but what about more complex API usage for services storing application state like a database or authentication service? Also, I’d advise testing in a staging or test environment before moving the code to production.

Summary

More and more applications rely on one or more third-party APIs, so it’s worth the time to define a set of performance criteria when evaluating which are the best one(s) for your application. To see how each API’s performance stack up to your criteria, you’ll want to use an APM tool like AppOptics.

If you have any questions on how to use AppOptics for benchmarking third-party services, feel free to contact me at mbeale@SolarWinds.cloud.

If you haven’t experienced the simple-to-install and easy-to-use AppOptics, maybe now is the time. Explore its comprehensive infrastructure server, virtual host, and —alongside auto-instrumented root cause, detailed service and transaction traces, and insights down to the line of poor performing code. Signup for a 30-day free trial.

Related articles

© 2021 SolarWinds Worldwide, LLC. All rights reserved.