Five Ways to Help Your Developers Analyze API Outages

When your product relies on APIs (external or internal), it is critical that you monitor those APIs in order to know when something goes awry that affects your customer’s perception of your product’s current status. There are many ways an API can be down and or adversely affect your product. An external API may simply be down. Or its response might be so delayed that you have to decide whether to present users with an incomplete result that omits the latest data from the slow API, or wait for the result and accept performance delays in your product.

If your production software expects a JSON response from an API, and you receive responses with invalid JSON, you need to know that so you can develop a solution for when that situation arises. If a valid response always includes a specific character sequence, a quick regular expression test should always be performed on responses from the API. For more complex requirements, Javascript can be applied to examine a response packet and verify that it meets specific requirements.

My last post describes how to apply Javascript to extract information from a SOAP format response from an API call. The Javascript in that example isolates a specific element in the SOAP response from a call to the Flickr API. The Javascript assert() command is used in multiple places to ensure that the response includes various required elements and data. For example, if an XML element is missing, or if the value in the SOAP XML element does not match what is expected, then the response is invalid.

The Javascript implementation included in the API Science platform provides immense flexibility for verifying and validating API responses. The API Science platform also includes four other methods for validating API responses: Response Code, REGEX, Validate JSON, and Max Response Time (ms). All of these can be used to validate API responses within an API Science monitor. The processing can examine each response based on all of these validation types. The API check result will include only those validations that did not pass. Thus, when your developer team reviews a failed check, they will immediately see which kinds of validation failed, and thus will be able to direct their attention to the specific problem at hand.

For this post, I included all five types of validation to my Flickr API monitor so that all possibilities are covered. Here is how the validations look when I edit the monitor:

The Javascript validation remains unchanged. It looks for a name element in the SOAP response and uses various assert() tests to determine if the response is valid.

The “Max Response Time (ms)” setting will make the API check fail if the response takes more than 1 millisecond to complete.

The “Response Code” setting will make the API check fail if the response HTTP status code is something other than 197.

The “Regex” setting performs a regular expression search on the response body, seeking the word “bluebirds”; the check fails if “bluebirds” does not exist in the response.

The “Validate JSON” setting verifies that the response body is valid JSON. If this is not the case, the API check fails.

Running the monitor produces a response body like this:

Here are the results of the validation process for the check:

All five validations failed. This is not a surprise, since the settings I entered for each type of validation are not valid. I did that to illustrate the types of message that result when a validation of each type fails.

How would our developer team address these problems? Let’s examine the Validation messages one by one.

“Response time exceed 1 ms”: One millisecond is too short a time limit for getting a response from almost any API (except perhaps an internal API). The checks summary for this monitor shows that a typical call takes at least 100 milliseconds to complete.

Here we can see that a reasonable Max Response Time might be 200 milliseconds. In that case only one of the last 10 checks would have failed due to a slow response.

“Response code is not 197”: 197 is not a valid HTTP Status Code. The typical HTTP success status code is 200 (which means “OK”).  Whether any other status code will be valid for your application depends on the APIs you are calling.

“Response does not match pattern ‘bluebirds'”: This is because ‘bluebirds’ is not in the response. However, you might want to ensure that other words appear in the response before your process attempts to use this response. For the example, you might perform a Regex search for ‘flickr.test.echo’ since that should always be present in the response body.

“Response is not valid JSON”: A SOAP response will not be valid JSON. Enabling the “Validate JSON” validation setting is useful only for APIs that return their response in JSON format.

“Problem: expected ‘Here is what we are looking for’ to equal ‘This is what we are looking for'”: In this case, the Javascript validation is correctly extracting the return item (‘Here is what we are looking for’), but the validation Javascript is testing for ‘This is what we are looking for’. The problem would be corrected by rewriting the Javascript appropriately.


The API Science platform provides diverse methods for validating responses from calls to APIs. The validation types can be combined as needed for your application. Viewing the failed validations for a specific API monitor check can provide the development team with a quick indication of where the problem might lie, facilitating their efforts to resolve the issues in the most timely and best way possible for maintaining a positive customer experience.

–Kevin Farnham