Validating API Responses and Performance Using JavaScript

Determining the present state of an API that’s critical to your product’s performance is the reason why companies apply API monitoring on a 24/7 basis. If your product depends on a sequence of calls to multiple APIs (perhaps internal as well as external), your product could appear “down” to your customers if even one of those critical APIs is down.

But how can you know for sure if one of your critical APIs is down? It’s certainly down if your call returns an invalid HTTP return code. If the API returns JSON, it’s down if the response body is not valid JSON. If the received response body doesn’t include a mandatory REGEX text sequence, then the API is down.

The API Science Validations documentation points to these means for validating your API checks. But in addition, the API Science platform supports validation of API responses using JavaScript:

Execute your own custom JavaScript for advanced testing of call results. By default API Science embeds the Chai JavaScript Library for running test assertions. Any assertion failure will flag this validation as failed and in turn will cause any user-defined alert rules to be triggered.

Using JavaScript lets you program as detailed an investigation as you need into an API’s response content to be assured that the API really is presently “up” with respect to how your product uses it.

Test Case: Validating xkcd.com API Responses

To illustrate this, let’s examine some of the ways you can use JavaScript to validate responses from a simple API monitor that calls a single API. xkcd.com is “a WebComic of romance, sarcasm, math, and language” that has an API. How can we validate this API is “up” with respect to your product’s needs?

To initiate this investigation, I went to my API Science Dashboard, and clicked on the “Add Monitor” button in the upper right of that page. Then, I created a monitor that calls the xkcd.com API:

xkcd_create_monitor

When this API call runs successfully, it returns a JSON response body similar to this:

{
  "month": "2",
  "num": 1481,
  "link": "",
  "year": "2015",
  "news": "",
  "safe_title": "API",
  "transcript": "((This is a faux-screenshot of a technical document))\n[[A figure sits at a computer upon a desk, apparently engrossed in the document which we now see before us.]]\nTITLE: API GUIDE\nRequest URL Format: domain\nuser\nitem\nServer will return an XML document which contains the requested data and documentation describing how the data is organized spatially.\nAPI KEYS: To obtain API access, contact the x.509-authenticated server and request an ECDH-RSA TLS key...\n\nCaption: If you do things right, it can take people a while to realize that your \"API documentation\" is just instructions for how to look at your website.\n\n{{Title text: ACCESS LIMITS: Clients may maintain connections to the server for no more than 86,400 seconds per day. If you need additional time, you may contact IERS to file a request for up to one additional second.}}",
  "alt": "ACCESS LIMITS: Clients may maintain connections to the server for no more than 86,400 seconds per day. If you need additional time, you may contact IERS to file a request for up to one additional second.",
  "img": "http://imgs.xkcd.com/comics/api.png",
  "title": "API",
  "day": "2"
}

The API Science JavaScript validation feature can be applied to investigate this JSON response to determine if a valid response was returned. To implement JavaScript validation testing, I edit my XKCD monitor and click the “Show Settings” link:

xkcd-editClick the “Show Settings” link and select “JavaScript” in the “Validations” column:

xkcd-edit-javascriptNow we’re ready to enter JavaScript code that will validate the API’s response text.

Using JavaScript to Validate API Response Timing and Data Size

The API Science monitoring platform provides a developer with access to all available information related to an individual monitor check, including timing information, HTTP headers, and the body of the API response. This information can be evaluated using JavaScript to create detailed levels of validation that will let you know immediately if an API that is critical for your product’s performance goes down or becomes unreliable.

Your product uses data extracted from an API (external or internal). Your customer clicks something and expects to see a result within the next second or two.

But, what happens if your customer has to wait 10 seconds to see the result they requested? Will they wait that long? Even if the page they requested from your app is ultimately received, if they had to wait 10 or 15 seconds to see that result, might they conclude that your product is, effectively, down?

For this reason, you need to know if an API that serves critical data for your product is suddenly experiencing unusual delays (even if the API ultimately delivers the requested information).

API Science’s monitors provide a “Max Response Time (ms)” validation. If the response from the API takes longer than n milliseconds, the API check fails, for example, this setting:

max_response_time_validation

will cause a check on my “br Ireland” monitor, when I call the World Bank’s Countries API from Ireland, to fail if a response is not received within 100 milliseconds.

World Bank Countries API queries return a fixed content size. So, simply specifying a time limit can suffice for assessing whether the API is up or down with respect to your product’s needs.

But, what if the situation is more complex? What if your product can provide quick looks in response to some requests, but your customers can also request large, detailed reports? In this case, it’s safe to assume that when your customer requests a quick summary, they’ll want to see it appear quickly on their device; meanwhile, if they then click your “Show Me All the Details” button, they’ll be willing to wait longer to receive the result.

In this case, a mathematical equation defines whether your customer will view your product as up or down. Your reasonable customer will consider: “how long did it take to receive the data I requested, compared with the amount of data I asked to receive?”

Can you apply your API Science monitors to evaluate your product’s performance based on this type of question? The answer is: “Yes!” — if you use API Science’s JavaScript validation capability.

Editing my “XKCD Monitor” and clicking “Show Settings,” I see:

xkcd_editIn the “Validations” pull-down menu, I select “JavaScript,” which opens a box into which I can type the JavaScript programming that I’d like to apply to each instance when my monitor runs a test on the xkcd.com API:

xkcd_javascript_entry

Here’s what I enter:

var timing = context.response.timing.total;
var size = context.response.meta.downloadSize;

if (timing > 5000) {
        assert(size > 10000, "Download size vs speed issue");
}

The timing variable returns the total number of milliseconds that took place between the API request being initiated and the full response being received. This is the value that defines success or failure when you create a “Max Response Time (ms)” validation described above.

In this case, however, we want to measure time with reference to how much data our customer requested (assuming they will be more patient if they requested more data). So, we create a size variable that stores the number of bytes that were downloaded in order to fulfill the request.

Now, using JavaScript and the Chai assert library, we can calculate the bytes per millisecond for the API call. If it took more than 5 seconds to download the data, then we check the download data size. If the data size was greater than 10,000 bytes, we consider that fine, and the API check is considered successful; if the data size was 10K bytes or smaller, we consider that a problem that might make our customer consider our own product to be “down.”

Applying JavaScript validation to your API Science API monitoring lets you make these distinctions, and lets you alert your 24/7 team when an issue arises.

Using JavaScript to Validate HTTP Headers in API Responses

There is much more that can be inspected and validated using API Science’s JavaScript validations capability, including examining the HTTP Headers that are returned with the XKCD API’s response.

Here are the response headers from a typical call to the API:

xkcd-response-headers

You might ask: “Why check the response headers? Isn’t the body all that really matters?” One reason is that, while the body is indeed what matters most in terms of data, the body of an API response can have any structure; whereas HTTP headers are defined by RFCs (Requests for Comments) that are agreed upon by the global World Wide Web community. Currently RFCs 7230 through 7237 define the requirements for HyperText Transfer Protocol (HTTP). Thus, unlike the structure of a response body (which can be entirely freeform), the structure of the HTTP headers segment for every API response is known. This makes analyzing the response headers relatively straightforward, whereas analyzing the response body may require more complex coding.

The HTTP response headers for my monitor’s call to the XKCD API provide numerous opportunities for validating that the latest API check succeeded in the manner your application requires. For example, the “Content-Type” header provides the MIME type with which the response body is formatted (“MIME” is an acronym for “Multi-purpose Internet Mail Extensions”).

The Content-Type header illustrates the value of applying JavaScript to validate headers. If your application expects to receive a JSON (JavaScript Object Notation) response body, then from the point of view of your application, this API is down if the response’s Content-Type header doesn’t include application/json.

You can verify this using an API Science JavaScript validation. Reviewing the XKCD response headers in the example above, we see that, indeed, the Content-Type header includes application/json.

Here’s how we set up a JavaScript monitor validation that will have an API check fail if the Content-Type header doesn’t include this. I edit my XKCD monitor and select “JavaScript” in the Validations pull-down menu:

xkcd_javascript_entry

In the text box, I enter:

var contentType = context.response.headers['Content-Type'];

assert( contentType.indexOf("application/json") >= 0, 
        "Not a JSON response");

The contentType variable retrieves the value that corresponds to the Content-Type entry in the response header. That string is then evaluated using a JavaScript indexOf command. This command identifies the location of the requested string (application/json) within the entire string (contentType). The value 0 means the requested string is at the very beginning of the full string. If the requested string does not exist in the searched string, then a value of -1 is returned.

The assert statement checks if the JavaScript indexOf command returns a value greater than or equal to zero. If the assert fails, a “Not a JSON response” message is issued.

With this validation in place, your team will know if a critical API suddenly ceases to deliver body content in the format your application requires.

Using JavaScript to Validate HTTP Response Codes in API Responses

In addition to using JavaScript to validate HTTP headers received by your API Science monitors, JavaScript can be applied for fine-grained validation of the HTTP Status Code for each API response.

The status code normally associated with a successful response is 200 (“OK”). However, all 2xx status codes indicate that from the server’s point of view a valid request was received, it was processed, and an appropriate response was delivered to the client. The 2xx status codes that are greater than 200 provide additional information about the processing performed by the targeted server, or by a proxy server along the Internet path between the client and the target server.

The Internet is a complex system. Request and response traffic can be routed in many different ways, across multiple chains of proxy servers that relay the information packages. Another complication is caching, wherein data is stored in temporary locations along the path between a client and server. Caching is done to reduce the overall traffic level, thereby increasing the speed individual clients experience. Routing and caching are just two of the factors that can cause widely variable response times when the same request is repeatedly sent from the same client to the same server (as in one of your API Science  monitors). For example:

response_time_variationThe 2xx HTTP status codes report information about what happened at the server and what happened as the response package traversed the path between the endpoint server and the requesting client.

This three-digit code can be used by your monitor to let you make decisions about when your product will consider the API up, and when it will consider the API down. If the status code indicates that, while the request was successfully processed, the response body will not include information your product depends on, then there’s no need to even attempt to process the body: from your product’s point of view, the API is down.

Here’s a brief review of what the 2xx HTTP status codes mean (in addition to “Success”):

  • 200 OK: no additional information is provided
  • 201 Created: a new data item was created on the server
  • 202 Accepted: the server has received the request, but hasn’t finished processing it
  • 203 Non-Authorative Information: a proxy server sent the response based on a response from the endpoint server, but the response isn’t identical to the response the proxy received
  • 204 No Content: the server processed the request and returned no content
  • 205 Reset Content: like 204, but the requester should reset its data view
  • 206 Partial Content: the content is a subset of the entire data set because the request asked for this
  • 207 Multi-Status: the response body is an XML document containing additional status information
  • 208 Already Reported: a follow-on related to 207
  • 226 IM Used: the response is a representation of one or more instance manipulations (IMs) applied to the current instance

If the response from this API is critical for your product, clearly status codes 200 and 201 are perfect, but status code 202 makes the API down. The remaining status codes are more complex: depending on the software that creates your product, API responses with some of these status codes may represent responses that can be processed.

Assume we’d like to have the API check fail if the HTTP status code is any value other than 200 or 201. Here’s how we implement that using API Science’s JavaScript validation. Edit the monitor, click “Show settings” and select “JavaScript” in the “Validations” pull-down menu:

xkcd_javascript_entry

In the text box, enter:

var statusCode = context.response.meta.statusCode;

assert( statusCode == 200 || statusCode == 201, 
        "Expected a 200 or 201 response");

Then, at the bottom of the page, click the “Save Monitor” button.

We’ve now created a JavaScript validation that will say the API check failed if the response HTTP status code is anything other than 200 or 201.

Using JavaScript to Validate JSON API Response Bodies

We haven’t yet investigated how you can use the API Science JavaScript validation facility to validate the API response body. We’ll use my XKCD monitor to see how this can be done. This monitor queries the XKCD API using a simple GET request sent to this API endpoint:

http://xkcd.com/1481/info.0.json

When the API is up, it returns an API response body similar to this:

{
  "month": "2",
  "num": 1481,
  "link": "",
  "year": "2015",
  "news": "",
  "safe_title": "API",
  "transcript": "((This is a faux-screenshot of a technical document))\n[[A figure sits at a computer upon a desk, apparently engrossed in the document which we now see before us.]]\nTITLE: API GUIDE\nRequest URL Format: domain\nuser\nitem\nServer will return an XML document which contains the requested data and documentation describing how the data is organized spatially.\nAPI KEYS: To obtain API access, contact the x.509-authenticated server and request an ECDH-RSA TLS key...\n\nCaption: If you do things right, it can take people a while to realize that your \"API documentation\" is just instructions for how to look at your website.\n\n{{Title text: ACCESS LIMITS: Clients may maintain connections to the server for no more than 86,400 seconds per day. If you need additional time, you may contact IERS to file a request for up to one additional second.}}",
  "alt": "ACCESS LIMITS: Clients may maintain connections to the server for no more than 86,400 seconds per day. If you need additional time, you may contact IERS to file a request for up to one additional second.",
  "img": "http://imgs.xkcd.com/comics/api.png",
  "title": "API",
  "day": "2"
}

This is a JSON response body: a list of keys, followed by a colon, followed by a value. In your API Science monitor, you can use JavaScript to validate any of these fields.

Let’s say your API at this moment relies on the response “day” value to be “2”. If the API isn’t returning this JSON value, then, from the point of view of your product, the API is down.

To configure this validation into your API Science monitor, edit the monitor, and click “Show Settings”. In the “Validations” pull-down menu, select “JavaScript”:

xkcd_javascript_entry

Enter the following code into the text window:


var body = JSON.parse(context.response.body);
var day = "2";

assert.equal( body.day, 
              day, 
              "Wrong day");

This code inspects the API’s JSON response body’s “day” variable, and tests if its value is “2”. If the value is “2” then the API check validation passes. If the values isn’t “2” then the check fails.

What will you see in your API Science monitor if this type of validation fails? “2” is the expected response for my XDCK monitor. If I edit the second line of my JavaScript validation code to be:

var day = "3";

and I then run my monitor, I will see this in the result:

xkcd_wrong_dayThe API is down from your product’s point of view, because your product depends on the value of the “day” variable for this particular query to be “2”.

JavaScript is a vast programming language that supports highly-detailed investigation of content that is returned in API response bodies. API Science provides unlimited ability for you to analyze response bodies to determine if, from your product’s point of view, an API you depend on is up or down.

Using Regular Expressions and JavaScript to Validate API Responses

Regular expressions (RegEx) are “a sequence of characters that define a search pattern, mainly for use in pattern matching with strings, or string matching.” Amazingly, the RegEx concept was invented back in the 1950s, by U.S. mathematician Stephen Cole Kleene, who was a student of Alan Turing, among others.

In his book Mastering Regular Expressions, Jeffrey E. F. Friedl writes:

Regular expressions are the key to powerful, flexible, and efficient text processing. Regular expressions themselves, with a general pattern notation almost like a mini programming language, allow you to describe and parse text. With additional support provided by the tool being used, regular expressions can add, remove, isolate, and generally fold, spindle, and mutilate all kinds of text and data.

Consider a response body from an API that your product depends on. Your product sent a request to the API, and a response was returned. But does that response contain the information that your product needs in order for your product to appear “up” to your customers?

API Science’s JavaScript validation capability provides you with the opportunity to apply regular expressions to analyze the content of an API’s response body, even down to the level of the text that’s in a particular JSON field. Here’s how we can do this.

My XKCD API monitor queries the XKCD API. A successful response has a body that looks something like this:

{
  "month": "2",
  "num": 1481,
  "link": "",
  "year": "2015",
  "news": "",
  "safe_title": "API",
  "transcript": "((This is a faux-screenshot of a technical document))\n[[A figure sits at a computer upon a desk, apparently engrossed in the document which we now see before us.]]\nTITLE: API GUIDE\nRequest URL Format: domain\nuser\nitem\nServer will return an XML document which contains the requested data and documentation describing how the data is organized spatially.\nAPI KEYS: To obtain API access, contact the x.509-authenticated server and request an ECDH-RSA TLS key...\n\nCaption: If you do things right, it can take people a while to realize that your \"API documentation\" is just instructions for how to look at your website.\n\n{{Title text: ACCESS LIMITS: Clients may maintain connections to the server for no more than 86,400 seconds per day. If you need additional time, you may contact IERS to file a request for up to one additional second.}}",
  "alt": "ACCESS LIMITS: Clients may maintain connections to the server for no more than 86,400 seconds per day. If you need additional time, you may contact IERS to file a request for up to one additional second.",
  "img": "http://imgs.xkcd.com/comics/api.png",
  "title": "API",
  "day": "2"
}

This is a JSON response body. The API Science JavaScript validation capability enables you to inspect JSON responses on a field-by-field basis.

Let’s say the information that’s critical for you product is located in the transcript field in the JSON body: if the word “faux” isn’t in the body, then the API is down as far as your product is concerned, because your product integrates what follows that text into the result you present to your customers.

How can you verify that the XKCD API is “up” for your product using your API Science monitor? It’s easier than you might think, using JavaScript and regular expressions…

Edit your monitor, and click “Show Settings”. In the “Validations” pull-down menu, select “JavaScript”:

xkcd_javascript_entryEnter the following into the text window:


var body = JSON.parse(context.response.body);

assert( body.transcript.search(/faux.*/i) > -1, 
              "transcript field does not contain 'faux'");

The regular expression is the text contained within the parentheses following search. What does this RegEx mean?

The /faux.*/ means the text should be searched for the text “faux”; if that’s not found, the search fails. The “.” means that “faux” must be followed by at least one character (i.e., the “faux” that was found cannot be at the end of the string). The “*” means that any number of characters can be in the string after “faux” (these characters would likely be the ones your product depends on). The “i” after the last slash means that the string should be analyzed in a case-insensitive manner (which might or might not be fitting for your particular application).

Save the updated monitor. Now, if a response is ever received where the JSON body’s transcript field doesn’t contain the word “faux” followed by other characters, your API Science monitor will declare the XKCD API “down” — and it will be time for your team and/or software to take appropriate action on your customers’ behalf!

Conclusion

Receiving a response from an API request provides no guarantee that the response contains the information that is critical for your product. The API Science JavaScript validation capability enables you to develop API monitors that validate the API responses your product depends on at a fine-grained level, so that your team will immediately know whenever your customers will view your product as being down or degraded.

–Kevin Farnham