7 Ways to Customize API Monitoring Using Global and Local Variables

It is possible to configure API monitors with rigidly fixed parameters. However, using the API Science platform, it is also possible to configure API monitors using both account-wide global variables and monitor-specific local variables, in order to provide your team with a deeper view into what your customers are experiencing.

Introduction to API Science’s Global Variables

The API Science platform provides capability for saving, retrieving, and deleting global variables. The Parameters section on the API Science Help page provides this description:

You can set any number of query string, POST, and PUT parameters using the Params input fields. Each time you add a new variable a new pair of Key/Value input fields will display. You can also append parameters directly to the URL as ?param1=value1.

Setting Parameters

To illustrate setting variables, I’ll create a new monitor by going to my API Science Dashboard and clicking the “Add Monitor” button on the right side of the page; then select and enter the basic configuration items for the new monitor:

Here, I’ve named my monitor “Set Dog and Meal” and given it the tag “dogs” (the reason will become plain in the next step). My monitor will use an HTTP POST command to access the API Science variables API (https://api.apiscience.com/v1/variables), running my monitor every hour from Washington, DC.

The basics are configured, but clicking the “Test Now” button at this point would result in an HTTP 401 return code. The 401 HTTP return code indicates that access to the API is unauthorized, because authentication is required. This problem can be addressed by clicking the “Show Settings” link.

“Show Settings” lets you provide the detailed configuration for your monitor, including parameters, request body, headers, validations, and pre-process script:

To address the authorization problem, I add an HTTP header with the key Authorization and the value Bearer xxx (where xxx is my API Science API key). [Note: to obtain an API Science API Key, go to your account’s “Settings” page and click on “API Keys.”]

I also add a Content-type header, specifying that the content I am posting is in JSON format (application/json).

And, I enter the parameters I’d like to set:

  • Variable my_dog with value Fido;
  • Variable my_meal with value steak;
  • Variable my_status with value hungry.

Saving the monitor, then running it, produces a successful check:

We’ve now successfully posted three parameters to the API Science parameters API using our API Science account to confirm that:

  1. my dog’s name is Fido;
  2. my upcoming meal is steak;
  3. I’m hungry.

Retrieving and Verifying API Science Global Variables

Having set API Science variables using the “Set Dog and Meal” API monitor, we’ll now create a new monitor that reads and verifies the status of those parameters.

The “Set Dog and Meal” monitor runs every 60 minutes, setting the following parameter/value pairs:

  • my_dog: Fido
  • my_meal: steak
  • my_status: hungry

To create a monitor to verify this status, I click the “Add Monitor” button on my API Science Dashboard, and define the basic configuration elements:

Here, I’m accessing the API Science Variables API (https://api.apiscience.com/v1/variables) using an HTTP GET command to retrieve data from the API. I’ve named the monitor “Dog and Meal Status,” tagged it into my “dogs” category, and the monitor will run every 10 minutes from Washington, D.C.

Clicking the “Test Now” button at this point returns a 401 HTTP return code because the API Science Variables API requires authentication. I click the “Show Settings” link to resolve these issue:

Here, I’ve added an HTTP header with the key Authorization and the value Bearer xxx (where xxx is my API Science API key). [Note: to obtain an API Science API Key, go to your account’s “Settings” page and click on “API Keys.”]

Saving and running this monitor retrieves all the parameters I’ve stored in the API Science Variable API store:

So, the current status is as expected: my planned upcoming meal is steak, I’m hungry, and my dog’s name is “Fido.”

Adding Monitor Validation

Right now, my monitor runs every 10 minutes. When I view the latest results for my “Dog and Meal Status” monitor, I will see successful results if the API responded and returned a list of parameters. But what if the primary value of our product is tied to the existence of a specific parameter?

Let’s say the key item for our product is the “my_meal” parameter. If that parameter doesn’t exist, then our product will be seen as down by most of our customers. How can we verify that the API is not only returning requested data, but is also returning the “my_meal” data that we know is most critical for our customer base?

The API Science platform provides multiple means for validating API responses, including:

  • Response Code – the HTTP status code that is returned with the API’s response text;
  • Regex – a Regular Expression search of the response text;
  • Validate JSON – verifies that the response text is valid JSON;
  • JavaScript – analyzes the response text using user-provided JavaScript;
  • Max Response Time (ms) – declares the API call to have failed if a response is not received after a specified number of milliseconds.

To verify that the response text includes “my_meal” we edit the monitor settings to include a Regex validation:

Now, whenever this monitor runs, it will be considered a failed run if “my_meal” is not included in the response text.

Next, I’ll demonstrate cases that test this validation.

Deleting API Science Global Variables

So now, we’ll create a new monitor that deletes an API Science global variable, then study what effect this has on the earlier monitors.

My “Set Dog and Meal” monitor sets the following variables (creating them if they don’t already exist):

  • my_dog: Fido
  • my_meal: steak
  • my_status: hungry

My “Dog and Meal Status” monitor retrieves the variables, and uses an API Science Regex validation to verify that the text “my_meal” was returned as part of the response body.

To demonstrate deleting variables, I create a new monitor:

Here I’ve configured a monitor named “Fido Eats” that accesses the API Science variables API (https://api.apiscience.com/v1/variables) using the HTTP DELETE; the monitor runs every hour from Washington, DC.

Clicking the “Show Settings” link lets me enter the required authentication credentials and configure additional details:

The Authentication HTTP header provides the API with my API Science API key. The Content-Type header identifies that the request body is in JSON (JavaScript Object Notation) format. The request body identifies the global variable to which the DELETE method will be applied (my_meal).

Saving and running the monitor produces this result:

The HTTP status code 200 indicates that the API call was processed successfully. The response body lists the current set of global variables for my API Science account, which lets me verify that a global variable named “my_meal” no longer exists.

But now, when I go to my “Dog and Meal Status” monitor, I see the following:

The monitor says the API is down. Scrolling down to “Recent Checks” I see:

The response code is 200, indicating that the actual call to the API succeeded (the API processed the request and returned a response). The status of the call is “Failure” because a specified validation requirement was not passed.

Clicking on the date and time link brings us to the details for the failed API check:

The call failed because the Regex validation for this monitor found that the response body does not include the required text “my_meal”; this is the case because the “Fido Eats” monitor deleted that variable from my account.

My “Dog and Meal Status” monitor won’t remain down forever, though. The next time my “Set Dog and Meal” monitor runs, the “my_meal” variable will be recreated (and reset to “steak”), and all three monitors will report statuses of “Success” — until the next time Fido eats and I come looking for my steak…

The three monitors created thus far illustrate how API Science global variables can be used to coordinate multiple monitors. The variables provide a means for communicating information between monitors, facilitating more complex verification of the status of the APIs that are critical for your product.

Introduction to API Monitor Local Variables

In addition to providing account-level global variables, the API Science platform supports the creation of local variables that are specific to an individual API monitor.

The fact is, APIs change over time. Also, your company’s own specific usage of a particular external API may change. And certainly your company’s need to monitor the performance of your own evolving internal APIs will change over time.

Employing API Science’s monitor local variables can make this necessary evolution easier to manage. Instead of creating a new API monitor every time something changes, if you configure your monitor using variables, you can simply change the variable, and your monitor is instantly reconfigured to analyze the performance of your new product.

Setting Monitor Variables Using a JavaScript Pre-Process Script

When you create or edit an API Science monitor, the “Settings” block in the screen lets you enter “Pre-Process Script” code. This code block provides the opportunity to enter JavaScript code that defines variable values which will be integrated into your standard monitor processing.

For example, let’s say we want to create a monitor that sends an HTTP GET request to some *.com domain, while also sending it a GET query parameter, and a specific HTTP header. To initiate this, from our API Science Dashboard, we can click the “Add Monitor” button. Clicking the “Show Settings” link brings up a page that includes a “Pre-Process Script” field like this:

Here, I’ve set variables that are local to my new monitor to define a:

  • testdomain: a portion of the URL this monitor will call;
  • queryparam: the value for a parameter this API is configured to receive as part of the GET query URL;
  • otherparam: another value the API is configured to receive and process;
  • otherheader: the value of an HTTP header this API knows how to receive and process.

So, here, we have configured an easily editable set of variables which can be utilized to create a flexible API monitor that is readily adaptable as the evolution of the external and internal APIs upon which our product depends proceeds.

Implementing Monitor Variables in Your Monitor

To implement these specified variables, we can edit the monitor settings like so:

Here I’m designating that the API I’m calling is http://, concatenated with the value of my context.testdomain variable, concatenated with .com/echo?foo=, concatenated with the value of my context.queryparam variable. In addition, I’m designating that I’m sending a myotherparam value of my context.otherparam variable, and the X-TestHeader value that’s represented by my designated context.otherheader value.

Testing the monitor

So, why not test this new monitor? Clicking the “Test Now” button yields this request result:

The variables resulted in this GET request:

http://scooterlabs.com/echo?foo=bar&myotherparam=myvalue

This request was sent with an added X-TestHeader HTTP header with a value of myheader, and an extra parameter named myotherparam with a value of myvalue.

Here’s the response:

The point of Scooterlabs.com is to provide an API endpoint that receives API requests, including variables and headers, and returns information on what was received and how it was processed–basically, the received information was read, turned into a response, and statistics about the transaction are also returned. So, here we see that the designated variables were received and processed into the response message.

This example illustrates how, as you monitor the critical external and internal APIs your product depends on, you can utilize monitor variables to add flexibility to your API Science monitors, in order to address the inevitable evolution those critical APIs will see.

How to Use the Results from One API Call in Another API Call

In many cases, the availability of your product depends on a sequence of API calls (to both external and internal APIs). Information retrieved from one API may be a critical input for your subsequent call to a different API. If the first call fails, the second can’t return a valid result.

So, a question is: can you create an API monitor that will test this complexity? Can you create a monitor that uses information contained in the response from one API call to create a new request that you send to either the same or a different API?

The API Science platform supports multi-step monitors that allow you to do this. The Multi-Step Example monitor is a six-step monitor that queries the Wordnik.com API. The monitor’s first step is to request an authentication token, which is required for subsequent access to the Wordnik API. Here’s an example Step 1 request:

Here, we are sending a request for a Wordnik authentication token, which we’ll use in our Step 2 request. Note: the “FILTERED” fields in the image represent private information that should not be made public (our API key and password). In creating your own Wordnik request, you’d substitute your Wordnik API key and password.

An example response to this request is:

Here, the response includes a token variable (the variable’s value is overwritten in the image) that allows us to submit additional requests to the API for more specific data.

Now, in the next step for this monitor, we can use this value that was received from the first step. To do this, in Step 2 we create a JavaScript variable named auth_token that is assigned the value of the token element that was returned from the first step:

The context element is a container for information that can be utilized in the Step 2 API call. But, after the Step 1 API call has completed, its response body remains available for interrogation by a JavaScript preprocess script that prepares the Step 2 API call.

Here, in the Step 2 Script:

    context.auth_token = JSON.parse(context.response.body).token

the Step 1 response body is parsed to extract the JSON token value. This value is stored as context.auth_token, then used as a parameter in the Step 2 GET request. Then, for the Parameter field for auth_token, you can use this syntax in the Value side {{context.auth_token}}. This permits Step 2 in this multi-step monitor to succeed:

In this example, we have gathered information from the JSON response to an API request, and integrated that data into a subsequent API request.

This illustrates the potential the API Science platform offers: if a chain of different API calls is required for your customers to consider your application to be up, you can configure your API Science monitors to ensure that your team will be aware of issues when and where they occur at any point in your API processing sequence.

Configuring API Monitors for Specific Response Formats

Global variables can be utilized to store any kind of data that is needed as a reference resource within a complex API monitoring process.

For example, global variables can be created by one API monitor, then that data store can be read as input data by another API monitor. Previously I illustrated how to create a monitor that sets global variables. The monitor accesses the API Science variables API using the POST method, and writes a set of variable names and values. Since global variables exist at the account level, these variables and their current values are accessible by any other monitor in the user’s account.

My post Retrieving and Verifying API Science Global Variables illustrates how an account’s global variables can be retrieved by a monitor, and provides a simple example of how a monitor can respond based on the retrieved values.

Deleting API Science Global Variables shows how an API Science monitor can delete global variables.

While these articles use monitors to manage and access an account’s global variables, global variables can also be created, read, updated, and deleted (CRUD) by directly accessing the API Science variables API using modern software platforms including cURL, Java, Python, etc.

Configuring API Response Format

Assume your product depends on a number of APIs, both external and internal, that can provide output in multiple formats (XML, JSON, etc.). If your internal software is optimized to process data in a given format, then ideally your API monitors would return their response data in that format.

The World Bank has an API whereby a call can request and receive basic information about a country. For example, to receive the summary information about Brazil, you can point a web browser to: http://api.worldbank.org/countries/br. If the API is up, entering this into your browser will produce an XML result similar to this:

WorldBankCountryXML

This may be the information your product requires. But, what if your product is tuned for working with JSON data, not XML? The API: Country Queries page shows that the World Bank’s countries data can be returned in either XML or JSON format. The API: Basic Call Structure page shows that the response format can be selected by adding &format=xxx (where xxx is xml, json, or jsonP) to the API call.

Given this, we can define the format in which we’d like to receive the response from the World Bank Countries API by appending a parameter to our call to the API, and have our API monitor validate a response delivered in that format.

In the case of the World Bank Countries API, we can create a monitor that explicitly requests JSON responses by defining a “format” parameter as being “json”:

When we run this monitor, a typical response body looks like this:

In this example, we’ve illustrated how to manually configure a monitor to query the World Bank Countries API, requesting that the data be returned in JSON format.

Next we’ll illustrate how API Science global variables can be used to automatically configure monitors with respect to parameters that an API accepts.

Using Account Global Variables to Receive API Monitor Data in Specific Formats

Calls to the World Bank Countries API can be configured to output the API data in multiple formats, including XML and JSON.

The API Science platform supports the creation of global variables that are accessible by all API monitors in a user’s account. Global variables are defined and set using a POST call to the API Science variables API (https://apiscience.com/v1/variables), as illustrated in my post Introduction to API Science’s Global Variables. For example, a monitor could be created that sets the global variable wb_format to have the value json:

With this global variable set, we can create a new multi-step monitor that retrieves the value of the variable, then creates a call to the World Bank Countries API that requests the API response to be returned in the specified format.

Step 1. Access the Response Format Global Variable

The first step in the multi-step monitor retrieves the value of the account’s wb_format variable:

The request asks the API Science variables API to return all variables for the user’s account. Here’s an example response body:

In this example, the account has assigned multiple global variables. But our current multi-step World Bank Countries API monitor requires only the wb_format variable. The Step 1 Validations code shown above verifies that the wb_format variable exists in the response body.

The JavaScript validation does the following:

  • var body = JSON.parse(context.response.body); assigns to JavaScript variable body the JSON response body shown above;
  • var fmt = "json"; identifies that this multi-step monitor expects to receive JSON-formatted data from the World Bank Countries API;
  • assert.equal(body.data.wb_format, fmt, "Invalid wb_format: " + body.wb_format); applies a JavaScript assertion that will make Step 1 of the monitor fail if the value of the wb_format variable is not equal to the fmt variable (i.e., json).

If Step 1 of the monitor succeeds (global variable wb_format is json), then Step 2 of the monitor will proceed.

Step 2: Call the API, Requesting the Response in the Defined Format

In Step 2 of the monitor, we extract the value of the wb_format variable from the Step 1 response, and use it to create our Step 2 API request.

Step 2 of our multi-step monitor looks like this:

Here, we define a local variable named format for this monitor, and we define it to be the value of resp_format that is defined in this step’s Pre-Process Script. The Pre-Process script accesses the response from Step 1, and assigns the value of that step’s wb_format variable to the local variable resp_format.

This value is assigned to the format parameter, which is sent as part of the request to the World Bank Countries API.

The World Bank Countries API receives this request, with the format variable set to json, and it returns its Step 2 response in JSON format, for example:

We’ve now illustrated how API Science global variables can be used to configure API monitors. The World Bank Countries API provides outputs in various formats. In this example, we define a global variable that selects the requested output format from the World Bank API. The variable is read in Step one of the monitor, then a request to the World Bank API is constructed based on the global variable’s value.

This illustrates how the API Science platform can be utilized to flexibly configure large numbers of monitors that are critical for your company’s product.

Conclusion

In this post we’ve described in detail how to utilize the API Science platform to test APIs that are critical for your product using global (account level) and local (monitor level) variables.

–Kevin Farnham