Analyzing API Performance Binned by Hour of Day

Performance data from the API Science API can be analyzed in many different ways. For example, a recent post presents A Graphical View of API Performance Based on Call Location. The analysis uses cURL statistics to compare the performance of monitors that call the World Bank Countries API from various locations around the globe over a one-week period:

This plot illustrates the point that the performance of calls to the Washington, DC-based World Bank API is significantly dependent on call location.

Another useful analysis is the performance of an API with respect to hour of day. That is, compute and display the average performance between midnight and 1 a.m., 1 a.m. to 2 a.m., etc. The performance data in the API Science API is timestamped using Universal Time. The performance data can be aggregated by hour using the call to the API.

In this experiment, we get the performance data for a monitor that runs in Ireland, calling the World Bank Countries API. The following csh shell script is executed at the command line to retrieve the data from API Science:

curl 'https://api.apiscience.com/v1/monitors/1572022/performance.json?preset=lastWeek&resolution=hour' -H 'Authorization: Bearer NN__my_authorization_code'

Here, 1572022 is the monitor number, preset=lastWeek defines that performance data for the past week is requested, resolution=hour means the performance data should be binned by hour of day, and the Authorization code is your key for accessing the API Science API.

The JSON output from the csh script that acquires the data is written to a file. Here is an example of the data for a particular hour:

"startPeriod":"2019-02-15T22:39:24.478Z","endPeriod":"2019-02-15T23:39:24.478Z"},{"averageResolve":38.69,"averageConnect":74.46,"averageProcessing":84.05,"averageTransfer":0.15,"averageTotal":197.35

For this analysis, we’ll use the startPeriod to define the hour of day. The two digits after the “T” in startPeriod identify which hour-of-day bin will include that data. In this example, the data will be binned into the Hour 22 data set.

Each hour-of-day bin will average at most seven sets of call statistics (one for each day of the week for that hour of day). If the World Bank API was down at times during the week, then some bins may average fewer call statistic instances.

As in the recent series of blog posts, we use Python to read the performance JSON into a Python dictionary:

After initializing arrays to store the summed data for each hour of the day, we step through the performance entries, using the hour of startPeriod to identify the bin into which each entry is merged:

The avgTot value is checked for its existence: if no monitor data was recorded for the given hour, perf['data'][i]['averageTotal'] will have the Python value None, and this value will be passed into avgTot. We only sum hours that have actual performance values into the hourly bins.

The average performance for each hourly bin is computed by dividing the summed performance for that hour of day by the count of performance instances for that hour of day:

The next steps in the Python script use Python’s MatPlotLib plotting library to create the plot and save it to a PNG file:

Here is an example plot: one week of data calling the World Bank API from Ireland, binned by hour of day:

This is interesting. Performance was relatively consistent for hours between midnight and 23:00 Universal Time. But, for some reason, performance between 23:00 and midnight on average took more than twice as long as performance for all the other hours of the day. This would be something for your team to investigate. You would not want it to be a regular pattern that between 23:00 and midnight your product always exhibits poor performance.

To perform this investigation, we’ll have to drill down into the hour 23:00 data. I’ll begin this investigation in my next post, by statistically analyzing the data within each hour-of-day bin.

–Kevin Farnham