API Caching - Reducing Network Calls

What is Caching?

Caching is the ability to store copies of frequently accessed data in several places along the request-response path. 

In terms of REST APIs, caching is storing the response of APIs having relatively high response time in an in-memory or distributed system, so that the response can be fetched from the stored system within a short amount of time.

Why implement caching?

There are generally two basic reasons for which REST API developers implement caching:

• Reduce Network Calls: Most API operations are read operations, so when data doesn't change then there is no need to fetch data again and again from a database. For this, we can have an in-memory cache where we can save the data as key-value pair and fetch the data from the cache. As a result of this network calls will be reduced significantly.
• Reduce Load on Database: When we are using a cache, then for most of the time we will be fetching data from the cache and won't be querying the database much. So, as a result, database query/computational costs are decreased and so the concurrency of the database increases.
  

Types of Caching

API Caching can be sub-divided into in-memory caching and distributed caching based on where data is being stored.

• In-Memory Caching: In memory caching stores data inside the application server itself. A certain portion of the memory in the application server is allotted for caching. It is not distributed and neither scalable.
  
• Distributed Caching: A distributed cache is a cache maintained by multiple servers, typically maintained as an external service which the application server can access. Distributive Caching is useful when we have to store lots of data in the cache. It can improve performance by scaling horizontally.

How much improvement is possible?

Let us take an example here. Suppose we have an API that fetches the transaction data for any user in an online shopping site. Let's assume the Database Access Time = 30ms,  Cache Access Time = 8ms and Computational Buffer = 5ms.

Now if a user makes the API call 10 times, then without using cache the total time taken will be = (30+5)*10 = 350ms
If caching is there, then 1st request won't get any data from cache, so the API will query the database for the first time, after that the result will be stored in the cache and from the subsequent requests data will be fetched and returned from the cache.
So, using cache total time taken will be = (30+8+5)*1 + (8+5)*9 =  160ms

Here we see that for 10 API calls, the total response time has reduced by almost 54%.

The problem - Cache Invalidation

There are only two hard things in Computer Science: Cache Invalidation and naming things
                                                                                                         - Phil Karlton

Caching is a great solution for reducing network and calls and improving the performance of our APIs. For a low response time, we want to cache our content for a longer period of time. But at the same time, we should be able to return fresh data whenever there is an update in our main database. Cache Invalidation is the process in which the content stored inside a cache replaced or removed as the data it contained is now no longer useful.
For example, there is a cache which contains the basic information of an user. An API fetches user  info from this cache and returns it to the client. Now suppose the user changes his phone number and the update happens in our main database. So for this, either the cache has to be updated with the new data or the cache has to be removed so that the API directly goes and fetches from the main database.
Now, when and how to invalidate a cache depends mostly on the business requirements, like whether the updated data has to be instantly visible to the client or it will be available after a certain interval of time, whether comprising on response time is possible or not, etc. For this there are these 3 techniques:

• Write-through cache: Here, data is updated in the database and cache at the same time. This won't compromise with the response time and updated data will be available instantly but the write operations will take much more time.
• Write-around cache: Here, data is directly written to database bypassing the cache. This reduces the write operation costs but the response time for read requests will be high.
• Write-back cache: In this case, data is directly written to cache only and success message is sent to client. Then at specified intervals the data update is done on main database by automated scripts. Here also updated data will be available instantly but there is risk of data loss if cache fails.

Well, that is pretty much about the basics of API caching. In this article, we covered what is caching and the different ways it can be implemented, also how it can drastically improve API performance. In the end we discussed about cache invalidation and it's techniques. If you have any doubts or suggestions to improve this article do mention them in the comments. Stay Safe and Happy Coding!