Caching

Caching will enable you to make vastly better use of the resources you already have as well as making otherwise unattainable product requirements feasible. Caches take advantage of the locality of reference principle: recently requested data is likely to be requested again.

Application Server Cache

Placing a cache directly on a request layer node enables the local storage of response data. If the request layer is expended to multiple nodes, it is still quite possible to have each node host its own cache. However, if your load balancer randomly distributes requests across the nodes, the same request will go to different nodes, thus increasing cache misses. Two choices for overcoming this hurdle are global caches and distributed caches.

Content Delivery (or Distribution) Network (CDN)

CNDs are a kind of cache that comes into play for sites serving large amounts of static media. In a typical CDN setup, a request will first ask the CDN for a piece of static media; the CDN will serve the content if it has it locally available. It if isn’t available, the CDN will query the back-end servers for the file, cache it locally, and serve it to the requesting user.

If the system we are building is not large enough to have its own CDN, we can ease a future transition by serving the static media off a separate subdomain using a lightweight HTTP server like NGINX, and cut-over the DNS from you servers to a CND later.

Cache Invalidation

When data is updated in the database, then that data has to be refreshed in the cache as well. This is called cache invalidation.

There are three main methods of cache invalidation

Write-through cache - Data is written to the cache and database at the same time.
Write-around cache - Data is written to the database writing to cache. Data is written to cache when a request results in a ‘cache miss’, at which point data in retrieved from database, written to cache, and send back to client.
Write-back (Write-behind) cache - Data is written to the cache without writing to database. Data is written to database asynchronously.

Cache Eviction Policies

FIFO: The cache evicts the fist block accessed without any regard to how often or how many times it was accessed before.
LIFO: The cache evicts the block accessed mostly recently without any regard to how often or how many times it was accessed before.
LRU: Discards the least recently used items first.
MRU: Discards the most recently used items first.
LFU: Counts how often an item is needed. Those that are used least often are discarded first.
Random Replacement: Randomly selects a candidate item and discards it to make space when necessary.

Reference