blog

Squirrel crawling down side of tree.

Ajax Caching, Transports Compatible with jQuery Deferred

by

Ever since the advent of memcached and its ilk, the server-side has been able to benefit from reduced load by caching recently or oft-requested resources. This hasn’t become any less important and valuable. If anything, in this era of the webApp, when native application look and feel is increasingly desired, speedy response to requests is critical for your application to meet the needs of your time-pressed users.

Server-side caching can’t save us from the delay that making a roundtrip from the server imposes, however, and if you’re serving hundreds or thousands of users at the same time, even small memory-cached items may, to the user, seem to be taking their sweet time showing up. For small amounts of oft-requested data that isn’t likely to change often, we can eliminate even these delays by shifting the burden client-side, with local storage and ajax request caching.

So, assuming we’re using jQuery, our ajax requests tend to look something like:

$.ajax({
    data:{data:data},
    dataType:'json',
    type:'POST',
    url:'/endpoint'
}).done(function(result){
    // Success! Do something with it.
}).fail(function(jqXHR){
    // Failure! Do something about it.
});

Now, we could start bracketing all our requests with if statements, to dip into our cache and check before we make our request, but this poses (at least) two problems:

  1. If we’re trying to take advantage of jquery Deferred (and we should be), then it messily complicates our $.when to need to have if statements around our ajax request.
  2. We end up needing two success blocks – one inside the if statement (if the value we’re looking for is in the cache), and one in our ajax.done block.

Instead, we want to extend $.ajax so that if the value we want is in the cache, we grab it from there, and if not, we make our request to the server – either way, we can handle the result without needing to write any extra code. This solution by Paul Irish gets us most of the way – adding some options to ajax that allows us to specify if we want to cache the result, what the cacheKey should be, time-to-live, etc. However, it’s designed to work with jQuery’s now deprecated ‘success’ and ‘failure’ callbacks, instead of taking advantage of the promises wrapped up in jqXHR. Changing that requires us to dip our toes into the deep end, with Ajax Transports.

The jQuery docs say it best, but in summary, an ajax transport provides two functions, send and abort, which are used by $.ajax to actually make the request (or, as the name suggests, abort it). Each request has its own transport, so we register a transport factory when we want to override the usual send and abort – such as when we want to draw from the cache, if available. Here’s an example:

/**
 * This function performs the fetch from cache portion of the functionality needed to cache ajax
 * calls and still fulfill the jqXHR Deferred Promise interface.
 * See also $.ajaxPrefilter
 * @method $.ajaxTransport
 * @params options {Object} Options for the ajax call, modified with ajax standard settings
 */
$.ajaxTransport("json", function(options){
    if (options.localCache)
    {
        var cacheKey = options.cacheKey ||
            options.url.replace(/jQuery.*/, '') + options.type + options.data;
        var value = storage.getItem(cacheKey);
        if (value){
            // In the cache? Get it, parse it to json, call the completeCallback with the fetched value.
            if (options.dataType.indexOf( 'json' ) === 0) value = JSON.parse(value);
            return {
                send: function(headers, completeCallback) {
                    completeCallback(200, 'success', {json:value})
                },
                abort: function() {
                    console.log("Aborted ajax transport for json cache.");
                }
            };
        }
    }
});

The important thing to remember about $.ajaxTransport is that if we return something, that’s considered the transport that should be used. If you return without conditions, that transport will be used for all ajax requests – not generally what we want. Instead, we specify conditions and return something only if the conditions are met, which will cause the standard transport to be overridden with our specified one.

In the above example, you can see we’re specifying ‘json’ before our factory function – this is the first limit on the factory, indicating that we should only even consider overriding the transport for ajax requests with dataType:’json’. Within the factory function, we have further checks against the cache and only if a value is found for the given (or generated, in line with Paul Irish’s plugin) cacheKey do we override the transport to return our value. By leaving out the ‘json’ specifier (or replacing it with, say, ‘script’ or ‘html’) we can change what will be considered a valid target for checking against the cache.

Also note the use of storage – this is just a simple global variable set to work with sessionStorage preferably, localStorage or, as a last-ditch effort, an object:

// We cache to sessionStorage (first choice), localStorage (second choice) or an object (last resort).
var storage = (typeof(sessionStorage) == undefined) ?
    (typeof(localStorage) == undefined) ? {
        getItem: function(key){
            return this.store[key];
        },
        setItem: function(key, value){
            this.store[key] = value;
        },
        removeItem: function(key){
            delete this.store[key];
        },
        clear: function(){
            for (var key in this.store)
            {
                if (this.store.hasOwnProperty(key)) delete this.store[key];
            }
        },
        store:{}
    } : localStorage : sessionStorage;

You can ignore or adapt this to your own purpose, either replacing storage with localStorage explicitly (as per the original plugin) or else subbing with another plugin that offers cross-browser local storage functionality (like one of these).

Finally, here’s a (very lightly) modified version of the original plugin to work with our new transport in providing cacheable ajax queries, with thanks again to Paul Irish.

/**
 * Prefilter for caching ajax calls - adapted from
 * https://github.com/paulirish/jquery-ajax-localstorage-cache, made to work with jqXHR Deferred Promises.
 * See also $.ajaxTransport.
 * New parameters available on the ajax call:
 * localCache   : true,        // required if we want to use the cache functionality
 * cacheTTL     : 1,           // in hours. Optional
 * cacheKey     : 'post',      // optional
 * isCacheValid : function  // optional - return true for valid, false for invalid
 * @method $.ajaxPrefilter
 * @param options {Object} Options for the ajax call, modified with ajax standard settings
 */
$.ajaxPrefilter(function(options){
    if (!storage || !options.localCache) return;
    var hourstl = options.cacheTTL || 5;
    var cacheKey = options.cacheKey ||
        options.url.replace( /jQuery.*/,'' ) + options.type + options.data;
    // isCacheValid is a function to validate cache
    if ( options.isCacheValid && !options.isCacheValid() ){
        storage.removeItem( cacheKey );
    }
    // if there's a TTL that's expired, flush this item
    var ttl = storage.getItem(cacheKey + 'cachettl');
    if ( ttl && ttl < +new Date() ){
        storage.removeItem( cacheKey );
        storage.removeItem( cacheKey + 'cachettl' );
        ttl = 'expired';
    }
    var value = storage.getItem( cacheKey );
    if ( !value ){
        // If it not in the cache, we store the data, add success callback - normal callback will proceed
        if ( options.success ) {
            options.realsuccess = options.success;
        }
        options.success = function( data ) {
            var strdata = data;
            if ( this.dataType.indexOf( 'json' ) === 0 ) strdata = JSON.stringify( data );
            // Save the data to storage catching exceptions (possibly QUOTA_EXCEEDED_ERR)
            try {
                storage.setItem( cacheKey, strdata );
            } catch (e) {
                // Remove any incomplete data that may have been saved before the exception was caught
                storage.removeItem( cacheKey );
                storage.removeItem( cacheKey + 'cachettl' );
                console.log('Cache Error:'+e, cacheKey, strdata );
            }
            if ( options.realsuccess ) options.realsuccess( data );
        };
        // store timestamp
        if ( ! ttl || ttl === 'expired' ) {
            storage.setItem( cacheKey + 'cachettl', +new Date() + 1000 * 60 * 60 * hourstl );
        }
    }
});

For the moment, if there’s a cache error (such as you’ve hit your limit), the new item fails to be stored. More advanced behaviour is left as an exercise to the reader.

+ more

Accurate Timing

Accurate Timing

In many tasks we need to do something at given intervals of time. The most obvious ways may not give you the best results. Time? Meh. The most basic tasks that don't have what you might call CPU-scale time requirements can be handled with the usual language and...

read more