blog

html5 logo

Ajax Upload XHR2, Take 2

by

It’s a pleasure to be able to interact with files in the browser at long last, isn’t it? Reading files in without needing to bounce them against the server first opens up a lot of possibilities – and getting progress from a chunked ajax upload is miles away from the indeterminate form uploads of days past.

Last time we touched this subject, I shared an (admittedly rough) jQuery plugin that allowed you to enjoy HTML5 ajax uploading and file reading with the familiar event interface, and convert any element into a drag-and-drop target.

At the request of reader Mateusz, let’s revisit our HUp plugin, and polish it up a little by adding a new feature – the ability to filter files to be read/uploaded by their file size, and/or their mime-type.

Filtering By Type

This can get tricky, given the wide variety of files and their associated mime types. We want to be able to specify something similar to the accept attribute on file inputs – but a little looser and friendlier, allowing just the extension to be specified, with the plugin handling the hard work of mapping that to a mime-type – at least, when we know the extension.

The logic for this is split up within hup.js. To begin with, we need to create an array of known mime-types,
and a mapping between extensions and their mime-types.

var filters = {},
    fileTypes = [];
    /**
    * Populate the filters and fileTypes object and array, with the former containing a mapping between
    * file extensions and their mime types, and the latter the mimetypes themselves.
    */
(function(mimetypes){
    var mimes = mimetypes.split(/,/),
        exts = [];
    for (var i=0, len=mimes.length; i < len; i+=2)
    {
        fileTypes.push(mimes[i]);
        exts = mimes[i+1].split(/ /);
        for (var j=0, jlen = exts.length; j < jlen; j++)
        {
            filters[exts[j]] = mimes[i];
        }
    }
})(
        "application/msword,doc dot,application/pdf,pdf,application/pgp-signature,pgp,application/postscript," +
        "ps ai eps,application/rtf,rtf,application/vnd.ms-excel,xls xlb,application/vnd.ms-powerpoint," +
        "ppt pps pot,application/zip,zip,application/x-shockwave-flash,swf swfl,application/x-javascript,js," +
        "application/json,json,audio/mpeg,mpga mpega mp2 mp3,audio/x-wav,wav,audio/mp4,m4a,image/bmp,bmp," +
        "image/gif,gif,image/jpeg,jpeg jpg jpe,image/photoshop,psd,image/png,png,image/svg+xml,svg svgz," +
        "image/tiff,tiff tif,text/plain,asc txt text diff log,text/html,htm html xhtml,text/css,css,text/csv," +
        "csv,text/rtf,rtf,video/mpeg,mpeg mpg mpe m2v,video/quicktime,qt mov,video/mp4,mp4,video/x-m4v,m4v," +
        "video/x-flv,flv,video/x-ms-wmv,wmv,video/avi,avi,video/webm,webm,video/3gpp,3gp,video/3gpp2,3g2," +
        "application/octet-stream,exe"
    );

You’re probably wincing a bit at the two var declarations at the top, but remember this is in a closure – those aren’t global, they’re local to the closure. The function that populates the array and object will run once, when the plugin closure itself first executes.

So what’s happening here? We have a mixed comma and space delimited list of known mime-types and extensions, with the extensions always next to their relevant mime-type. We first split the list on commas and push the first element (the mime-type), to our fileTypes array. Then, we split the i+1 element (the file extensions) on space, and create a key for each of the extensions with the value the mime-type.

Why build our array and object from a string like this, instead of just specifying them statically? Well, besides the advantage of being able to build two objects for the price of one (that is, both the array and filter), this allows for easy editing and extension of our admittedly incomplete mime-type and file extension list. Need to add another file extension that should be considered application/octet-stream? Just add a space and the new extension next to exe there. Want to have .md files considered as text/plain? Just drop the extension in next to log.

Next, when the plugin is called on an element with accept set in its options:

/**
* Translate the accept string or array into an array of mime types, based on the mime types in filters.
* Input should look like the expected extensions:
* "swf, wmv, mp4" or ['swf', 'wmv', 'mp4']
* Or like mime type categories, or the mime types themselves:
* "application/*, application/pdf" or ['image/<em>', 'plain/text']
* @param {array|string} accept
*/
Hup.prototype.acceptFilters = function(accept){
    var mimes = [];
    // Ensure accept is an array of extensions or mime types
    if (typeof accept === 'string' || accept instanceof String)
    {
        accept = accept.split(/,/);
    }
    for (var i=0, len = accept.length; i < len; i++)
    {
        var mime = accept[i].trim().split(///);
        if (mime.length > 1)
        {
            if (mime[1] === '</em>')
            {
                // Every mime-type that begins with mime[0] now needs to be pushed into the mimes array
                for (var j=0, jlen = fileTypes.length; j < jlen; j++)
                {
                    var fileType = fileTypes[j].split(///);
                    if (mime[0] === fileType[0]) mimes.push(fileTypes[j]);
                }
            } else {
                // Pass the mime type through unmolested
                mimes.push(mime.join('/'));
            }
        } else {
            // Only an extension has been specified - map to the mime type
            if (mime[0] in filters) mimes.push(filters[mime[0]]);
        }
    }
    return mimes;
};

Here, we allow the plugin user to set accept on the plugin as either a string or an array, and generate our mimes filter array from it. This allows us to be more permissive than the standard accept filter, with: "wmv, audio/, swf" or ['wmv', 'audio/', 'swf'] both being valid. Notice the wildcard in the above example? In acceptFilters, we split on the forward slash and if the second part is a wildcard, we add every mime-type that matches the first part to the mimes filter array. If both parts are specified (ie. ‘plain/text’), we pass it through unchanged (save for trimming any whitespace) – this allows the user to add mime-types that aren’t in our list (ie. video/ogg, audio/flac, etc.). These can be combined – so if you want to allow any audio type, and include ones that aren’t explicit in our list, you can specify ['audio/*', 'audio/flac', 'audio/alac', 'audio/ogg'] to do just that.

Finally, when we’re iterating through the fileList, we make our checks against the mimetype of the file in the fileList and, if its not in our accept filter, we fire a new event of type FILE_TYPE_ERROR and continue on to the next file in the list.

 /**
* Process the files in the fileList, uploading them if a url is specified, otherwise reading them into
* memory and passing them on to be used in the browser.
* @param files
* @param upload
*/
Hup.prototype.processFiles = function(files, upload){
    var that = this,
        processed = 0,
        accept = this.options.accept,
        accepted = false,
        maxSize = this.options.max_file_size,
        fprocess;
for (var i=0, len = files.length; i &lt; len; i++)
{
    // Check file against mime accept restrictions if any restrictions are set
    if (accept.length)
    {
        accepted = false;
        for (var j=0, jlen = accept.length; j &lt; jlen; j++)
        {
            accepted = (files[i].type === accept[j]);
            if (accepted) break;
        }
        if (!accepted)
        {
            this.input.trigger(Hup.state.FILE_TYPE_ERROR,
                {state:Hup.state.FILE_TYPE_ERROR,
                    error:'File type is '+files[i].type+', accepted types are '+
                        accept.join(',')+'.'});
            continue;
        }
    }
    // Check file against size restrictions
    if (maxSize &amp;&amp; files[i].size &gt; maxSize)
    {
        this.input.trigger(Hup.state.FILE_SIZE_ERROR,
            {state:Hup.state.FILE_SIZE_ERROR,
                error:'File size is '+files[i].size+', max file size is '+maxSize+'.'});
        continue;
    }
    // Create new DeferXhr or DeferReader and listen on its progression and completion to fire the appropriate
    // events for interested listeners on our input
    fprocess = (upload) ? new DeferXhr(this.options, files[i]) :
        new DeferReader(this.options.read_method, files[i]);
    fprocess.progress(function(progress){
        that.input.trigger(progress.state, progress);
    }).done(function(res){
        that.input.trigger(res.state, res);
        processed++;
        if (processed &gt;= len){
            that.input.trigger((upload) ? Hup.state.FILE_UPLOAD_ALL : Hup.state.FILE_READ_ALL ,
                {state:(upload) ? Hup.state.FILE_UPLOAD_ALL : Hup.state.FILE_READ_ALL, files:len});
        }
    }).fail(function(res)
    {
        that.input.trigger(res.state, res);
    });
}
};

Filtering By Size

This is absurdly simple by comparison – the code above makes it clear we simply test the file size against the max size that the plugin user has set, and if we’re above that, fire an event of type FILE_SIZE_ERROR with a meaningful error message.

The test against maxSize before testing the fileSize is to allow the default value (0) to mean no maximum file size has been set – relying on js to understand 0 as a ‘falsey’ value. If max size is 0, then we skip the file size test entirely. Similarly, if no mime-types are in the accept array, we skip filtering the files by their type.

Better Docs

The documentation for HUp was a bit rushed. I followed the lead of my (still very alpha) PHP Framework KisKit End Shameless Plug and tidied it up a bit. Take a look for yourself, and feel free to open up an issue if you find bugs / want enhancements – your request might well make for a good future post. šŸ˜‰

+ more

Accurate Timing

Accurate Timing

In many tasks we need to do something at given intervals of time. The most obvious ways may not give you the best results. Time? Meh. The most basic tasks that don't have what you might call CPU-scale time requirements can be handled with the usual language and...

read more