More is Less

2011-04-29T00:00:00Z

When adding JavaScript to a small site or blog, most web developers will instinctively reach for jQuery, however, most of the time the issues they are trying to resolve could had easily been fixed, in far less than the 214KB jQuery adds to their uncompressed page weight.

DOM

A large amount of the CSS selectors used in jQuery scripts consist of a single id, tag, or class name, and this is just as easily accomplished with native API’s.

Todays browsers have near perfect support for getElementById, getElementByTagName, and getElementByClassName, with the latter being unsupported but easily emulated in older versions of Internet Explorer.

function getElementsByClassName(el, className) {
    if (el.getElementsByClassName) {
        return el.getElementsByClassName(className);
    }
    else {
        var elements = el.all ? el.all : el.getElementsByTagName("*"),
            regex = new RegExp('(^|\\s)' + className + '(\\s|$)'),
            result = [];
        for (var i = 0, l = elements.length; i < l; i++) {
            if (regex.test(element[i].className)) {
                result.push(element[i]);
            }
        }
        return result;
    }
}

Admittedly this does not look as aesthetically pleasing as the jQuery DSL but it is no any less readable.

// before
$("#status").html(status);
// after
document.getElementById("status").innerHTML = status;

Events

Once we have our DOM elements the next step is generally to bind events to them, this is not consistent across browsers, but fortunately the problem was solved a long time ago, and a well established polyfill can be found in the addEvent method graciously hosted by the QuirksMode Blog¹.

// before
$(buttonElement).click(function(e) {
    alert($(this).html());
    return false;
});
// after
addEvent(buttonElement, "click", function(e) {
    alert(this.innerHTML);
    return false;
});

When it comes to events, it turns out the jQuery DSL does not really add much on top of the existing solutions.

AJAX

Normalizing the XMLHttpRequest object across browsers is also a fairly easy job, and there are countless blog posts dedicated to the subject, including a simple solution by PPK².

// before
$.get("/ajax.php", function(data) {
    var el = $("#content");
    el.html(data);
});
// after
sendRequest('/ajax.php', function (req) {
    var el = document.getElementById("content");
    el.innerHTML = req.responseText;
});

Again the jQuery DSL adds little to the existing solutions, with the existing solutions adding very little to the page weight.

Effects

Depending on the target audience and the relevance of the effects, one could argue that they are better implemented as CSS3 transitions and transformations, leaving the slower browsers such as IE with a fast but less rich experience, but if you must have animations in IE, then this is one area where doing it yourself is probably a bad idea, and is best left to an effects library.

Conclusions

Before reaching for a library, we should fully evaluate the problem and ask ourselves whether the said library will justify its self? Are there more appropriate choices? And do we need a library at all?

No More Clever JavaScript

2011-01-01T00:00:00Z

There is much clever JavaScript out there now days, and maybe it’s the python developer in me, but I don’t like it one bit. I truly believe that we should strive for simplicity, elegance, and above all readability in our code.

Check your Ego at the Door

As developers we all have egos to satisfy and it can be hard not to show off, but it seems we often forget that it is harder to write simple code than it is to try to be clever. We also seem to forget that code needs to be maintained, and just because something makes sense to us now, does not mean it will do so in the future, or to others.

If popular knowledge is to be believed, then debugging code is twice as hard as writing it, should this ring true, what chance do our more junior colleagues have when they are tasked with its future maintenance?

Most of the cleverness seems to stem from efforts to either reduce lines of code, or to make small micro optimisations, neither of these provide enough of a benefit to warrant a decrease in readability. Minification yields a far more significant decrease in size, and the speed of modern JavaScript interpreters — as well as the hardware they run on — is increasing at a rate far surpassing any improvements in our ability to debug code.

Do we want our legacy to be one of torturous debugging or a joyful experience extending that which we leave behind? Which would you rather find at your next project?

The Plea

JavaScript is finally starting to gain recognition, moving beyond the browser, into everything from game engines to web servers, now is our chance to define ourselves, lets try not to screw it up.

So, can we quit with the clever stuff now, and instead focus on creating simple, elegant, and above all readable code?

An Event Driven Frontend

2010-12-15T00:00:00Z

After spending the last year with node.js I have come to appreciate JavaScript’s strength as an event driven language, and yet in the browser, beyond binding callbacks to DOM events and AJAX requests, the majority of front-end code seems to be developed in a mostly procedural way.

Recently when faced with taking on a codebase that had grown unmanageable due to the procedural model that had been employed thus far, I began by refactoring the code around a central event dispatcher, allowing many of the large methods — that had previously been created — to be broken down into smaller more loosely coupled pieces.

Benefits

By introducing an asynchronous model to the code, we were able to break much of it into smaller more maintainable pieces, including a clean break between presentation — such as animations or manipulating the DOM — and the business logic, involving AJAX requests and API calls.

A more loosely coupled system also helped to simplify the development, allowing smaller related sections to be developed independently and by different members of the team.

Solutions

YUI3 has custom event handling baked in¹, and I highly suggest YUI as a framework for any large scale javascript project.

The same technique could be created using jQuery custom events on the document element, although it would be advisable to use one of the dedicated plugins available such as jQuery pubsub by Peter Higgins².

There are also library agnostic implementations such as the brilliant PubSubJS³ by Morgan Roderick — whom I was fortunate enough to speak with at the fullfrontal⁴ pre conference drinks earlier this month — and after reading through the code on github I can highly recommend his library.

Conclusion

The exact implementation of this concept does not matter — and will vary massively depending on your choices of framework and libraries — but it should be seriously considered for any medium to large scale javascript project.

The Unscalable Side of Redis

2010-06-02T00:00:00Z

There is much buzz about redis lately, and most of it with just cause, redis is a blazing fast single threaded data store, that is simple to shard when dealing with large datasets, and can be easily replicated when reads are a problem.

The Problem

Much of the more advanced code I’ve seen, including some of my own makes good use of the redis set data-type and the intersection and union operators to facilitate filtering.

Recently I had the chance to experiment with trying to port a huge dataset to redis, which resulted in a possible edge case where redis would have to calculate the intersection of three ridiculously large sets (700K items), this caused a freeze of about four seconds and during that time no other operations were possible, in a production environment, locking of this nature could cause catastrophic results and has pushed back any plans of an all redis backend.

This is a problem that cant really be solved by the existing scaling mechanisms redis provides, and it is problem that could affect any code that relies on automatically adding values to a set, especially many of the auto completion and search solutions I’ve seen presented.

Possible Solutions

Recently I was fortunate enough to be able to speak to Rob and Adam from playnice.ly at the recent Redis London meetup, and thrash out a few ideas on the problem.

The first solution would be to eliminate the problem sets, or to prevent their use in intersections.
Another option, although not always possible, would be to pre-calculate the results on write.
And finally there is the option of breaking the set into smaller pieces and calculating the intersection of each piece.

What Next?

The solutions above are a good starting point, but far from fully formed, and this is an area which deserves far more attention, I’m hoping that by putting the word out there, that the redis community can start to move forward towards concrete solutions and that the code we are producing doesn’t come back to haunt us in the future.

Work Hard, Work Later

2010-05-26T00:00:00Z

In any complex web application there is a need to perform operations on user request, these operations can become long running, especially when the server is under load. In order to mitigate this, a web application should leave most of the work for background processes.

Benefits

Long running actions make the interface feel sluggish, and are subject to termination by the user, and as mentioned in a previous article, a slow website can harm conversion and lead to unhappy users.

Long running processes can also mount up and when under heavy load, can prevent the webserver from dealing with new requests.

Process

When a request is received, the application should do the minimum work required for it to maintain the user interface, and should instead add any remaining work to a job/task queue.

As far as job queues go, you probably cant go wrong with Gearman¹, although there are many others, such as the ruby focused Resque², as well as the python focused Celery³ and Beanstalk⁴, which should be considered to see which suits you best.

The exact technology does not matter so much as the process, taking the example of a blog comment, the backend should validate the form submission and report ack to the front-end, if the submitted data is valid, a job should be added to the background job queue to process the comment, performing the database insert, Askimit or Typekit validation, and sending any notification emails.

In our comment example, a sudden spike in the rate of comment submissions will result in a larger queue and a delay in the comments arriving in the moderation list, rather than causing excess load on the database, and the user gets immediate feedback without having to wait for the various processes involved to complete.

Conclusion

A job queue is a great tool in the quest for responsive web applications, allowing us to perform complex processing in response to user action actions, without causing the interface to become unresponsive, and in the mean time helping to deal with any unforeseeable spikes in traffic.

Grasping the Future, Today

2010-05-19T00:00:00Z

HTML5 and CSS3 have arrived, and the general consensus seems to be that we should all go out and start using the bountiful wonders it brings.

Modernizr allows us to make use of these new features by providing a consistent API, querying the support for various HTML5 and CSS3, and enabling us to provide fallback’s for older user agents.

CSS3

By adding classes to the root element, Modernizr allows us to build conditional branches in css, by prefixing a selector with the feature we wish to detect.

ul.menu {
    -moz-column-count: 2;
    -moz-column-gap: 20px;
    -webkit-column-count: 2;
    -webkit-column-gap: 20px;
    column-count: 2;
    column-gap: 20px;
}

.no-csscolumns ul.menu li {
    float: left;
    margin-right: 20px;
}

.no-csscolumns ul.menu {
    margin-right: -20px;
    overflow: hidden;
    zoom: 1;
}

The new multi column layout features of CSS3 allow us to create a vertically orientated multi column list without the needing to split the list or change the order of the items within, and Modernizr can help us fall back to a traditional horizontal floated list in unsupported browsers.

HTML5 API‘s

The other side of modernizr is a javascript API, allowing for consistent and standard approach to feature detection in javascript conditionals.

if (Modernizr.geolocation) {
    navigator.geolocation.getCurrentPosition(function(position) {
        var coords = position.coords;
        alert('Location: ' + coords.latitude + ', ' + coords.longitude);
    });
}

The GeoLocation API allows a web browser to determine the longitude and latitude of the device running the browser, but is not widely supported.

How Fast is Fast Enough?

2010-05-10T00:00:00Z

With the recent changes in googles page rank algorithm; the subject of page optimization is back on everyones mind, and people are once again looking to cut precious milliseconds from their loading times.

There are also many other reasons to optimise your page speed, for-instance, case studies have shown a correlation between loading time and conversion rate¹, and everyone loves higher conversion rates.

Instead of the standard approach, where developers delve into the code hunting or sore spots and optimising until things appear fast, I lean towards a system of targets, where predefined quotas are assigned to execution time and page weight, and both designers and developers make achieving them a part of their standard workflow.

Why Targets

Targets give us something to work towards, they tell us when we have done enough, and prevent our expectation from drifting upwards over time, and with the entire team working towards the same goal, the optimization effort becomes easier to manage.

The problem is that testing page loading times after every change is time consuming, but by fixing the values, we can then write unit tests to automate the process, allowing the work to be shifted to a continuos integration server.

Clear targets can also be used as part of the decision making process, if a feature will push the loading time beyond the targets, is the feature likely to generate more conversion than will be lost due to the longer rendering time?

And finally there is the impact on mobile broadband users to consider, where connections are generally slower and bandwidth far more costly.

What Targets

As one second is the accepted time we can spend doing things before the user becomes susceptible to distraction², we should aim to have the page completely rendered by then, this means page execution time, downloading its content, any external assets, rendering, and initial javascript execution.

Ideally, execution time should feel instantaneous to the user, this gives us a window of 100ms[2] and should be our target for getting things done, with a maximum of 250 ms if we are to hit our one second overall target.

When it comes to page weight things start to get a little more complicated, according to OFCom the average connection in the UK is around 4Mbps³. Even if we dedicate half our remaining time, for the user to download everything in 500 ms, we need to aim for a page weight of around 200KB, and should generally strive for about half that.

In total we have now used up to 75% of our one second target, this gives us another 250 ms for the browser to execute javascript, and finish rendering the page.

Conclusions

So to summarize, by adopting a targets orientated approach, we should be able to keep on top of our page execution times and weight, providing a faster and more fluid user experience, resulting in happier users, higher conversion, reduced bandwidth bills, and partially future proof our sites towards the growing number of 3G connections.

Summary

Criteria	Target
Execution Time	100 ms
Page Weight	100KB
Maximum Execution Time	250 ms
Maximum Page Weight	200KB

The Little Database that Could

2010-05-05T00:00:00Z

When redis first crossed my radar last year, I had some serious doubts about the numbers people were quoting, doubts my own testing¹ inevitably prove to be completely unfounded.

Performance, though impressive, is but a small part of what makes this little database so amazing. Despite being clasified as a key-value store, what really sets redis apart is its rich set of datatypes, including not only strings, but lists, sets, and hashes as well.

Try Redis

Installing redis is ridiculous simple, and you can get it up and running in a matter of seconds. In a new terminal try:

$ wget http://redis.googlecode.com/files/redis-1.2.6.tar.gz
$ tar -xzf redis-1.2.6.tar.gz
$ cd redis-1.2.6
$ make
$ ./redis-server

One of the simplest ways of interacting with redis is via the interactive console, in a new terminal you can launch the cli client with the following command:

$ cd redis-1.2.6
$ ./redis-cli

And you can now interact with redis via its rich yet simple set of commands

redis> KEYS *

redis> SET foo bar
OK
redis> KEYS *
foo
redis> GET foo
bar
redis> SADD breakfast bacon
(integer) 1
redis> SADD breakfast eggs
(integer) 1
redis> SADD breakfast fruit
(integer) 1
redis> SMEMBERS breakfast
1. bacon
2. fruit
3. eggs
redis> SREM breakfast fruit
(integer) 1
redis> SMEMBERS breakfast
1. bacon
2. eggs
redis>

Usage

March brought with it my first opportunity to deploy redis into my works production environment, and I have since been de-normalizing much of the sore spots in our existing relational database into a cluster of redis servers.

So far redis has proven its self to be an ideal storage solution for:

High request volume environments such as comet based message queues.
A denormalized push cache.
URL routing information.
An intermediate store for analyical data.

Using redis has not only resulted performance gains, but in places its simplistic model has also lead to more readable code.

130,000 reads and 80,000 writes per second on my machine ↩

Hello World

2010-05-01T00:00:00Z

After some long term procrastinating over what to do with my personal website, I have finally hammered out some code and a design I’m satisfied with.

Now i find myself with the proverbial blank slate and a no idea where to be begin, but I will endeavour to fill the pages with nuggets of mac, linux, and web development goodness.