Browser tests: a look at standards

If speed plays an important role in the every-day browsing experience, the same can be said about compliance with web standards. With all browsers and web designers using a common language, users would have a more consistent, browser-independent experience and designers would spend less time working around browser-specific bugs and incompatibilities. A very desirable goal, but “are we there yet”?

The simplest answer is ‘No’. But as with speed, some browsers are closer to the objective than others. Lately, a number of new online tests were launched to check different aspects of rendering. In a way, these are more straightforward than speed tests, because the results are not influenced by the particular hardware where the applications are installed. I ran the same set of browsers from my previous speed test through them, so let’s see what we can find out:

• Acid3 is the oldest of them, being launched more than 2 years ago. Some browser makers dismissed it, not entirely without reason, while others jumped to the opportunity to show the world a perfect score. This situation is reflected until the current day: both Google Chrome and Opera score a perfect 100 with Firefox coming in a close second. The Internet Explorer Platform Preview also shows big improvements, with a score of 68, up from the meager 20 points of version 8.
• The CSS3 Selectors Test checks compatibility with more than 500 CSS selectors. Here most browsers pass with a perfect score, with the exception of Internet Explorer 8 that only supports about 60% of the tested selectors. Firefox 3.7 also misses 2 tests, probably a small regression that will be soon fixed.
• Sputnik is a JavaScript conformance test suite recently released by Google. It’s also the most extensive, containing more than 5000 tests. Naturally it takes a long time to complete and most browsers (except for Opera) slow down to a halt when they reach 60%, probably indicating a particularly complicated test or a common flaw. The results are nearly identical to the presentation on the official page; Internet Explorer 9 has achieved notable improvements in this area as well, coming very close to Firefox.
• Lastly, a new test checks how much of the new HTML5 features are implemented in browsers. It was developed by the same graphic designer who contributed the CSS3 test mentioned above. Right now it runs 160 tests, but a beta version with a maximum possible score of 300 is under way. Here, Chrome is in the lead with Firefox and Opera on the second place with similar scores. It’s a little surprising to see that Internet Explorer hasn’t improved the HTML5 features at all in the preview compared to the current release, they both score only 19 points!

Browser	Acid3 (max 100 points)	JavaScript conformance (number of failed tests out of 5246 total)	HTML5 (160 max points)	CSS3 (supported selectors of 578 max)
Chrome 4.1	100	224	118	578
Firefox 3.6	94	266	101	578
Firefox 3.7a5	97	267	104	576
Internet Explorer 8	20	467	19	349
Internet Explorer 9 dev	68	277	19	578
Opera 10.53	100	81	102	578

Of course, no test can magically show which of the browsers has the best standard support, as each of them checks different aspects which are equally important and only a portion of the possible real-life cases. Not to mention a lot of these standards aren’t even finalized. Nevertheless, they can give some image of the current situation. From these results, it’s safe to say Opera and Chrome have the best overall score and Firefox is not very far behind. As usual, Internet Explorer has it’s own idea of standards. While in terms of speed the preview is already on par with Firefox, it still has a lot to go until we can say the same about support for web standards.

I also made a chart to better visualize the numbers in the table above. Since each test has a maximum score, I divided each result by this theoretical maximum to obtain numbers between 0 and 1 that can be compared more easily. I enhanced the differences on the axis for JavaScript, because otherwise they would not be easily visible. A “perfect” browser would fill the entire square. You can see how Internet Explorer covers only a small area at the centre of the image, having the smallest score in each of the four tests.