There are so many browsers out there, with Internet Explorer, Firefox, Chrome, Safari and Opera just being the most popular ones. Internet Explorer has been the king of all browsers regarding the market share, but mainly because it was pre-installed on all Windows versions after Windows 95. But over the last months, Internet Explorer’s marketshare has decreased to only slightly above 60% while Firefox almost managed to break the magical 25%-barrier, and even Chrome can claim more than 6% although it was released just one and a half years ago.
Since more and more users seem to consider switching from Internet Explorer to other browsers, they all have the tough choice of finding the right browser. I created this browser roundup to compare the latest versions of all major browsers and show their strengths and weaknesses.
The Contestants
I used the latest available final versions of all these browsers. Beta or Alpha versions are not considered as they may be unstable and thus do not qualify for everyday use. Internet Explorer was running in 64-bit, all other browsers in 32-bit.
- Opera 10.51 (10.51 Build 3315)
- Chrome 4.1 (4.1.249.1036)
- Firefox 3.6 (3.6.3)
- Safari 4.0 (4.0.5 531.22.7)
- Internet Explorer 8 (8.0.7600.16385 64bit)
The Benchmark System
This is the system used for benchmarking all these different browsers. All other tasks were terminated during the benchmarks except for Avira AntiVir.
- Processor: AMD Phenom II X4 955 (3.2GHz)
- Mainboard: Gigabyte GA-MA790XT-UD4P
- Memory: 2×2GB A-Data DDR3-1333 (CL 7-7-7-24 2T)
- Graphics: Gainward Nvidia GTX 260 (680/1400/1150MHz)
- Hard Disk: Samsung Spinpoint F3 1000GB (7200RPM, 32MB cache)
- Operating System: Windows 7 Ultimate 64-bit
The Features Tested
- Performance – JavaScript and Rendering tested with Sunspider, Peacekeeper, Dromaeo, V8
- Standards Conformity – Compliance to HTML, CSS and JavaScript standards tested with Acid3, CSS3 Selectors Test
- Loading Speed – Time to load websites tested with Numion Stopwatch on three websites
- Startup Speed – Time it takes the browser to launch
- Memory Usage – The amount of memory used with 1, 11 and 25 tabs
- Customization – The amount of addons and themes available
- Security – Vulnerability to different hacks, block rate of malicious websites
Testing the security was a really hard job. The only reliable data was the block rate. All browsers passed the hacking tests and the amount of vulnerabilities over the last years does not allow to judge the security. Data for the mean time of patch release was not available.
The Results – Overview
I analyzed and created charts for every single test, but these detailed statistics won’t be interesting for most people, so here are the overall results. A German version of this summary can be found on PCMasters.de.
The results of all tests have been summed up and weighted in order to create these final results. Further details on the weighting can be found below. All results based on time (Loading Speed and Startup Speed) and the memory usage are inverted so that higher scores are always faster. Security is left out due to the reasons mentioned above. Please remember: All tests were executed on the system named above. Results on other systems may vary. Numbers (e.g. addons, themes, vulnerabilities) were current at the time of creation of this benchmark (14th of April 2010) and may increase in the future).
The Results – Analysis
Performance: This measurement of performance indicates how fast websites can be drawn, scrolled or zoomed after all the content have been loaded, and how fast complex applications such as Google Docs are running. Opera and Chrome are clearly leading regarding the JavaScript and rendering performance, closely followed by Safari. The popular Firefox browser is quite slow compared to the ones named before, and Internet Explorer is really much slower than the others. This aspect is especially important for slow computers like netbooks.
Standards Conformity: There are some standards like HTML and CSS that define how the website is supposed to look like. Web developers should write their HTML-Code according to the standards so the website can be displayed correctly. But when the browser used by a visitor does not comply to these standards, the website may not be displayed correctly or some buttons may not work at all. This really slows down web development because developers have to create different versions of their scripts and have to test all browsers which is quite costly. Low standards conformity in Internet Explorer 6 caused Google to dis-continue support for this browser in order to force users to migrate to better ones. All browsers except Internet Explorer 8 comply quite well to the web standards, just Firefox takes a little hit by scoring just 95% on Acid3. Users should really choose a browser that scores high in this section in order to be able to view and use every website perfectly.
Loading Speed: This comparison shows how long it takes browsers to load websites using the same internet connections. As you can see there isn’t much difference between all browsers, Internet Explorer is slightly faster than the rest, but only by a few percent. Opera and Chrome could perform better using their features Opera Turbo or Chrome DNS Prefetching, but these features were disabled during the test.
Startup Speed: Determines how long it takes the browser to start up and load the front page (google.com in this case). Internet Explorer 8 is the fastest one here, but regarding the absolute values this doesn’t count much: Internet Explorer needs 0.8 seconds while Firefox, the slowest starting browser takes 2.4 seconds. Assuming the average user would start his browser 10 times a day, using Internet Explorer over Firefox would only save 16 seconds.
Memory Usage: Loading a website and rendering its contents always needs some of your system memory (RAM). But different browsers use their memory more or less efficiently, as the chart above is showing. Firefox and Safari were most efficient, Internet Explorer least efficient. Considering the fact that Internet Explorer used 521MB with 25 tabs open and 291MB with 11 tabs open, most users with modern computers (having 2-4GB of memory) won’t notice this much. Older and slower computers or netbooks mostly have limited memory (512-1024MB) and thus are more dependent on memory-efficient browsers like Firefox.
Customization: This includes the amount of addons and themes available for each browser. Many users don’t like the default interface of their browser and want to change it, or just want to add more features such as ad-blocking or notificiations (Facebook, Mail, etc) to their browser. This can be provided by using addons. The more addons and themes are available, the higher is the probability of finding just the ones you need. Firefox is most customizable with more than 12000 addons and 400 themes. Chrome and Opera with 4000/1400 addons and 400/120 themes can be adjusted as well, but Internet Explorer and Safari don’t offer many options here.
There is no way of summing up all these points to come to the conclusion “this is the best browser, this is the worst one”, because every user considers the aspects differently. Thus I will try to give some recommendations who should use which browser. I won’t recommend Internet Explorer anywhere due to its bad implementation of web standards and the shortcoming regarding performance.
The Results – Recommendations
Netbook-User: Netbooks have quite slow processors and thus need a browser with high performance, given by Opera and Chrome. But most Netbooks also have very limited memory, thus the browser should be quite memory-efficient like Firefox and Safari. Firefox is performing quite bad in all the performance tests, so Opera, Chrome or Safari would be the best choice for Netbook users.
Poweruser: If you surf the web every day for a long time using many different tabs you really need performance and memory efficiency. Again, Opera, Chrome and Safari offer these features in combination with perfect implementation of web standards. Safari’s loading speed is slightly better than Opera’s and Chrome’s, but Opera and Chrome have their Turbo and DNS Prefetching technologies respectively to boost page loading. It’s a difficult choice.
Customizer: If you want your browser to look exactly the way you need it, every button to be at the right place, and every feature to be directly available inside the browser, Firefox would be the choice. This browser offers more than 12000 addons and 400 themes for users for whom performance doesn’t matter that much.
Average User: The average user surfs the web only occasionally to check mails, research something or watch some video on YouTube. He doesn’t need any fancy features or designs and just wants to enjoy his web experience. Here almost every browser would fit, but most of them have a lot of features nobody actually needs. Chrome offers good performance, a simple interface with just the buttons required to do normal surfing and comfortable features such as an integrated history search inside the URL box.
The Results – Detailed Statistics and Analysis
The following part will show detailed statistics of the different tests and explain some results. All raw-data is stored in an Excel file you can download here, containing the exact scores and timings, URLs of tests and websites used and comments regarding errors or settings.
Performance:
Opera is leading regarding the performance, directly followed by Chrome and Safari. Firefox isn’t performing that well, but Internet Explorer clearly looses this battle. Its performance is far below the other browser’s, and the Internet Explorer 8 wouldn’t even start the Dromaeo test due to its bad implementation of web standards. Sunspider results are inverted to show runs per minute. On the testing system Opera was fastest with 6174 points in Peacekeeper and 434.52 in Dromaeo. Safari leads with an average of 595.4ms in Sunspider, and Chrome with 4397 points in V8.
Standards Conformity:
In order to check whether a browser complies to web standards, the Acid3 and CSS3 Selectors Test were used. Opera, Chrome and Safari pass Acid3 with 100%, Firefox scores 92% and Internet Explorer only 20%. All browsers except Internet Explorer passed the CSS3 test with 100%, Internet Explorer just passed 349 of 578 tests.
Loading Speed:
In order to measure how fast a browser can load different websites, the numion’s StopWatch was used. Loading speed was tested on three different websites and recorded as the average of two tests. Loading times for ESPN range from 2.2 to 3.5 seconds, from 0.5 to 0.9 seconds for Google and from 2.1 to 4.5 for this website. As you can see, the results are not really homogeneous, so one can’t conclude that one browser is loading faster than the other one.
Startup Speed:
When looking at the startup speed (starting the browser and loading the homepage google.com), Internet Explorer is clearly the winner and Firefox clearly the looser. But when considering the time it actually takes the browser to launch, from 0.8 seconds for Internet Explorer and 2.4 seconds for Firefox, this doesn’t really affect the browsing experience and can be neglected.
Memory Usage:
In order to measure the real memory usage (which can be quite difficult for browsers like Chrome who run multiple processes) the total memory usage of the whole system was measured before browser launch, after 1 tab, 11 tabs and 25 tabs. The memory usage results are inverted to show lower usage as higher score. Running one tab with just google.com in it, all browsers except Safari required between 55MB and 61MB, while Safari took only 32MB. Running more tabs, Firefox clearly leaves the concurrents behind with just 165MB for 11 tabs (others between 219MB and and 291MB) and 271MB for 25 tabs (others between 372MB and 521MB). For calculating the total score the memory usage for 25 pages was weighted only 50% because most users never open that many tabs. You can find the list of pages opened in all browsers here.
Customization:
To compare the ways of customizing the different browsers, the amount of addons and themes was assessed. Firefox has more than 12000 addons and, followed by Chrome with only above 4000. Similarly, Firefox has most themes with more than 400, closely followed by Opera. Internet Explorer and Safari didn’t have many addons and no official way of customizing the interface or theme. There were some third-party applications to chance their looks but they can’t be considered as official themes.
Security:
Comparing the security of different web browsers was a really hard job. The company scanit released a test that tries to crash browsers using various security leaks, but every browser managed to resist all of them. Then the block rate of malicious websites was considered: Internet Explorer blocks 85% of all malicious links tested, followed by Firefox and Safari with 29%, Chrome with 17% and Opera with only 1%. Just blocking links doesn’t make the browser safe from hacker attacks, so this does not really show the security of a browser. Then the public security vulnerabilities (leaks that are known to the browser developers and hackers that were not closed by a patch before being known publicly) were counted (inverted in the diagram). There was no data available for Chrome, and since Safari only had 4 of these vulnerabilities it is clearly leading here. The last diagram shows the amount of vulnerabilities found for each browser in the NVD. But since some browser exist for a very long time (e.g. Firefox and Internet Explorer) and some just for a few years (Chrome), I can’t really compare the amount of vulnerabilities ever found. This is why there is no total security score in the summarized diagram.
I hope this browser roundup gives an overview over the features and possibilities offered by each browser. If there are any suggestions, please comment! Screenshots to prove all results are available, but too big to be uploaded here. If you want me to prove any of the results, just comment or write an email and I will send the screenshots.
