Apache Benchmark, Wordpress framework, time = ?

After my article on Wordpress framework I gave some of them a try and developed sites with Hybrid, Thematic and Sandbox.

I am a bit funny when it comes to performance and I was bothered about all those functions I was not using and all those files php had to include compared to the default theme. So I did a quick benchmarking of all those framework using apache built in benchmark tool.

A quick intro on the ab command

Apache benchmark tool is a built in tool you can use on any apache install. I will assume here that you have apache installed on your computer and know the basics in terminal command on a windows environment. So open a terminal (start>run>cmd) and go to your Apache folder (use ‘cd nameoffolder’ to go in a folder). Once in the apache folder go into the ‘bin’ folder. From here you can run the ab command and start benchmarking.

Ab allow you to simulate as many queries as you want and even simulate multi users accessing the same site. For my test I wanted to run 100 time the front page of a blog with 10 simultaneous users. Here is how it looks in the terminal:

ab -n 100 -c 10 http://localhost/wp-site-framework/

Don’t forget the / at the end, otherwise ab will run the test on accessing the folder itself instead of running the index.php. And that’s an example of what I get for the valid-extend theme running as a child theme of Hybrid:

C:\Program Files\Apache Software Foundation\Apache2.2\bin>ab -n 100 -c 10 http:/
/localhost/yoursiteisvalid-live/validnews/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done


Server Software:        Apache/2.2.9
Server Hostname:        localhost
Server Port:            80

Document Path:          /yoursiteisvalid-live/validnews/
Document Length:        4084 bytes

Concurrency Level:      10
Time taken for tests:   15.225 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      435800 bytes
HTML transferred:       408400 bytes
Requests per second:    6.57 [#/sec] (mean)
Time per request:       1522.541 [ms] (mean)
Time per request:       152.254 [ms] (mean, across all concurrent requests)
Transfer rate:          27.95 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.6      0      16
Processing:   841 1470 364.3   1417    2584
Waiting:      841 1470 364.6   1417    2584
Total:        841 1470 364.3   1417    2584

Percentage of the requests served within a certain time (ms)
  50%   1417
  66%   1557
  75%   1666
  80%   1728
  90%   2086
  95%   2211
  98%   2366
  99%   2584
 100%   2584 (longest request)

So I’ve run this test against 9 themes and framworks and here are the results:

• Default: 1567.620
• Sandbox: 1275.655
• Whiteboard: 1543.202
• Carrington: 1508.017
• Carrington JAM: 1535.063
• Thematic: 1542.326
• Hybrid: 1380
• Valid-extend (child theme of Hybrid): 1522.541

Conclusion

The results are in milliseconds per completed request. As you can see there isn’t a massive difference between all the framework and I think the difference between Sandbox is more due to the lack of images. I believe this could be different run with 100 concurrent users like it can be on busy blogs so if you fancy running the test yourself send us the link in the comments

• 
• 
• 
• 
• 
•