lies, damned lies and statistics

Framework Performance

I’m always a bit skeptical about performance numbers when I see them, here’s yet another reason to doubt the numbers when you see someone run performance numbers.

The basic apache configuration doesn’t make any sense. Take a look:

StartServers         5
MinSpareServers      5
MaxSpareServers      10
ServerLimit          150
MaxClients           150
MaxRequestsPerChild  10000

The mongrel configuration for Rails is using 10 processes.  Apache has between 5 and 150 processes.

Django is configured to use mod_python to embed a python process per apache worker.

Disregarding the fact that there is absolutely no source code available for any of the test runs - which makes me even more skeptical - this Apache configuration is naive.  Of course Rails is running slower than Django - the Apache processes are starving waiting for Mongrel to serve data up.

If anything - I’m impressed that Rails has performance numbers that are impressive.  I’m even more doubtful about the PHP performance numbers.  Yahoo uses PHP, digg.com runs PHP.  Those sites seem to scale just fine.  If something is screwy in PHP’s performance - it could be a bad php.ini configuration, bad code in symfony - hell who knows?  There’s no code to inspect!

C’mon folks.  I do Python day in and day out, I love Django, but having bad benchmarks doesn’t help anybody.