Contributed by jason on from the benchmarks-are-meaningless-but-we-cant-help-ourselves dept.
NetBSD developer Jaime Fournier has published a new round of Ruby benchmarks comparing the various BSD projects and Ubuntu Linux. Some readers will remember he performed a similar test a couple of years ago. A quick review of the data reveals improvements by OpenBSD. The following is a snippet from the full report available on his website.
Benchmarks from ruby-benchmark-suite run on ruby-1.9.2-p180 with rubygems-1.5.3 Benchmarks found at https://github.com/acangiano/ruby-benchmark-suite Dragonfly 2.9 - With Hammer root filesystem FreeBSD 8.2 - Default disk layout NetBSD 5.1 - Default disk layout OpenBSD 4.8 - Default disk layout Linux - Ubuntu 10.10 Server EditionThe hardware which is an intel i7 with a dedicated 40GB SSD used for each os with default installs. FreeBSD dmesg here http://linbsd.org/fbsd.dmesg OpenBSD dmesg here http://linbsd.org/obsd.dmesg These are the mean values taken from each benchmark. The name format is benchmark_parameters_iterations Charts might come as more OS's are added. Warning: No attempt to ensure the same compiler was used on each os as this would be unrealistic given these systems come with compilers and typically would be used to build ruby. Email any questions or rants to jaimef@linbsd.org Totals Wins by OS 190 NetBSD 5.1 100 Dragonfly 2.9 88 Linux - Ubuntu 10.10 65 OpenBSD 4.8 46 FreeBSD 8.2
Editor's Note: Some readers may point out that OpenBSD performance is near the bottom of the list. I would remind those users that if web application performance is your key requirement, then Ruby probably shouldn't be your language of choice anyways. :)
(Comments are closed)
By Tom Van Looy (tvlooy) tom@ctors.net on twitter.com/tvlooy
"Lower is better! Smaller number wins..." so that's a good thing not?
Comments
By Tom Van Looy (tvlooy) on twitter.com/tvlooy
> "Lower is better! Smaller number wins..." so that's a good thing not?
Shit, I get it now.
By David Trotter (fossala) on
> "Lower is better! Smaller number wins..." so that's a good thing not?
The lower the number the better applies to the tests. But the numbers where OpenBSD is second to last is the results (who won how many times) thus higher the better.
By Dave Steinberg (redterror) on
It seems like he punted on the hard part here.
Comments
By John L (JohnL) j@bitminer.ca on bitminer.ca
The missing measurements (where the platform does not complete the test) would be more concerning.
Picking a few "web-like" tests at random, I see OpenBSD within 1 standard deviation most of the time and slow only on a file IO and much better on a threads test. A true believer in benchmarks would select only these micro and macro tests of concern and evaluate those.
Too bad there isn't a numerical security test that lets OpenBSD win based on, say, number of remotely exploitable holes in the default install.
Comments
By Jaime Fournier (jaimef) on http://mauthesis.com
>
> The missing measurements (where the platform does not complete the test) would be more concerning.
>
> Picking a few "web-like" tests at random, I see OpenBSD within 1 standard deviation most of the time and slow only on a file IO and much better on a threads test. A true believer in benchmarks would select only these micro and macro tests of concern and evaluate those.
>
> Too bad there isn't a numerical security test that lets OpenBSD win based on, say, number of remotely exploitable holes in the default install.
>
>
There are many problems with these microbenchmarks.
These initial results are for baseline purposes as these tests
are used quite a bit within the ruby community.
The next tests are more web specific with RubyOnRails and a full
stack including database.