Dude, please try to be accurate.

People who skew the truth and deliberately mislead people make me angry. This post is a direct result of that phenomenon. If I sound irritated, well, it's only because I am. People who make a deliberate attempt to denigrate any platform based on artificial or manipulated results need to be called out on it. Letting a lie grow is, in my opinion, the same as telling it yourself...

In a recent post on his blog, Mr. Phil Parsons, self-professed CF "disliker" ("hater" seems a bit strong lol) and PHP aficionado, released several bits of code that claimed to demonstrate ColdFusion's performance inferiority to PHP. So I decided to see what was up with that myself. I wanted to see if it was true. I would have been happy to report that, yes, PHP is in fact faster than ColdFusion, here's my results. However the more I experimented and optimized both the PHP version and the ColdFusion version, the angrier I got about Mr. Parsons' blatant misrepresentation of the facts.

(NOTE: I didn't want to disprove him, I wanted to find out if it was true. If it's true, so what? This was a thought exercise, not an attempt to redeem ColdFusion (which really doesn't need redeeming anyway!))

Much thanks to Elliot Sprehn for helping me get a decently optimized prime number scanner that's practically bit-for-bit identical between PHP and ColdFusion. It's funny, because even his PHP code was pretty poorly written. According to the Mr. Parsons, the best PHP could do to calculate prime numbers up to 10,000 was 128ms. According to our optimized version of the scanner, PHP is capable of calculating prime numbers up to a value of 10,000 in 11.51ms.

Here's where the results get interesting:

ColdFusion, pathetic, ugly, poorly performant ColdFusion, despised and unpleasant to write with, actually did the same thing in 8.214ms.

Sorry Mr. Parsons... your "benchmarks" suck. ColdFusion is a full 3 and a quarter milliseconds faster!

I'm sure this will make or break a preponderance of web applications currently in development. /sarcasm

And incidentally, Mr. Parsons' professed goal with his post was specifically to prove ColdFusion inferior to PHP.

Ooops.

The test platform was:

  • Hardware: Apple MacBook Pro 2.8 GHz Core II Duo w/8GB RAM
  • Web server: Apache/2.2.15 (Unix) x64 (OS X Snow Leopard Default)
  • PHP: PHP 5.3.3 (cli) (OS X Snow Leopard Default)
  • CFML Engine: Adobe ColdFusion 9.0.1 x64 on JRun 4

And, for the record, none of the 3 server components were optimized in any way. Out-of-the-box and go.

Here's the code I used for each:

PHP:

view plain print about
1<?php
2
3function & primes($to) {
4 $primes = array(2, 3);
5 for ($i = 5; $i <= $to; ++$i) {
6 $root = sqrt($i);
7 for ($j = 0; $primes[$j] <= $root && $i % $primes[$j] != 0; ++$j);
8 if ($primes[$j] >
$root) {
9 array_push($primes, $i);
10 }
11 }
12 return $primes;
13}
14
15function arrayAverage($array) {
16    return array_sum($array)/count($array);
17}
18
19$times = array();
20
21for ($i = 0; $i < 1000; $i++ ) {
22    $t = microtime(true);
23    $primes = primes(10000);
24    array_push($times,microtime(true)-$t);
25}
26
27echo number_format(arrayAverage($times) * 1000, 5) . " miliseconds for " . count($times) . " iterations.";
28
29?>

Didja notice I had to roll my own average function? WTF?

ColdFusion:

view plain print about
1<cfscript>
2function primes(to) {
3 var primes = [2,3];
4 for (var i = 5; i
<= to; ++i) {
5 var root = sqr(i);
6 for (var j = 1; primes[j] <= root && i % primes[j] != 0; ++j);
7 if (primes[j] >
root) {
8 arrayAppend(primes, i);
9 }
10 }
11 return primes;
12}
13
14times = [];
15for (c=1;c<=1000;c++) {
16    start = getTickCount();
17    primes(10000);
18    arrayAppend(times,getTickCount()-start);
19}
20writeOutput( arrayAvg(times) & " miliseconds for " & arrayLen(times) & " iterations.");
21</cfscript>

Related Blog Entries

Comments (Comment Moderation is enabled. Your comment will not appear until approved.)
Jared,

I don't dislike ColdFusion based on it's performance, I dislike certain pitfalls like the lack of full object oriented support (such as class level methods and abstraction) and the super patchy support of cfscript (I know this is fixed in version 9 but look back at 8 which I have to use every day).

In fact there a lot of things I like about ColdFusion.

While my post wasn't about how best to calculate and build an array of prime numbers, I thank you for taking the time to put that together. And guess what running your test on my machine put PHP at 11.015 and CF at 6.558 which doesn't suprise me as I expected ColdFusion to better PHP in this test which I stated in my post.

I'm not a PHP aficionado, I just prefer the language.

Funny how you only showed the results of the one test though, did you manage to get CF to instantiate objects quicker?

Mr. Phil Parsons :)
# Posted By Phil | 1/30/11 4:59 AM
Phil,

I dedicated 3 full posts to your "tests".

There's the object instantiation one:

http://www.web-relevant.com/index.cfm/2011/1/30/OK...

And there's the output buffer one:

http://www.web-relevant.com/index.cfm/2011/1/30/Ou...
# Posted By Jared Rypka-Hauer | 1/30/11 5:21 AM
Thanks Jared,

I'm editing my original post to remove some of my bias based on the comments it has received. Carry on with your hate campaign if need be but please feel free to drop the condescending formality and use my first name.
# Posted By Phil | 1/30/11 5:44 AM
I'd like to thank Phil for being a good sport on his post and comments to his post. It could seem like we (Cf-ers) are piling on. Hopefully these threads will remain civil :-)
# Posted By Mike Henke | 1/30/11 12:30 PM
There appears to be a difference in the code
PHP (line 7): for ($j = 0; $primes[$j] <= $root && $i % $primes[$j] != 0; ++$j);
CF (line 6): for (var j = 1; primes[j] <= root && i % primes[j] != 0; ++j);

I'm no expert in CF or the prime finder equation demonstrated here, but it seems like these should match. Since this is a nested loop, this seems like it could have a drastic change to the output time...
# Posted By Keaton J | 1/30/11 3:22 PM
Very interesting. Can you compare the speed of a prime calculating function I wrote a while back?
http://www.codersrevolution.com/index.cfm/2009/8/1...
On my crappy slow CF8 home PC, it runs about 19% faster than your function. I've also found doing time trials for anything under 10 ms can be sketchy since internal rounding of nanoseconds can make a big difference, so I was comparing our functions at 1 million primes (5 seconds vs 6.2 seconds)
Thanks!
# Posted By Brad Wood | 1/30/11 6:57 PM
@Keaton : I assume you're referring to the initial value of "j". ColdFusion uses 1-based arrays and PHP uses 0-based arrays.
$primes[0] is the equivilant of primes[1].
# Posted By Brad Wood | 1/31/11 1:49 AM
@Brad:

OK so with your algorithm I can get the primes to 10k in just a decimal fraction over 14ms. With the one I was originally using from Elliot, I get the results to 10k in about 9ms.

If I expand the test to calculate primes to 100k, over 100 iterations I get an average of 160ms with yours and 130ms with Elliot's.

Something seems odd here.
# Posted By Jared Rypka-Hauer | 1/31/11 2:29 AM
Thanks for comparing. What in particular do you find odd about the results?
Obviously Elliot's function did better in your tests, but I'd say there are a number of factors that probably affected it like CPU speeds and me using 32 bit CF8 while you 64 bit CF9. If one function was more memory intensive or CPU intensive, one of those factors could easily give it a slight advantage on a given platform.
# Posted By Brad Wood | 1/31/11 3:17 AM
Brad,

After spending some time looking at things with Elliot last night we discovered something interesting. Figuring the primes to 125k, his function was faster. At 250-500k the two were almost neck-and-neck. At 1M, however, yours was faster by about 1/5 of a second. Which means that your function is more stable.

However, I am curious... if 2 is the only odd prime number, why are you checking every number from 1 to n when you could just check the odd ones and cut out half the processing? I tried to adapt your function to that but it assumes that arrayLen(arr) == num and breaks when you try to do anything else.

Any thought on updating it to that and seeing if there's any performance boost?
# Posted By Jared Rypka-Hauer | 1/31/11 11:56 AM
@Jared: Good question. There's a good answer too, but it's complicated. :)

First of all, read the Wiki page on the Sieve of Eratosthenes that I linked to in my blog post. It is the algorithm that my function is based on, and its goal isn't to find out what numbers in the array ARE prime. Instead it does the opposite and finds what numbers in the array AREN'T prime. Every left is prime.

Secondly, I assume you are using the third (and most optimized) version of the function in my post. It is constructed very carefully to never have to search the array, and to never modify (i.e. delete) from the array once it creates it as both of those operations can be very costly when performed a few million times.

The second reason I don't delete from the array is so I can easily find any number in the array with searching while still mark the non-prime ones. In my function, the value of the array always matches the index unless I have changed the value to "0" to indicate it is a non-prime.

An associative array (struct) might also be used, but I believe I remember trying it and looking up keys was too expensive for hash maps. I just couldn't beat the performance of finding the 1 millionth index of an array (as opposed to finding an item in a giant struct with the key of "1000000"), but the catch was that the array needed to contain every number between 1 and num for my "trick" to work.

So, back to your original question: why don't I skip the even numbers? Well, technically I do. Well, I short circuit them anyway. In fact, I short circuit every number which has already been marked as a non-prime. That is what the following line of code does:
if(!arr[p]) continue;

If the outer loop reaches a number in the array which has already been marked as a non-prime, it skips the inner loop right then and there. For instance, when the multiples of 3 are found-- 3, 6, 9, 15, etc will all be marked non-prime. Therefore there is no need to find the multiples of 6, 9, or 15 etc since their multiples are already a multiple of 3 and will have already been marked off. That means the list of array indexes that are allowed to be considered diminishes quickly as the outer loop grows. Just like the Wikipedia examples show the list shrinking. I'm leaving everything in my array, I'm just ignoring it.

I believe this may be why my function does well with larger numbers of primes because it is optimized to quickly whittle the list down on every pass of the inner loop so each increment of the outer loop skips more and more numbers.

Sorry for the long reply. :) Re-hashing this in my head has already given me a couple new approaches I want to try.

~Brad
# Posted By Brad Wood | 1/31/11 6:45 PM
Thanks for the followup, Brad... I appreciate it.

And yes, I'm running the third version from your blog.

I wonder if you wouldn't mind going back to your blog post and comment the hell out of that function so it's very apparent what each line is doing? I have a general idea, but the specifics are hard to reverse-engineer just by reading the code (and without putting it thru a debugger to watch states change and stuff).

That would be cool to see.
# Posted By Jared Rypka-Hauer | 2/1/11 2:35 AM
@Jared: I took some of your advice and was able to make my function quite a bit faster. In order to quit hijacking the comments of this post however, I have blogged my changes here:
http://www.codersrevolution.com/index.cfm/2011/2/1...

~Brad
# Posted By Brad Wood | 2/1/11 5:29 AM
BlogCFC was created by Raymond Camden. This blog is running version 5.9.7. Contact Blog Owner