In today’s virtual service environment, customers and providers are facing a huge dilemma: lack of comparison

During the old days you had a box with one or more processors of a certain performance type and clock speed. Comparing two different boxes was pretty easy through CPU benchmarks and memory configuration. Then came the new multi core processors and comparing two CPUs, even of the same type, got difficult. So Intel and AMD provided us with Names and ID Numbers to estimate the relative performance… within their own brands.

Virtualization takes the already hard to compare performance a few steps further. We are no longer talking about processor cores, but about constructs like a “quarter of a vCPU per minute”. Try to compare this performance to anything you’ve managed in the past. You simply can’t!

Amazon introduced us to the impossible to grasp ECU (EC2 Compute Unit), IBM is selling the VCU (Virtual Compute Unit) to it’s outsourcing customers and most of the other cloud and outsourcing service providers struggle by creating their own or jumping onto already defined Measurements. Currently none of these are intended to be comparable.

But they all use some kind of Benchmark to create their “Compute Units”. And here comes the next obstacle: There are hundreds of benchmarks and none of them are actually comparable.

The wildest thing of creating a server processing benchmark was modifying the Amazon ECU to be based on PassMark

“Daniel Berninger of goCipher Software proposed broad adoption of the ECU and a mapping of 1 ECU to a 400 PassMark score.”

Sounds good, until you notice PassMark is a CPU only benchmark and their database consists mainly of Home PC processors. One problem with this is that AMD processors usually do pretty well on these lists, but are much slower in an actual data center operation setup. Another difference is, that the CPU by itself does not say anything about server performance.

I know the VCU, IBM is selling is based on RPE2, which is an aggregation of most of the relevant server system benchmarks. Some customers are directly asking for a RPE2 or tpm-C equivalent for service offering. Then you also have your crazy SAP guys, that only talk in SAPS.

And now I’m facing one simple question: “How can you make all of these comparable?”

That’s pretty much, where my headache started yesterday. How can you compare the intentionally uncomparable units?

I know how to get from RPE2 to tpm-C to SAPs and back, but where can I cut into the ECU oder UCU (Universal Compute Unit) or Whateveryousell-Unit. Three factor equasions with two unknowns are really hard to tackle, if you need a simple number in the end.

And never was that good at math.