Quarta-feira, 31 de Dezembro de 2008

A fast implementation of WSTest in Java

I've recently come across the Microsoft updated versions of the WSTest Web Services Benchmark and the .NET StockTrader Sample Application. They wasted no time bragging about the results :-)

Microsoft encourages people to download the benchmark kit and perform their own tests, so I did that. I will ignore the StockTrader App for now because is more complex to install and analyze, I will focus on the WSTest benchmark. The .NET/WCF results are very good and the guys at benchmark labs seem to really know their stuff. It's a pity that the benchmark choose to compare .NET/WCF against WebSphere, probably the most expensive, slow and cumbersome of all Java Application servers.

In the Java-Land there are faster solutions to choose from. I decided to implement my own version of the benchmark to verify just how fast or how slow can a Java implementation be.

The test is essentially a XML serialization/deserialization benchmark, so I picked the speedy JiBX as the framework for Java/XML data binding. JiBX is only as fast as the underlying XML parser and the fastest Stax parser that I know is the Aalto XML Processor. We will also need an HTTP layer and for this I really like the Mina Http Codec.

With all the ingredients in place it didn't took long to produce a benchmark implementation that doesn't suck :-). The code is available here.

And what about the results? Unfortunately I don't have a server class machine laying around for running proper tests. However, I do have VirtualBox and two virtual machines, one with "Windows2003 Server" that runs the "Self-Hosted" WSTest application and another with "Ubuntu 8.10 Server" that runs the Java implementation using the sun-java-jdk-1.6.10 jvm. Using soapUI as a load generator the "linux/java" setup runs circles around the "windows/.net/wcf", in some cases the throughput numbers are more than twice as high. Of course that these results should be taken with a truckload of salt. The tests should have been performed with a proper server machine, using Windows 2008 Server in the .NET setup and with several machines running the load generators. I would love to hear from someone that has a "benchmark lab".

Update: The http bits are now handled by Grizzly, the performance seems to be better.

Update: Check the follow-up post for a more detailed performance test.

tags: ,
published by luisneves às 01:01
perm link | comment | add to favourites

.search this blog

.Fevereiro 2009

Dom
Seg
Ter
Qua
Qui
Sex
Sab
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

.recent posts

. Alternative serialization...

. WSTest, some numbers - Up...

. A fast implementation of ...

. European PGDay 2008

. Need for speed

. Receiving events/notifica...

.archives

. Fevereiro 2009

. Janeiro 2009

. Dezembro 2008

. Outubro 2008

. Maio 2008

.tags

. todas as tags

.subscrever feeds