Summary
This article is a response to the lack of information on the net about the performance differences among the 4 most well known Java Persistence API (JPA) implementations: Toplink Essentials, EclipseLink, Hibernate and OpenJPA.
Besides results and conclusions, the full test code is also available in case you want to repeat the test yourself.
For the tests performed for this article, nothing except JPA was used. No web pages, no web or application server. Just java threads, JPA and MySQL. I give more details in the next sections.
Note: In case you are using JPA with Axis and/or the Google Web Toolkit (GWT), this other article focused on working with JPA, Axis and GWT could be of interest for you.
Description of hardware and software
- Ubuntu 8.10 Intrepid Ibex
- MySQL database, version 5.0 (installed from the official Ubuntu repositories).
- Java Virtual Machine 1.6
- Driver jdbc for MySQL 5.1.
- Eclipse Ganymede
- The employees database example for MySQL, courtesy of Patrick Crews and Giuseppe Maxia (url below in the references section)
- JConsole for resources monitoring
- GIMP 2 to capture screens
- Hibernate EntityManager and Annotations 3.4.0
- Toplink Essentials version 2 build 41
- Openjpa 1.2.0
- Eclipselink 1.0.2
Inserting thread loop gets an arbitrary employee and makes a copy of him/her, letting MySQL generate a new emp_no. This was the only modification I did to the employees database: the emp_no is auto-generated.
Querying thread loop executes these queries in sequence:
- A query returning the number of female employees.
- A query returning the number of male employees.
- A query returning all employees hired since an arbitray date.
- A query returning all employees born after an arbitrary date.
- A query returning all women who have earned more than an arbitrary salary.
- When the program starts, it waits 2 minutes for the monitoring infraestructure to be ready (connecting JConsole to the JVM, basically).
- It then starts 2 of the so-called inserting threads. I start the inserting threads before the querying threads trying that the queries do not always return the same (which will eventually happen, anyway).
- After starting the inserting threads, the program starts running 18 of the querying threads, inserting a pause of 10 seconds before starting next. This is so that they do not execute the same query at the same time.
- The program runs the threads for 30 minutes. After that time, it sends a stop signal to the threads, which will safely make them stop after the next inserting or querying round. The main program waits 15 minutes for the threads to stop and the jvm memory to stabilize.
- Before stopping, the threads provide information about the number of inserts/queries they have executed.
Before every test, the inserted records were deleted. In this way, every implementation started with the database exactly in the same situation.
Number of queries+inserts executed | Number of queries executed | Number of inserts executed | Max mem occupied during the test(Mb) | Mem occupied after the test(Mb) | |
OpenJPA | 3928 | 3530 | 398 | 96 | 61 |
Hibernate | 12687 | 3080 | 9607 | 130 | 79 |
Toplink Essentials | 5720 | 3740 | 1980 | 55 | 25 |
Eclipselink | 5874 | 3735 | 2139 | 57 | 25 |
The memory occupied after the test is the amount of memory that remained reserved after finishing the test.
I have emphasized the highest and lowest values for each of the columns.
Conclusions
Nevertheless, I consider that there are a number of conclusions that one can draw watching the monitored data:
- There is not an implementation that clearly has the best performance. Some had a very good CPU or memory performance and some did it very well when inserting or querying. But none of them was outstanding as a whole.
- The number of records inserted by Hibernate was extremely higher than it was for any other implementation (4 times more compared to Eclipselink and 24 times more compared to OpenJPA). However, Hibernate was also the JPA implementation that executed the lowest number of queries, although the differences in this value (3080 for Hibernate vs 3740 for Toplink Essentials) are not so extreme as for the number of inserts.
- Hibernate was also the implementation that consumed more memory. But having into account that it inserted many more records than the others, it sounds reasonable.
- OpenJPA had the lowest value of inserts+queries.
- The number of inserts executed by OpenJPA was extremely low, compared to the others.
- The usage of CPU in the case of Toplink Essentials and Eclipselink was extremely low.
References
Ubuntu: http://www.ubuntu.com/
Employees database: http://dev.mysql.com/doc/employee/en/employee.html, https://launchpad.net/test-db/
Openjpa: http://openjpa.apache.org/
Toplink Essentials: http://www.oracle.com/technology/products/ias/toplink/jpa/download.html
Hibernate JPA: http://www.hibernate.org/397.html
Eclipselink: http://www.eclipse.org/eclipselink/
MySQL: http://www.mysql.com/
Eclipse: http://www.eclipse.org/