Investigating the Performance of Reactive Libraries in a Quarkus Microservice

What’s the performance benefit of using reactive libraries in my Java applications? Can I go step-by-step, or do I need to go all-in and refactor the whole code to gain performance benefits of these reactive libraries? If you are dealing with similar questions, this blog is for you! In this blog, we will review different refactoring options for a Quarkus-based Microservice using frameworks such as RESTEasy Reactive, Hibernate reactive and Mutiny from a performance perspective. A typical service architecture of a micro-service is shown in figure 1. In this blog post we will outline which reactive frameworks we have used in this evaluation on the different layers of the architecture. Before going over the details and which combinations of reactive libraries are possible and their performance implications, let us start with the mechanism that is at the core of most reactive libraries.

Investigating the Performance of Reactive Libraries in a Quarkus Microservice

Figure 1: Service Architecture [1][2]

IO-Threads

Traditionally, Java-Enterprise frameworks assign a thread per request. If one thread interacts with a remote source, then that thread is blocked. In other words, this thread cannot process another request as displayed in figure 2. This system places a limit on achievable concurrency. On the other side, the reactive thread mechanism seen in figure 3, relies on non-blocking I/O and therefore the same thread can manage multiple requests. When communicating with a remote service (API or database) instead of blocking the thread, an I/O operation is scheduled with a continuation attached to it. For instance, if a request comes in that retrieves an entity from the database, instead of blocking, that request schedules an I/O operation and attaches a continuation so that other requests can start their execution.

Investigating the Performance of Reactive Libraries in a Quarkus Microservice

Figure 2: Worker Threads
Investigating the Performance of Reactive Libraries in a Quarkus Microservice

Figure 3: I/O-Threads

Reactive Libraries for Java

RESTEasy Reactive

The point of entry in your Java-based microservice applications is usually a REST-based endpoint where the RESTEasy Reactive library supports you in the transition to a reactive request processing shown in figure 4. This is especially convenient if you already use the non-reactive version of RESTEasy as of today.

Investigating the Performance of Reactive Libraries in a Quarkus Microservice

Figure 4: RESTEasy Reactive in a Microservice [3]

RESTEasy is responsible for modeling your business domain as a resource. With an application protocol such as HTTP over REST it is possible to communicate with the underlying resource. It checks an incoming request and provides a response appropriate to the outcome of the business transaction. RESTEasy Reactive has multiple ways to be utilized. You can start working with RESTEasy Reactive by simply adding the @Blocking annotation as shown in listing 1. In this case the library would use traditional worker threads instead of I/O-threads.

Copy to Clipboard

Listing 1: RESTEasy Reactive with @Blocking

If the processing of the request should happen on the I/O-Thread aswell, then omit the @Blocking annotation and make sure that the underlying code has no blocking interaction as shown in listing 2. With RESTEasy you can also utilize reactive types.

Copy to Clipboard

Listing 2: RESTEasy Reactive returning a reactive type

Mutiny

Mutiny is a reactive library that supports asynchronous programming with Observable like types. It is event-driven meaning that you can express what you want to do upon a given event (i.e., failure, success, completion). The use of a reactive programming library will affect the majority of your codebase therefore it is positioned at the core of your application as shown in figure 5 and will of course lead to the increased refactoring effort of all libraries mentioned in this blog post.

Investigating the Performance of Reactive Libraries in a Quarkus Microservice

Figure 5: Mutiny in a Microservice

You only have two reactive types in the library. A Uni representing a single item or a failure and a Multi representing no items or potentially an infinite amount of items. Let’s compare a traditional implementation of a service method with a version utilizing Mutiny. In listing 3, the service method starts with calls to other services. These calls retrieve the concrete values referenced in the payload of the request. The total value of all included items is calculated before storing and returning the entity.

Copy to Clipboard

Listing 3: Fully Imperative Service Method

If Mutiny was utilized, then the results of external services are concurrently combined into a single object as shown in listing 4. That object is then mapped to a new object before it is persisted. The second version also does not reactively interact with external services or the underlying datastore. Instead it simply wrapps the results of those calls inside of reactive types.

Copy to Clipboard

Listing 4: Service Method utilizing Mutiny

Hibernate Reactive

Hibernate is an Object Relational Mapping (ORM) framework which also released a reactive version of its library recently. In a Java-based microservice, Hibernate typically represents the data access layer as shown in figure 6:

Investigating the Performance of Reactive Libraries in a Quarkus Microservice

Figure 6: Hibernate Reactive in a Microservice [4][5]

The hibernate framework empowers you to more easily write applications with persistence in mind. The core functionality is to simplify the mapping of java objects to a relational data model. To interact with the traditional hibernate in a Java application you can utilize the Java Persistence API (JPA) to inject an EntityManager that provides the necessary methods to communicate with the underlying datastore as shown in listing 5. If you have a solution with RESTEasy Reactive and a blocking Hibernate JPA-implementation, then you would have to set the @Blocking annotation on the api method to make sure that the processing is dispatched to a worker thread and no exception is thrown.

Copy to Clipboard

Listing 5: Traditional Hibernate utilizing the Java Persistence API

To utilize Hibernate Reactive instead of the traditional Hibernate version, you cannot use JPAs EntityManager but need to use the Hibernate Session objects instead. Hibernate reactive introduces specific session types depending on the Reactive framework which is used in the application. In our example we use a Mutiny.Session was we use Mutiny as reactive framework. This Mutiny.Session can be injected in two ways using a direct Injection as shown in listing 6 or using a SessionFactory as shown in listing 7. The single session essentially provides the same methods as a simple EntityManager but instead of returning commonly known types, every result is wrapped inside a reactive type. When using a single Mutiny Session, you are not allowed to share that session between concurrent reactive streams. If you do, you’ll encounter an exception. To avoid that, you have to inject a Mutiny.SessionFactory.

Copy to Clipboard

Listing 6: Hibernate Reactive using Mutiny.Session
Copy to Clipboard

Listing 7: Hibernate Reactive using Mutiny.SessionFactory

Performance Test – Implementation Variants

In the following we will outline the performance evaluation we have performed on different implementation variants of an example Microservice. We have refactored the application in several steps until it became fully reactive. For each step we have performed a performance test to see the impact of the reactive refactoring. Overall seven variants have been implemented: and are shown in table 1. If a column is marked, then the corresponding reactive library has been used.

#Number RESTEasy (reactive) Mutiny Hibernate (reactive) Notes
1 X
2 X @Blocking
3 X
4 X X @Blocking
5 X X
6 X X X @Blocking
7 X X X

 

Table 1: Implementation Variants

Performance Test Setup

The REST-Service under test provides basic ordering capabilities typical in an e-commerce platform. In particular, the following endpoints are available:

  • GET & POST /orders
  • GET & PUT & DELETE /orders/{orderId}
  • GET /orders/{orderId}/items
  • POST & GET & DELETE /orders/{orderId}/items/{itemId}

We are going to take a closer look at the HTTP-Post on /orders. This method out of all the available api methods is the most complex one. It interacts with six external sources while the rest of the api either changes a value from an entity or simply returns that entity.
A suitable test environment was provided to conduct the experiment and is shown in the next figure.

Investigating the Performance of Reactive Libraries in a Quarkus Microservice

Figure 7: Software Experiment Setup [6][7]

The experiment is orchestrated with Kubernetes in which every container has resource limits defined. The Order-Service is instrumented with the java-auto-instrumentation of OpenTelemetry. The instrumented Order-Service through its agent provides telemetry data to a Jaeger service. Lastly, for load testing JMeter was used with an additional plugin providing arrival thread groups.

During the execution of a test run, the Order-Service is built and a Docker image is pushed to RETITs docker registry. Afterwards, Kubernetes deploys the core containers of the application (i.e., Order-Service, PostgreSQL, Jaeger) and follows that with a deployment of a Maven container which executes a JMeter project with the included test script. Before finishing the test execution, results are archived for later use and the previous deployed containers are deleted.

Performance tests are run with a consistent workload of 1000 requests per minute. This workload was derived from an average customer coversion rate of 3% and the traffic of a randomly chosen e-commerce website. Different test cases exist, as seen in the next table 2, that try to mimic a workflow on an e-commerce website as close as possible.

#Number Use Case Description Arrivals/minute Request Per Iteration
1 Online Shop Order The user creates an order and after checking out receives the order information 300 2
2 Rescinding Orders The user wants to rescind the order he created 50 2
3 Changing Payment Information The user wants to change the payment information of orders he created 50 2
4 Order Creation With Multiple Items The user creates an order and adds additional items 25 4
5 Order Creation With Unwanted Items The user creates an order, adds multiple items and removes an unwanted item 20 5

 

Table 2: Test Cases executed in Software Experiments

The Online Shop Order test case simply consists of the user creating an order and after checking out receiving the order information. This workflow will receive the bulk of arriving requests. In particular, every minute test case #1 should be executed 300 times. The experiment also models the user Rescinding an Order with 50 arrivals per minute. In contrast to the first use case, the user rescinds the order after creating it. Changing Payment Information with 50 arrivals per minute and Creating Orders With Multiple Items with 25 arrivals per minute also portray typical use cases in e-commerce platforms. The former simply changes the referenced card in the created order, while the latter adds multiple items after the order creation. Lastly, test case #5 describes the Order Creation With Unwanted Items, meaning that some of the added items will be removed.

With the average conversion rate of 3% for e-commerce applications and the traffic count for a randomly chosen e-commerce site, we have derived roughly 1000 requests per minute for a representative average workload. The request amount is distributed across all previously defined test cases. The arrivals per minute do not equal requests per minute. Every test case consists of at least one request. Multiplying the arrivals per minute and request per iteration results in requests per minute.

Performance Test Results

The test setup was executed for all the seven implementation variants. The resulting response times can be seen in the figure 8. The fully imperative solution averages a response time of 50 milliseconds (ms) while the variant with a blocking RESTEasy Reactive has a mean response time of 49ms. There is another set of implementations that is equally comparable. A variant with RESTEasy Classic, Mutiny and Hibernate Classic has a mean response time of 31ms. Replacing RESTEasy Classic with RESTEasy Reactive and the @Blocking annotation produces a mean response time of 30ms. It can be seen that the overall response time distribution has a lower tendency for the RESTEasy Reactive Blocking version. The performance of every variant utilizing Hibernate Reactive is almost identical. Comparing the three solutions presents only slight changes in the mean response time or the range of data. Additionally, every variant with Hibernate Reactive seems to be outperformed by its counterpart where Hibernate Classic is the difference.

Investigating the Performance of Reactive Libraries in a Quarkus Microservice

Figure 8: Software Experiments Test Results

The negative impact of Hibernate Reactive in the current environment is most likely caused by how the library obtains reactive sessions. A session is obtained through the withSession method of a Mutiny.SesssionFactory. The session is automatically associated with the current reactive stream and also automatically closed after work has been done. If a nested call to withSession occurs, then that same session is used. During a test run, this causes the performance test to start & stop multiple sessions. With the already low response times the overhead of managing multiple reactive sessions impacts the overall performance in the current testing environment unfavorably. Using single Mutiny sessions in the current setup is not functional, because a single Hibernate Reactive session cannot be shared between concurrent reactive streams. This is especially the case for the createOrder method where the results of different external services are combined. Additionally, because the performance test are executed with concurrent thread groups, a single injected Mutiny session would not work. Given the aforementioned handling of the sessions in Hibernate reactive, a detailed look on the transaction behaviour of the different session handling options will be done in a seperate blog post.

Conclusion

Out of all reactive libraries utilized, the hardest one to get used to is Mutiny. The performance gain with Mutiny can be substantial, especially when combining the results of unrelated external services. Making the switch to RESTEasy Reactive can be simple if no reactive types are returned. Blocking RESTEasy Reactive already provides a minor performance improvement. The transition to Hibernate Reactive can be challenging. The data access layer is usually separated from the service layer. Even if the data access is put into dedicated objects, the service classes which make use of these objects would have to be refactored because Hibernate Reactive either uses reactive types from Mutiny or the asynchronous type CompletionStages. It requires less work and would provide a significant performance gain if Mutiny alone is used on top of existing data access objects. The easiest way to get started with Mutiny is to introduce it at the entry-level of your application.

Useful Links

References