Back to Blog

Endre Sara

Microservices in the Real World: How We Automated Memory Management for Our Containerized Java Application

This is the first of a multi-part series sharing our journey to application elasticity.

Most enterprise applications run in Java for the benefits of performance and security, but this technology has some unique challenges: Java needs its own memory management for each process/service. When you re-architect an application to be a collection of services that means you have more services that require memory management. Very quickly we found that scripts don’t scale—services are unique and dynamic, especially in Production.

Challenge

Turbonomic, like many of its customers, is modernizing their business application to leverage the speed-to-market benefits of a microservice architecture. It also enables our application to manage 20x more workloads. With a number of customers running 200,000+ workloads our application’s ability to scale with and automate Java_Container_HighFivethese environments is critical.

In 2009 we started with a monolithic Java application that was running as a Tomcat web application. Managing the memory was simply a matter of configuring the Tomcat’s maximum heap from the startup script based on a percentage of the memory allocated for the VM. But, when we re-architected the application as microservices, containerizing each functional component, managing Java memory allocation became 30x more complicated. 

Each component needed different amounts of memory depending on the following things:

  • The environment to which it was deployed
  • The size of the public and/or private cloud estate it manages
  • And, the number of applications in those environments being managed--increasingly dynamic as our customers deploy more and more dynamic and elastic applications. 

Originally, we configured the max heap size as an environment variable that was used as part of the Java options in the container entry point. Our developers faced the unfortunate task of writing scripts for memory requests for each Java process. This doesn’t scale. Also, it’s 2019 and we’re all about making application resource management automatic. So the question became: how do we make our application configurable so that software can manage these resources?

Solution

Historically, like everyone else, we’ve had to hardcode our Java configurations, but with recent updates these configurations can be more dynamic. In Java 11 a new option was introduced (later back ported to Java8u191), which was specifically for Java applications running in containers. It allows us to use cgroup memory for Java heap configuration (-XX:+UseCGroupMemoryLimitForHeap) and to configure this a percentage of the container limit. For more details see here and here.

The containerized version of our application was initially deployed with Java8, so we enabled experimental VM options (-XX:+UnlockExperimentalVMOptions), but then later when we upgraded to Java11 only the previous option were necessary.

Since the Java application is the only process running in the container, we wanted to use most of the container memory for Java heap. So, we increased the percentage of the cgroup limit to be used for Java heap to 75% instead of the default 25% (-XX:MaxRAMPercentage). So far, this was sufficient to still have enough memory for the JVM overhead and use the memory effectively for the actual application. For reference, see here

Breaking up an application into microservices creates complexity and it’s no less true with Java applications. The complexity here was a result of having to determine memory allocation for each individual service—in this case, 30 services replaced our previously one monolithic application. We don’t want our developers spending time writing scripts for each individual service, so we had to find a way to dynamically and automatically adjust the memory allocation based on the specifics of each service. 

Solution: automate the management of the Java memory allocation based on container allocation. Now you’re asking, “Yeah, but don’t developers determine the resources allocated to containers?” Answer: not if they have Turbonomic automatically determining the right container configurations based on the real-time needs of the service.

The bottom-line is that container allocation can be managed through APIs. And APIs open a whole world of possibilities for automating resource management. We love containers, but we also love Java. And this is just one example of how we got the best of both worlds. For those of you facing similar challenges with your enterprise applications, I hope you found this post helpful. In fact, I’d love to hear some of your war stories, so leave a comment!