We’ve been reviewing various knobs mostly related to the infrastructure resources and uncovered some complexities which make the manual decision process dangerous and inefficient. But let’s not forget the ultimate goal of doing all these actions – we need to assure application performance. So while turning the infrastructure knobs may give applications required resources, there are a fair amount of actions you can do within the application itself - which just adds even more complexity.
Application knobs ????
Let’s look at some of them. Applications can be very diverse; after all they are responsible for delivering end-user functions which vary a lot. However, there is some pattern common to all – many applications perform so-called transactions which could be considered as atomic operations delivering to a recipient a result of a query. For example, a SQL query to a database to update a list of online shoppers, or an HTTP GET operation to retrieve an ad page from a web server.
In order to process a transaction, an application needs to consume some local resources (like memory to store the results) and then submit it to another application using some communication protocol (like sending an HTTP request over TCP/IP). To perform all of these, an application also uses underlying computing resources. If you need to perform more than one query at a time you could start using multiple threads which are light-weight computing processes carrying out the requests.
So if we use this model then what could be a good measure of application performance? One of them could be the transaction processing speed – how many transactions per second you can process and a related metric – or application response time, i.e. How long a user will have to wait to get the results back. To guarantee expected performance we need to make sure that every time an application issues a transaction it has all needed resources and delays are minimal.
Let’s start with the expected performance. Let’s assume an application server needs to process 4 parallel page retrieval requests from connected web servers. So it would need to allocate 4 threads in its thread pool so all of them are available when requests come in. If the expected performance grows (say you establish a Gold service tier guaranteeing 8 parallel transactions) one of the actions you could do is to extend the thread pool and add more threads. Sounds fairly straightforward. However, in order to accomplish the required parallelism the underlying VM needs to map every thread to an individual vCPU otherwise requests will have to wait in the queue which will increase the response time.
So you cannot just add threads, you also need to increase the number of vCPUs. As an application owner you could do that in your VM by doing resizing – certain OSes can even allow you to do it in real time dynamically addressing increased demand. But remember, we just added virtual CPU but the true parallelism is accomplished by the availability of physical processors in the underlying hosts. So even if you have 8 vCPU but the underlying host has only 4 physical cores, 4 other threads will have to wait to get scheduled after the 1st 4 finished their slices.
Let’s assume you found a host with 8 physical cores and moved your VM there. But if there are other 8-way VMs on that host, they will be competing for these 8 cores and will have to wait in the ready queue – again increasing the application response time. To minimize the wait time you’d better find a host with the smallest ready queue for 8-way VMs. But does this host have enough RAM to satisfy the Java heap demand as we discussed earlier?
Now we see that such a simple control action performed by an application owner could trigger a series of steps which might also be needed to address the original goal – to keep the application performance healthy. The same lesson again: no single action can be taken in isolation, they are all related and while a correct set of actions does exist, figuring it out in real time is practically impossible for human mind.
We will look at more application and cloud related actions next time, for now we are asking the same question – do you know which steps you need to take to assure application performance and keep your infrastructure efficient?