This blog is part of a series in which we share some of the results from our State of Performance in Modern Applications Survey. Want to binge read the results? Download the full report here.
The criticality of application performance to the business is undisputed. The fact that over 60% of organizations are measuring performance is a testament to its importance. But the methods by which they do so require an evaluation of ROI. It’s a business decision to invest in measuring application performance, so the question becomes under what circumstances does this investment pay off?
Most organizations that measure performance are measuring availability, not response time or transaction throughput.
When we asked respondents how their organization is measuring application performance, it was promising to see that over 60% are measure it in some form. But the most common approach was measuring availability, as opposed to managing to Service Level Objectives (SLOs), which typically take the form of response time or transaction throughput.
We expect organizations to increase their focus on managing to SLOs, especially given their direct correlation to the customer experience. Today’s end-users don’t have the patience for slow performing applications. For example, 40% of mobile web users are unlikely to return to a website that they had trouble accessing in the past (Source: Morgan Stanley Research “A New Software Stack for the Digital Era,” May 2019).
Watch How You Can Automatically Assure Application SLOs
Turbonomic allows you to set service level objectives (SLOs) and the software will automatically ensure that they are continuously met.
We do see distinctions across industries, most notably that Media and Entertainment and Retail/E-Commerce organizations are leaders when it comes to measuring SLIs and defining SLOs.
⅓ of organizations measure application performance continuously in real time, while 50% measure it periodically.
While a third of respondents said they measure application performance continuously, half responded that they measure it periodically. It’s not easy to get response-time, it’s an investment with tradeoffs that need to be evaluated. And in some ways microservices actually makes it harder because they are more distributed and complex. There’s a cost to collecting response time data on multiple fronts:
- The labor cost of implementing the data collection; it must be done for every service within an application.
- The labor cost of monitoring and making use of the data collected. You need to be committed to defining what metrics best indicate the good (or bad) state of the service, usually this is defined by the Line of Business or Application/Product Owners.
- The overhead cost of the implementation. What starts out as a lightweight service executing a specific function risks turning into the opposite if you have to add processes to that service just for the purposes of observability.
- Lastly, if you’re using a commercial solution, there’s the financial cost of licensing.
So the question for an organization is: Is there something being lost by not measuring performance continuously or does periodically measuring it capture what they need?
When we break this question out by industry, Financial Services (47%), Service Providers (38%), and Media and Entertainment took the lead in measuring performance continuously.
While every organization cares about application performance, it’s still an investment that requires resources and consideration of ROI. Another way to look at it is to evaluate the operational and financial resources an organization has to put towards continuously measuring performance versus the extent to which their customers are a captive audience. For example, Retail/E-Commerce are much more vulnerable to customer churn than Financial Services: you’re more likely to switch brands for a sweater or shoes, if the application experience is bad, than you are to switch banks (...Up to a point, I love my mobile banking app, which coupled with their customer service makes me a loyal customer. Without that app though, I’d be looking elsewhere. I don’t have time to bank anywhere but on my phone.) But, Financial Services also generally have more resources ($$$) to spend and bigger margins to work with than Retail/E-Commerce and other industries. Meanwhile, Service Providers are contractually obligated to provide a certain level of performance to their customers. Breaking their SLAs has costly consequences so measuring performance continuously is well worth the investment. On the other end of the spectrum, Education and Government generally have captive audiences and not nearly as much resources so it’s no surprise that they are less likely to measure performance continuously.
What’s your take on the frequency of measuring application performance? Will every industry need to measure it continuously or are there some scenarios where it’s never going to be necessary? In your experience, when does continuously measuring application performance pay off?