<img alt="" src="https://secure.bomb5mild.com/193737.png" style="display:none;">

Turbonomic Blog

Machine Learning, Cost vs. Performance and The Edge

Posted by Jacob Ben-David on Dec 12, 2019 9:36:04 AM
Jacob Ben-David
Find me on:

Last week approx. 65,000 IT professionals and executives converged on to Las Vegas for the most significant cloud event in the world: AWS re:Invent. It is hard to describe the spectacle, noise, excitement, and energy at the event.


The expo was full with vendors competing for attention in an overcrowded and highly competitive marketplace. Banners and LED screens were screaming 'Cost reduction,' 'AI / ML-based,' and 'Automation,' at innocent and confused bystanders (with bags full of swag).

It was quite hard to understand what companies were doing, simply because most their promotion materials looked the same.

If you followed the new product launches and announcements (77 according to AWS) leading to re:Invent and throughout the show, there were three prominent themes:

Significant efforts to provide more performance at a lower cost

Amazon realizes that the cost/performance ratio is top-of-mind for its large users, and they are investing heavily in it. Let's look at some of the announcements related to this theme:

  • AWS Savings Plans - announced in November (during Microsoft Ignite), it is the evolutions of Reserved instances. It offers the same discount of RIs but significantly simplifies the consumption and management. Users commit to a $/hr, and that's it (at least with Compute Savings Plans). Click here to learn more
  • AWS Compute Optimizer - announced during the first day through a blog, that many missed. It's an ML-based cost optimization offering that will scale workloads up or down. It is the 4th offering from AWS of disconnected tools designed to help cost management. The other tools are Cost Explorer, Trusted Advisor, and Amazon EC2 Resource Optimization Recommendations service.
  • New M6g, R6g, and C6g instances - based on the second-generation of the custom AWS Arm-based chip (Graviton2), the new instances will offer a 40% better cost/performance ratio compared to the Gen 5 instance (for example M5). M6g is in preview right now, and the rest will arrive in 2020.
  • Inf1 instance for EC2 - based on AWS custom-designed chip (Inferentia) created to address ML Inference (e.g., predictions or applying “trained” capabilities to new data), which according to AWS, Inference is responsible for the majority of the costs related to ML. The new instance will offer 3x higher throughput and up to 40% lower cost-per-inference compared to G4 instances (based on NVIDIA T4 GPUs)
  • AWS Nitro - although not new, both Andy Jassy & Werner Vogels spent significant time talking about the continuous effort they invested into the Nitro system to provide more performance at lower costs. The Nitro system essentially is an effort by AWS to offload many traditional hypervisor capabilities to dedicated hardware and software to provide high performance, HA, and security while reducing virtualization overhead. A new announcement at the show was
    AWS Nitro Enclaves, which allows customers to further isolate and secure their confidential data running on EC2 instances.


The future is ML/AI, serverless and PaaS

Out of the 77 new releases and announcements, most (20) were Machine Learning focused. Amazon has expanded its ML-based services portfolio as well as the tools for developers and scientists to create their own ML solutions.

  • Amazon SageMaker - SageMaker itself is not new; it is a fully managed service that provides users with the ability to build, train, and deploy ML models. Andy Jassy spent a significant time of his keynote on SageMaker and the new features, starting with Amazon SageMaker Studio, the first web-based IDE for ML developers. Various development tools for SageMaker were announced designed to model, trace, debug, and train ML models.
  • AWS CodeGure - this one is very cool. It is an ML-based automated code reviewer/profiler service designed to help developers optimize their code. It finds and fixes issues in code such as race conditions, resource leaks, and wasted CPU cycles, among other things. AWS promotes it as "It's like having a distinguished engineer on call, 24x7". Amazon shared statistics comparing the improvements they gained on Prime Day 2018 vs 2017 after using CodeGuru to optimize the most expensive lines of code (325% increase in CPU utilization and 39% lower cost)


  • Other noteworthy services include Amazon Fraud Detector - fully managed fraud detections service combining ML, user data and 20 years of Amazon experience of online fraud detection); Amazon Kendra - enterprise search service powered by ML with connectors to various sources such as SharePoint, file systems, SFDC, JIRA and more.
  • Amazon Fargate for EKS - we all knew it was coming, and here it is. The existing/older Fargate was built upon ECS, which was based on Docker. As the market fully adopted Kubernetes, it was only a matter of time until AWS will offer Fargate based on K8S. Fargate allows users to run containers without having to manage, patch, and scale their EKS clusters.

Moving the Compute closer to the data and users (Edge)

The last theme revolves around AWS efforts to extend its cloud and move the compute closer to the end-users. There were 3 announcements related to that:

  • AWS Outposts now GA - AWS On-premises focused solution, designed for workloads that require low latency access to on-premises systems, local data processing or storage. It provides the same experience, APIs, and services as AWS. The current version includes Amazon EC2, EBS, ECS, EKS, EMR, VPC, and RDS (preview). S3 is coming soon. There will 2 variants, Native AWS (GA) and 'VMware Cloud on AWS' (coming in 2020).
  • AWS Local Zones - a new architecture model that extends AWS regions closer to large cities to deliver "single-digit-millisecond latency" to local end-users. Currently GA with invitation-only in Los Angeles. Ships with EC2, EBS, FSx, VPC, and ELB. RDS is coming soon.
  • AWS Wavelength - the result of a close partnership between Amazon and Verizon (US), Vodafone (EU), and SK Telecom (South Korea). Verizon CEO was on stage for this announcement. Similar concept as AWS Local Zone, this service extends AWS infrastructure to the edge of 5G networks by placing AWS hardware (based on AWS Outposts) within the telcos datacenter. It is designed for applications that require "single-digit-millisecond latency," such as gaming and VR, among others.

It was almost impossible to keep up with all the announcements, and I admit, I missed a few. In his blog, Marco Meinardi from Gartner, pointed out that most people missed two major announcements from AWS that might be the beginning of a shift in AWS’ approach to multicloud (they were not big fans). The first was that AWS CloudFormation (Infrastructure as code solution) will support third-party resource providers, just like Terraform’s Provider concept. The second was that AWS Config will also support third party resources (such as Github repos, MS Active Directory or any on-premises servers) extending its CMDB/Config Management capabilities beyond AWS. Should ServiceNow or Hashicorp be worried? Probably not.

As a final thought, I would like to leave you with a screenshot from the keynote. During the keynote, Andy Jassy spoke about the four key points for a successful transformation (he said 6, but I was not able to find the other two). I agreed with every point listed, mainly because I have seen it with my own eyes. The organizations that decided to be aggressive with their transformation goals, while management provided full support and alignment, were successful with their transformation. They invested and trained their teams as well as didn’t let the fear of change hold them back.


In case you can’t view the image for some reason, the 4 points are:

  1. Senior leadership team conviction and alignment
  2. Top-down aggressive goals
  3. Train your builders
  4. Don’t let paralysis stop you before you start

I highly recommend listening to this specific section (about 6-7 minutes) and the examples he uses.

In Summary, AWS re:Invent and Microsoft Ignite before it is an excellent indicator of the velocity of innovation. As a technology geek, I am genuinely excited to be in this space and can't wait to see what the future holds up.

I believe that the tools available today for Machine Learning (such as SageMaker) will unleash groundbreaking solutions by a small group of talented individuals, i.e., mini startups.

Watch our AWS re:Invent Recap Webinar Here

Also, be sure to download the free ebook we published with O'Reilly on AWS IaaS Solutions:

New call-to-action



Topics: AWS, Cloud

Subscribe Here!

Recent Posts

Posts by Tag

See all