The Magic Of Autoscaling With Serverless

One of the advantages of Serverless touted by cloud providers is seamless autoscaling. Ever since cloud computing gained traction, autoscaling is seen as the biggest advantage organizations can leverage for unexpected traffic. Autoscaling was seen as a major leap from traditional IT where capacity planning is critical and resource inefficiencies (and hence increased costs) are the norm. With the success of web applications and the unpredictability in traffic patterns, traditional approaches to scaling servers were not working. In this post, we will talk about how autoscaling works in the cloud and the advantage of Serverless in scaling up and down to meet the demands.

Autoscaling in Virtual Machines, Kubernetes and Serverless

With Cloud Computing and programmatic access to infrastructure, autoscaling became the norm. Users need not worry about whether their applications can meet a sudden spike in demand and the underlying infrastructure can be scaled up the meet the demand without any manual intervention. Once the demand cools down, the resources can be automatically brought down. In this case, the resources can be used efficiently with the associated cost savings compared to traditional approaches to scaling resources to meet demand.

The autoscaling in the cloud comes in different flavors depending on what type of cloud service you are using:

  • Unlike traditional IT, where servers should be procured in advance to meet the scaling needs, the cloud gives programmatic access to scaling up just as the demand spikes and scaling down immediately after the demand goes down. With virtual machines on the cloud, users can use an autoscaling service that will automatically add additional virtual machines to meet the demand by taking into account CPU, memory, and network usage. While the autoscaling services can seamlessly bring up virtual machines and route traffic, it adds up significant operational overhead for developers to ensure that applications and the dependencies scale well in a scale-out architecture. Plus, virtual machines take a few minutes to boot up, and scaling is not instantaneous and some minimal capacity planning is critical for high performance. Plus, autoscaling with virtual machines might lead to resource inefficiencies if it is not managed well
  • With containers gaining traction and Kubernetes becoming the de facto container orchestration tool, scaling became much more seamless because Kubernetes uses a declarative model and developers can simply define their end state in a YAML file. Kubernetes will take care of autoscaling, drastically reducing the operational overhead on developers. However, if Kubernetes is deployed on virtual machines in the cloud (say, on top of Amazon EC2), there is still an operational overhead in scaling the underlying nodes. Services like AWS Fargate take away these operational complexities of managing the virtual machines but developers are faced with YAML complexity to ensure that the Kubernetes environment meets the scaling needs of the application
  • Serverless computing, especially the hosted offerings, takes the pain out of autoscaling and makes it seamless for developers to scale their application to meet the demand. Serverless offerings like AWS Lambda, Azure Functions, Catalyst, and others scale the infrastructure to meet the demand without any operational overhead or YAML complexity. Developers can just focus on the business logic and code and the Serverless compute offering will seamlessly handle the scaling needs. Since the compute costs are calculated based on invocations and it shuts down automatically after execution, autoscaling is so easy that anyone with zero operational knowledge can handle it. Moreover, the resource usage in the case of Serverless is more fine-grained and, therefore, the cost savings are much better without any resource waste

Autoscaling Patterns in Serverless

With Serverless compute, also known as Functions as a Service, there are some distinct autoscaling patterns that can be used to meet the demand. You could either invoke the function for every request and scale based on the requests. With certain Serverless offerings like AWS Lambda, it meets the cold start problem and you need to use a warm pool to avoid the delay due to cold start. But, keeping a warm pool costs more money in the case of hyperscale providers like AWS. The next-gen Serverless platforms like Catalyst, Nimbella, IBM Functions, and others ensure resource optimization in the backend to avoid the delays due to cold start.

Another approach to scaling with AWS Lambda and other Serverless offerings with the cold start problem is to allow a function invocation to handle multiple requests per function invocation. While this solves the colds start problem and even save some money in the invocation costs, it doesn’t completely avoid the cold start. Plus, it adds additional overhead on how applications are architected to follow this complex invocation pattern.

Serverless offers the most efficient (both in terms of resources and cost) way to autoscale to meet the demands. While using services like AWS, it is important for the developers to understand some of the constraints such as cold start and the complexity associated with concurrency. There are other platforms available that eliminate the cold start problem and give you a more straightforward way to autoscale your applications.

Serverless Vs Containers – The Differences

Serverless technologies like AWS Lambda functions first offered in late 2014 is changing how developers deploy their applications. Developers saw the convenience and started using AWS Lambda for some basic use cases. As the technology matured, we are seeing more and more use cases for Functions as Service offerings like AWS Lambda, Zoho Catalyst, Azure Functions, etc.. With large scale adoption, serverless is becoming mainstream with increased enterprise adoption because it offers three main advantages to an organization: increased agility, seamless scalability and availability, and infrastructure cost savings. The key question is how does containers stack up against serverless in terms of these advantages.

  • Increased Agility: In current software development model, developers should consider where the code is deployed and create the necessary configuration file and other artifacts to support the deployment process. For example, developers writing a microservice to be deployed in a Kubernetes environment should create the necessary Kubernetes artifacts to support the deployment. They also have to understand Kubernetes concepts, as well as figure out how to run the microservice and artifacts on their development machine to test them before pushing it to the DevOps pipeline. By contrast, with Functions as a Service like AWS Lambda or Zoho Catalyst, developers only need to worry only about the business logic they have to code, which could even follow a certain template, and then just upload the code to the serverless platform. This speeds up things significantly and helps organizations to ship products and solutions quickly and rapidly make changes based on feedback. Agility without the unnecessary operational overhead for developers is the biggest advantage for serverless
  • Scalability and Availability: With traditional approaches, developers and DevOps teams have to factor in the scalability and availability for the application they build it is deployed. It typically includes the peak load expectations, whether an auto-scaling is needed or a static deployment will suffice, setting up the necessary automation for auto-scaling and requirements for ensuring the required availability. This is the case for applications deployed on virtual machines as well as containers. Serverless platforms remove the need to worry about these factors, since they are capable of executing the code when needed, and ensuring availability. The scalability is easily achieved by invoking the functions parallely to meet the load. It is the responsibility of the cloud provider to handle this in a seamless way. So, scalability and availability are provided by default and handled by the cloud provider without any overhead for the developers
  • Infrastructure Cost Savings: When we deploy an application in a production environment, we generally need to keep it running 24×7 since in most cases it is hard to predict the specific time periods the app needs to be available. That means there are many idle times between requests even though we pay for the infrastructure full time. For an organization that has to deploy hundreds of services the costs can escalate quickly. With serverless functions, it is guaranteed that your code will get computing resources allocated only when it is executed, which means you pay exactly for what is being used. This can cut costs significantly. However, this also constraints users from running only certain types of use cases

How is serverless computing different from containers?

Containers are essentially what the name describes: a comprehensive software encapsulation that gets delivered and used as a standalone application environment. The most common form of this is how applications get distributed at runtime. Everything and anything that’s needed to run and interact with a piece of software gets included or packaged together. That often entails bundling the software code, runtime and system tools, software and foundational libraries and default settings. In the case of containers – through platforms like Docker – they help solve the more common problems that arise from cross-platform use.

When moving from one computing environment to another, developers often run into obstacles. If the supporting software is not identical, it can cause a series of hiccups. And since it’s not uncommon for developers to move from a dev environment to a test environment, or even from staging into production, this becomes a widespread issue. That’s just concerning the software itself, as other issues can appear from network topologies, security and privacy policies, as well as varying tools or technologies. Containers solve this by wrapping everything up nicely into a runtime environment. Even though containers offered this advantage of encapsulating entire environments and removing the friction due to different environments on DevOps pipeline, they came with operational overhead which became a burden for developers.

In the case of Serverless, all these tasks are completely abstracted away by the cloud service provider, giving developers a simple interface to deploy their code. Developers need not bother about the application dependencies, environments, etc.. They just need to encapsulate the business logic into code and this can then be deployed onto serverless platform. Developers need not worry about the deployment environment and since the same service will be used for both development and production, they are not faced with the issues of making the code work in different environments. The entire heavy lifting is done by the cloud provider and the developers focus on the innovation.