Cloud computing has become a widely accepted and adopted paradigm for general-purpose computing. As a result, the migration rate from on-premise to the cloud for enterprise IT infrastructure and software has significantly increased over the past few years. One of the primary reasons for increased adoption is that the widely popular cloud service offerings such as Infrastructure-as-a-Service (IaaS), offer a pay-per-use model where the customers are only billed for the amount of resources that they rent. However, these offerings increase the burden on the developer to manage the infrastructure resources themselves. In most cases, the developers over-provision the required number of resources to handle sudden surges in service requests. This leads to significant resource underutilization and resource idling. According to a recent study, 50% of the energy used in data centers is by idle resources. To this end, an emerging cloud computing paradigm called "Serverless Computing" has the potential to reduce wasted energy consumption by idle computing resources.
Function-as-a-Service (FaaS), a key enabler of the serverless computing paradigm offers an attractive cloud model with several advantages such as no infrastructure management, scaling-to-zero for idle resources, a fine-grained pay-per-use billing policy, and rapid automatic scaling on a burst of service requests. As a result, it has gained significant popularity and widespread adoption in various application domains such as machine learning, federated learning, edge computing, and scientific computing. In FaaS, applications are developed as small pieces of code called functions that are executed in response to event triggers such as HTTP requests. According to recent estimates, the market for serverless computing is going to increase from USD 13.47 Billion in 2021 to USD 86.94 Billion by 2030.
All major cloud service providers such as Google, Amazon, and Microsoft have their own commercial FaaS offerings such as Google Cloud Functions, Google Cloud Run, AWS Lambda, and Azure Functions. Typically, these cloud service providers receive around 1.1 billion function invocations each day. These functions are executed in the internal geographically distributed data centers of the cloud providers which are forecasted to consume between 8% to 13% of the world's total electricity by 2030. However, none of the current commercial FaaS offerings consider the carbon-efficiency of the different geographical regions for executing the functions.
As the first step towards sustainable serverless computing, we propose and implement GreenCourier. GreenCourier incorporates an intelligent scheduling policy that delivers serverless functions in different geographical regions depending on the availability of renewable resources at any given point in time. GreenCourier builds on Kubernetes, the de-facto platform for container orchestration in the cloud, and Knative, an enterprise-level platform for building serveless applications.
Figure 1. The different system components of GreenCourier.
To reduce carbon emissions on function invocations, we implement a scheduling plugin for Kubernetes. Kubernetes is an open-source system for automating the deployment, scaling, and management of containerized applications. For placing functions within a cluster, Knative uses the default scheduling strategy provided by Kubernetes. In this work, we implement a scheduling policy based on marginal carbon emission data received from the carbon-aware sdk to make carbon-aware scheduling decisions across geographically interconnected Kubernetes clusters.
One of the major reasons for using Knative for GreenCourier was that it is currently used in production by Google for their serverless offerings, i.e., Google Cloud Functions and Google Cloud Run. Moreover, Kubernetes is offered as a service by all major cloud providers such as Google Kubernetes Engine (GKE), Elastic Kubernetes Service (EKS), and Azure Kubernetes Service (AKS). We use Liqo to establish seamless Kubernetes multi-cluster topologies, supporting heterogeneous on-premise, cloud and edge infrastructures. Liqo supports features such as - peering, offloading, Network Fabric, and Storage Fabric. For interconnecting different Kubernetes clusters, Liqo uses Virtual Kubelet. When a cluster establishes a peering, a new instance of the Liqo Virtual Kubelet is spawned to seamlessly extend the capacity of the cluster, by providing an abstraction of the resources of the remote cluster. The provider combined with the Liqo network fabric extends the cluster networking by enabling Pod-to-Pod traffic and multi-cluster east-west services, supporting endpoints on both clusters.
For selecting the most carbon-efficient region to schedule the function, GreenCourier utilizes the Carbon Aware SDK. We aggregate data from the carbon SDK using a metrics collector and provide periodic updates to the Kubernetes plugin. Our metrics collector is designed with modularity in mind and can be easily extended to support different data sources. On function invocation, Knative creates Kubernetes objects in the management cluster (Figure 1). GreenCourier listens to the creation of such objects and automatically schedules them on the most carbon-efficient geographical region. All decisions for placing functions are taken at run-time by GreenCourier.
GreenCourier has several advantages:
From the user's perspective, using GreenCourier is extremely simple. The user only needs to implement a function supported by Knative and deploy it on our management cluster. On function invocation, all tasks such as resource provisioning, scheduling, and autoscaling are managed automatically.
We have developed and tested a minimum viable product (MVP) that schedules functions in low-carbon regions on invocation. To evaluate the performance of GreenCouier, we deployed three geographically distributed clusters as shown in Figure 1. Following this, we deployed different types of serverless functions on our management cluster and performed load tests using k6 - an open-source regression testing tool by Grafana Labs. All experiments are performed on Google Kubernetes Engine (GKE) with Knative. The results of our evaluation are publicly available in our Carbon Hack 22 repository - Test Result.
To accommodate the varied nature of serverless functions in our load testing, we chose five different functions to represent different types of workloads. These functions are as follows: (I) inferencing a convolutional neural network (CNN Serving), (ii) performing several operations on an image (Image Processing), (iii) inferencing a recurrent neural network (RNN Serving), (iv) inferencing a logistic regression model (LR serving), and (v) processing a video (Video Processing). More details about these functions are readily accessible in our Green Hack 22 repository - Functions. The load testing scripts are also available for public view - Test Scripts.
On performing load tests with GreenCourier across the different functions, we observed the enhanced carbon-aware decisions taken by our intelligent scheduling policy in function placement. To this end, we observed a 60% improvement in function placement as compared to Knative for placing functions in the cluster deployed in the most carbon-efficient region as shown in Figure 2.
Figure 2. Comparing the efficiency of GreenCourier for placing functions with Knative.
Similary, the placement of functions in carbon-efficient regions should reduce the carbon footprint for those functions being executed. Towards this, we observed a 40% reduction in carbon emissions for the different functions with GreenCourier as compared to Knative as shown in Figure 3.
Figure 3. Comparing the carbon efficiency of GreenCourier with Knative.
With the increasing adoption of the serverless computing paradigm and an estimated increase in the market size of 650% by 2030, our solution will have high reach in reducing carbon emissions. Moreover, a reduction of 40% in carbon emissions as shown in Figure 3 classifies our approach as a high-reduction strategy. In addition, we expect the carbon reduction to increase when we have an increased number of geographically distributed clusters.
GreenCourier is a ready-to-use, vendor agnostic software, which can be easily deployed with a single command using Helm - a package manager for Kubernetes. Currently, this can be done by downloading the code and running the following command:
$ helm install GreenCourier carbon-scheduler/charts/
Since GreenCourier is designed to co-exist in presence with the default Kubernetes Scheduler, we should include schedulerName: kube-carbon-scheduler in the function specification when deploying the function using Knative.
GreenCourier is implemented using the well-defined scheduling plugin API provided by Kubernetes. The usage of Scheduler API in GreenCourier, which is a mature Kubernetes feature makes it stable and ready for production. Kubernetes and Knative communities are well-known for their documentation, thus making is easy for anyone to adopt and extend the GreenCourier’s codebase.
Going forward we would like to improve GreenCourier in the following aspects thus enabling widespread adoption. An estimated timeline for the different objectives is shown in Figure 4.
Figure 4. Extending GreenCourier.
It is important for us to bring in new adopters or developers into our community by promoting our solution in tech forums like KubeCon, ServerlessCon, KnativeCon, and other CNCF hosted events. It will be possible with the help of connections and mentors from the software foundations like GSF and CNCF. Any monetary recognition will surely help us nudge towards attaining our goals and shape the rapidly growing field of sustainability in Serverless Computing and make it possible to advertise and market our solution to the world which we would like to be a better, more habitable, and sustainable place for all to live.
Video Pitch: YouTube