GreenCourier

GreenCourier: Towards sustainable Serverless Computing

Problem

Cloud computing has become a widely accepted and adopted paradigm for general-purpose computing. As a result, the migration rate from on-premise to the cloud for enterprise IT infrastructure and software has significantly increased over the past few years. One of the primary reasons for increased adoption is that the widely popular cloud service offerings such as Infrastructure-as-a-Service (IaaS), offer a pay-per-use model where the customers are only billed for the amount of resources that they rent. However, these offerings increase the burden on the developer to manage the infrastructure resources themselves. In most cases, the developers over-provision the required number of resources to handle sudden surges in service requests. This leads to significant resource underutilization and resource idling. According to a recent study, 50% of the energy used in data centers is by idle resources. To this end, an emerging cloud computing paradigm called "Serverless Computing" has the potential to reduce wasted energy consumption by idle computing resources.

Function-as-a-Service (FaaS), a key enabler of the serverless computing paradigm offers an attractive cloud model with several advantages such as no infrastructure management, scaling-to-zero for idle resources, a fine-grained pay-per-use billing policy, and rapid automatic scaling on a burst of service requests. As a result, it has gained significant popularity and widespread adoption in various application domains such as machine learning, federated learning, edge computing, and scientific computing. In FaaS, applications are developed as small pieces of code called functions that are executed in response to event triggers such as HTTP requests. According to recent estimates, the market for serverless computing is going to increase from USD 13.47 Billion in 2021 to USD 86.94 Billion by 2030. 

All major cloud service providers such as Google, Amazon, and Microsoft have their own commercial FaaS offerings such as Google Cloud Functions, Google Cloud Run, AWS Lambda, and Azure Functions. Typically, these cloud service providers receive around 1.1 billion function invocations each day. These functions are executed in the internal geographically distributed data centers of the cloud providers which are forecasted to consume between 8% to 13% of the world's total electricity by 2030. However, none of the current commercial FaaS offerings consider the carbon-efficiency of the different geographical regions for executing the functions.

As the first step towards sustainable serverless computing, we propose and implement GreenCourier. GreenCourier incorporates an intelligent scheduling policy that delivers serverless functions in different geographical regions depending on the availability of renewable resources at any given point in time. GreenCourier builds on Kubernetes, the de-facto platform for container orchestration in the cloud, and Knative, an enterprise-level platform for building serveless applications. 

Solution

                                                       Figure 1. The different system components of GreenCourier.

To reduce carbon emissions on function invocations, we implement a scheduling plugin for Kubernetes. Kubernetes is an open-source system for automating the deployment, scaling, and management of containerized applications.  For placing functions within a cluster, Knative uses the default scheduling strategy provided by Kubernetes. In this work, we implement a scheduling policy based on marginal carbon emission data received from the carbon-aware sdk to make carbon-aware scheduling decisions across geographically interconnected Kubernetes clusters.

One of the major reasons for using Knative for GreenCourier was that it is currently used in production by Google for their serverless offerings, i.e., Google Cloud Functions and Google Cloud Run. Moreover, Kubernetes is offered as a service by all major cloud providers such as Google Kubernetes Engine (GKE), Elastic Kubernetes Service (EKS), and Azure Kubernetes Service (AKS). We use Liqo to establish seamless Kubernetes multi-cluster topologies, supporting heterogeneous on-premise, cloud and edge infrastructures.  Liqo supports features such as - peering, offloading, Network Fabric, and Storage Fabric. For interconnecting different Kubernetes clusters, Liqo uses Virtual Kubelet. When a cluster establishes a peering, a new instance of the Liqo Virtual Kubelet is spawned to seamlessly extend the capacity of the cluster, by providing an abstraction of the resources of the remote cluster. The provider combined with the Liqo network fabric extends the cluster networking by enabling Pod-to-Pod traffic and multi-cluster east-west services, supporting endpoints on both clusters.  

For selecting the most carbon-efficient region to schedule the function, GreenCourier utilizes the Carbon Aware SDK. We aggregate data from the carbon SDK using a metrics collector and provide periodic updates to the Kubernetes plugin. Our metrics collector is designed with modularity in mind and can be easily extended to support different data sources. On function invocation, Knative creates Kubernetes objects in the management cluster (Figure 1). GreenCourier listens to the creation of such objects and automatically schedules them on the most carbon-efficient geographical region. All decisions for placing functions are taken at run-time by GreenCourier.

GreenCourier has several advantages:

• Production-ready tech stack: GreenCourier's tech stack includes Kubernetes, Knative, and Liqo. All of these technologies have been adopted by numerous companies and are well-established projects.  So, the time to production is almost close to zero.
• Vendor agnostic:  GreenCourier can be deployed in any of the different Kubernetes services offered by the different cloud providers or in on-premise Kubernetes clusters which use Knative as the FaaS platform. Since GreenCourier was designed as a plugin, it is extremely easy to deploy and use.
• Support for multiple programming languages: Since GreenCourier builds on Knative, users can write their functions in several programming languages such as C++, C#, Go, Java (Spark), Java (Spring), Kotlin, Node.js, PHP, Python, Ruby, Scala, and Shell.

From the user's perspective, using GreenCourier is extremely simple. The user only needs to implement a function supported by Knative and deploy it on our management cluster. On function invocation, all tasks such as resource provisioning, scheduling, and autoscaling are managed automatically.

What we have achieved during the hackathon

We have developed and tested a minimum viable product (MVP) that schedules functions in low-carbon regions on invocation.  To evaluate the performance of GreenCouier, we deployed three geographically distributed clusters as shown in Figure 1. Following this, we deployed different types of serverless functions on our management cluster and performed load tests using k6 - an open-source regression testing tool by Grafana Labs. All experiments are performed on Google Kubernetes Engine (GKE) with Knative. The results of our evaluation are publicly available in our Carbon Hack 22 repository - Test Result. 

Impact

To accommodate the varied nature of serverless functions in our load testing, we chose five different functions to represent different types of workloads. These functions are as follows: (I) inferencing a convolutional neural network (CNN Serving), (ii) performing several operations on an image (Image Processing), (iii) inferencing a recurrent neural network (RNN Serving), (iv) inferencing a logistic regression model (LR serving), and (v) processing a video (Video Processing). More details about these functions are readily accessible in our Green Hack 22 repository - Functions.  The load testing scripts are also available for public view - Test Scripts.

On performing load tests with GreenCourier across the different functions, we observed the enhanced carbon-aware decisions taken by our intelligent scheduling policy in function placement. To this end, we observed a 60% improvement in function placement as compared to Knative for placing functions in the cluster deployed in the most carbon-efficient region as shown in Figure 2.

.  

                              Figure 2. Comparing the efficiency of GreenCourier for placing functions with Knative.

Similary, the placement of functions in carbon-efficient regions should reduce the carbon footprint for those functions being executed.  Towards this, we observed a 40% reduction in carbon emissions for the different functions with GreenCourier as compared to Knative as shown in Figure 3.

                                             Figure 3. Comparing the carbon efficiency of GreenCourier with Knative.

With the increasing adoption of the serverless computing paradigm and an estimated increase in the market size of 650% by 2030, our solution will have high reach in reducing carbon emissions. Moreover, a reduction of 40% in carbon emissions as shown in Figure 3 classifies our approach as a high-reduction strategy. In addition, we expect the carbon reduction to increase when we have an increased number of geographically distributed clusters.

Feasibility and Extension

GreenCourier is a ready-to-use, vendor agnostic software, which can be easily deployed with a single command using Helm - a package manager for Kubernetes.  Currently, this can be done by downloading the code and running the following command:

$ helm install GreenCourier carbon-scheduler/charts/

Since GreenCourier is designed to co-exist in presence with the default Kubernetes Scheduler, we should include schedulerName: kube-carbon-scheduler in the function specification when deploying the function using Knative.  

GreenCourier is implemented using the well-defined scheduling plugin API provided by Kubernetes. The usage of Scheduler API in GreenCourier, which is a mature Kubernetes feature makes it stable and ready for production.  Kubernetes and Knative communities are well-known for their documentation, thus making is easy for anyone to adopt and extend the GreenCourier’s codebase.

Vision

Going forward we would like to improve GreenCourier in the following aspects thus enabling widespread adoption. An estimated timeline for the different objectives is shown in Figure 4.

                                                                                    Figure 4. Extending GreenCourier.

• Disaggregated Control Plane: Currently in our solution, our management cluster is a single point of failure.  If the management cluster is down, it won't be possible to deploy or invoke functions to the geographically distributed target clusters.  Towards this, we plan to implement a HA (Highly Available) mechanism to tackle this centralized approach.
• Add support for new data sources: In GreenCourier, the metrics-collector interacts with the carbon-aware SDK to collect, aggregate, and normalize carbon efficient scores for the different clusters.  Currently, it supports the carbon-aware SDK and WattTime as data sources.  We plan to enhance the metrics-collector to support more data sources such as Electricity Maps and aggregate the data from the configured sources.
• Scheduling for stateful functions: GreenCourier currently supports East-to-West communication between the different workloads deployed in non-identical Kubernetes clusters.  However, if a function is dependent on locally present data/volume, but moved to a different cluster for execution it will fail in our current implementation.  Towards this, we plan to extend support for functions that have a dependency on local data present in a particular cluster by extending our solution to create a volume fabric that will send dependent data to the target cluster where the function is scheduled to be executed.
• Carbon-aware Serverless Federated Learning: We are working on the development of a framework called FedLess, that leverages serverless computing technologies to enable Federated Learning across a fabric of heterogeneous devices. To this end, we plan to integrate GreenCourier into FedLess to enable carbon-efficient collaborative training of DNN models using Federated Learning.

It is important for us to bring in new adopters or developers into our community by promoting our solution in tech forums like KubeCon, ServerlessCon, KnativeCon, and other CNCF hosted events.  It will be possible with the help of connections and mentors from the software foundations like GSF and CNCF. Any monetary recognition will surely help us nudge towards attaining our goals and shape the rapidly growing field of sustainability in Serverless Computing and make it possible to advertise and market our solution to the world which we would like to be a better, more habitable, and sustainable place for all to live.

References

1. Immersion cooling heats up - Greengard, Samuel
2. Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider
3. How much energy do data centers use?
4. Global Serverless Architecture Market To Reach USD 86.94 Billion By 2030

Resources and Links

Repository: GitHub

Live Deployment: Link (carbonhack/123456). Usage.

Video Pitch: YouTube