Green Webhook

The Pitch

The Problem

Data centers consume a massive amount of electricity globally.

“On a global level, data centers consume approximately 200 terawatt-hours (TWh) of electricity, or nearly one percent of global electricity demand, while contributing to 0.3% of all global CO2 emissions, according to the International Energy Agency.” 1

“...data center energy usage in some countries could increase to levels of 15 to 30 percent of their total domestic electricity consumption by the end of the decade” ¹

Given how much electricity data centers consume how can we use the Carbon Aware SDK to reduce the emissions caused by this demand.

The Solution

Green Webhook is a proxy or gateway that creates a carbon aware webhook based on a list of endpoints that are hosted in different geographical locations.

The proxy dynamically decides the location with the lowest carbon intensity and invokes the endpoint for that location.

Note: Currently Azure regions are the only supported locations, however in future Green Webhook will be able to support any locations that the Carbon Aware SDK supports.

Example of a Green Webhook: End of Financial Year Summary

Consider a system that needs to generate an end of financial year PDF report based on a webhook event that fires when the report is requested.

It is a compute intensive process to generate the report so it is worth making the process carbon aware.

To do this the system that generates the report is deployed to three endpoints that run in three different geographical regions which have different carbon intensity characteristics.

The Green Webhook is used to dynamically determine and call the endpoint that has the least carbon intensity whenever the event to generate the report is triggered.

The Green Webhook software is given the following information:

1. https://europe.myendpoint.com [Azure Europe region]
2. https://asia.myendpoint.com [ Azure Asia region ]
3. https://us.myendpoint.com [ Azure US region ]

and generates a carbon aware webhook URI (e.g. https://greenwebhook.netlify.app/.netlify/functions/webhook/cla1358lr00040 ) which can be configured as the endpoint to be called when the financial year report event is triggered.

When the event triggers a HTTP request to Green Webhook it will use the Carbon Aware SDK to dynamically decide which Azure region has the lowest carbon intensity and forward the request to the corresponding endpoint.

Currently it enables the reduction of carbon emissions of Azure Cloud compute workloads and in future as the Carbon Aware SDK supports more locations it will enable reductions in many other areas such as personal devices (phones & laptops), smart home devices and IoT devices.

What did we achieve?

We built a working and fully deployed front end and backend web application for managing and using Green Webhooks that can location shift workloads between different Azure regions.

In addition we have nearly completed addion of time shifting support to enable scheduling of workloads based on a user provided maximum delay for a webhook request.

We also integrated the website with Gitbook to enable easy documentation and provided easy instructions for other developers to clone the code and run the project themselves locally.

Green Webhooks Impact

The Green Webhook project has the potential to result in massive reductions in carbon emissions.

IT professionals can easily use Green Webhooks to intelligently schedule very large workloads via a simple user interface.

Some examples of workloads that can be shifted are:

• Database backups
• Website scheduled jobs that use a globally distributed database
• Video and audio encoding 
• Machine learning training and inference
• Schedule reports, batch jobs
• Any asynchronous trigger that has a SLA of more than 5 minutes to complete

Because Green Webhook is a platform built on top of the Carbon Aware SDK the impact is only limited by what users build with the platform.

What do we need now?

The main things we need to do in order to take the Green Webhook project to production are:

1. Production database hosting (Postgres)
2. Production hosting of the Carbon Aware SDK and subscription to Watttime.org
3. A custom domain name
4. A security audit & secure code review
5. Upgrade of our Netlify hosting
6. Upgrade of our SendInBlue email hosting or other email provider
7. Implement Auth0 authentication or similar login service 
8. Improved audit logs and observability
9. Latitude & longitude support for by Carbon Aware API to unlock many use cases Azure cloud

Our Future Vision

Our vision for GreenWebhook is for it to grow into an "Intenet Grid Operator". A carbon aware webhook that location and time shifts the workload that is triggered by any HTTP request. By being the intelligent "internet grid operator" we can provide demand shaping that works in harmony with the electricity grid supply shaping in order to reduce the carbon emissions the grid causes.

Try out Green Webhook now!

https://greenwebhook.netlify.app

Github Project

https://github.com/jonparker/green-webhook

References

1. https://datacenterfrontier.com/data-center-management-solutions-reduce-costs-and-carbon-emissions