Platypus

See our demo dashboard (wip)

(static version)

The problem

Infrastructures (networks and datacenters) consume about 56% of the final energy and emit about 36% of the Green House Gaz estimated for ICT [GreenIT, empreinte environnementale du numérique mondial, 2019]. Limiting their impacts is a key aspect to ensure greener digital services. Since all infrastructures (on-prem, VM, docker, k8s, public cloud…), relies on servers, we believe that addressing the physical machine layer is an opportunity to reach a wide range of use cases. 

Furthermore, we need a solution which can address on-premise infrastructure, where the optimizations have a much higher potential than in a cloud environment where important optimizations are easier to implement, thanks to the tooling and fine-tuned metrics a user can get from the cloud provider. 

Since in most of the time people cannot choose where to run their workloads and because we do not want our project to lead to overload electrical grids, to demand shifting - moving workloads to the electricity grid with the lowest carbon intensity, we preferred "demand shaping" - moving workloads in time, on the same electricity grid.

Indeed, demand shaping is a great opportunity, to support system adminsitrators and SREs in the difficult path of ICT environmental impacts reduction. Yet, it comes with a high risk of shifting the optimized operational emission (usage) to the embedded emissions (manufacture, transport, ...). Indeed, what if you always run most of your workload in a specific time frame ? You might need to buy new servers to absorb the pick, leading to an increase of the embedded impacts.

You want to see the importance of the embedded impacts ? Look by yourself !

Electrical carbon intensity is a required yet not a sufficient metric. The problem that must be solved is the following : how to optimize the time when your workloads run without having to build new machines ? In other terms, how to optimize both embedded and operational emissions ?

PLATYPUS: report, understand and reduce IT impacts

Platypus is a plug and play decision-making tool for anyone managing servers. It provides actionable environmental metrics and recommendations to reduce the environmental impacts of your infrastructure, through demand shaping. It relies on three pillars 

• Report impacts

  • Our agent collect inventory metrics from the server and match it with environmental metrics from Carbon Aware API and BoaviztAPI

  • The data can be explored in our dashboard or retrieved via API

• Understand impacts

  • Understand demand shaping opportunities

  • Understand link between embedded and operational emissions

• Reduce impacts

  • Give recommendations regarding the time you run your workloads (e.g. cron jobs)

  • Highlight the highest and lowest carbon-intensive time periods to identify demand shifting opportunities.

How Platypus uses the SDK

Platypus is built from several open-source microservices developed by the Boavizta collective. Carbon Aware API is a new game changing piece in our open source ecosystem. The API has been integrated into BoaviztAPI which provides data and methods for the evaluation and modeling of ICT hardware impacts. We rely on the Carbon Aware API to report in our dashboard both historical data (what should have been done) and forecast data (what could you do). The intensity picks are highlighted to visually show the best ranges where to run the workloads, while keeping in mind that embedded emissions shouldn’t be forgotten.

The project relies on several pieces : Boagent, BoaviztAPI, Scaphandre, and of course the Carbon Aware API, that enriches the data already present, which includes carbon intensity metrics, forecasting, allowing the solution to provide recommendations for workload management.

How impactful it can be

• The periodicity of scheduled workloads : cron jobs, batch jobs, rundeck, awx/ansible tower, ... is a potential for high reduction, as moving an endless serie of jobs in a less carbon intensive time window could significantly reduce yearly impacts.

• The types of workloads usually time-scheduled (database backups, system backups, encryption, decryption, data sanitization, etc.) are often high resources demanding workloads. Moving them in a less impactful period is a potential of high reduction.

• Evaluating embedded impacts and encouraging demand shaping, is a lever to make IT people take the embedded impacts into account (which is often overlooked). We think it is also a lever of low reach, but high reduction.

• Enabling patterns detection at the infrastructure level: enables SREs to feed product teams with environmental impact metrics and suggestions of optimization (timely scheduled workloads may be in the product scope), then regulated the impact of the product. This is a potential of high reduction and a higher reach.

• Working on any environment (cloud, on-prem, VM, k8s), from the infrastructures point of view let people install platypus in almost any environments allowing a high level of reach.

Opportunity to get production ready

Technically speaking, the project can be deployed easily on any GNU/Linux server, with docker-compose (we intend to enable the usage of other packaging solutions to fit more production-oriented requirements). Deployment is therefore plug and play which enables easier adoption.

All pieces of the project are open-source solutions that already have contributors and a community. Companies collaborating in the Boavizta working group will work on this project and improve it, as they do for the other open-source projects of the ecosystem.

With the potential price, we could free up time to finish an MVP. Our next step is to confront Platypus to real companies use cases and get feedbacks.

The development is allowed by the R&D time of companies engaged for a better understanding of ICT impacts (collaborating via the Boavizta collective). Benefitting for a direct return on investment with this project will help accelerate it's development and efforts to spread the initiative accross our networks of clients and partners.

As the parts of this project are solutions made for ICT environmental impact measurements, accuracy and of course transparency are key. The methodologies we rely on, especially for evaluation of embedded impacts, are developed and improved by the Boavizta working group, which has been working for about 2 years on a more open and viable evaluation of ICT impacts.

While the local dashboard is helpful to have an understanding of local resources consumption and impacts, those metrics can be gathered in a Time Series Database like Prometheus, or scraped as an API to be integrated in other tools. The /metrics and /query are sisters of the /web route so metrics can be helpful in many different contexts.

The vision

As metrics can be fetched in observability stacks, dashboards can be built (with grafana or other data visualization tools) at a service scale. Our dashboard is mostly a plug and play illustration of how to leverage those metrics to reduce operational and embedded impacts, thus help with adoption.

This project is the top of the iceberg of an ecosystem : open-source projects focussed on evaluating a part of ICT impacts, but doing it as accurately as possible. This way, we benefit from the most advanced automated evaluation of ICT environmental footprint we know : the Boavizta methodologies. Those methodologies (and tools) provide metrics on multi-criteria impacts based on Life Cycle Assessment criterias. To date, those impact criterias include : Global Warming Potential / GWP (GHG emissions), abiotic resources depletion / ADPe and Primary Energy usage / PE.

The Carbon Aware API enables us to evaluate GWP more dynamically than before and give advice to user to move workloads to better time. We would also like to improve the other impact criteria as well with more dynamic impact factors, which could be a common ground for work with the GSF and the Watttime team, if of interest.

The project has been developed with bare metal machines in mind, but it can be used in other contexts, like cloud instances.

URLs to the project

https://github.com/Boavizta/platypus

The content of our work can be seen in the platypus repository, and in the “greenhack22” branches in each of the dependencies/projects (boagent, boaviztapi and scaphandre).