Circa - carbon nice scripting

Circa

ca., abbreviation for circa (latin), meaning approximately

1. The Problem

Inspiration for circa came from thinking about the question: what's the simplest thing that just might work?

It would be great to have a task scheduler that was carbon aware, as there are very many daily computing jobs that are not time sensitive.

There are millions of jobs running on computers at exactly midnight or 6am that use up electricity and could easily be time-shifted to run when the energy supply was cleaner.

However, replacing a task scheduler feels like a tricky business. It's a core bit of server infrastructure; there are trust and security issues.

You'd also need to change the schedules to say when a task needs to be completed and, ideally, give it information about how long it might take to run. Making this backwards compatible could get complicated too.

Hmmm.

Let's take a step back.  We want something that:

• can be easily and incrementally adopted - no need to replace everything
• has minimal overhead - familiar set up and easy to use
• can be built upon - is easy to experiment with, but a reliable foundation.

Okay -- so, what about trying to create something that's sort of unix-y?  A little command like nice that follows the philosophy of:

Write programs that do one thing and do it well.  Write programs to work together.

Circa is an attempt to do this.

Its goal is to be the smallest, friendliest step that anyone managing a server could feasibly take down the road to carbon aware, time-shiftable computing.

2. The Solution

Circa -- or ca for short -- is a smol command that waits until the electricity supply is slightly cleaner before continuing.

Easy to use

You can use it like this:

ca 6h my-cpu-intensive-task.sh

Which will find the best half-hour (with the lowest carbon footprint) in the next 6 hours to run the task in.

You can also use it in your daily task scheduler (e.g. cron) like this:

0 0 * * * /usr/local/bin/ca -d 20m 6h /path/to/my/daily-report-script.sh

Which will find the best 20 minutes between midnight and 6am to run your report in.

Easy to set up

Circa is very lightweight, written in C and can be installed from a release using a familiar (to systems administrators and devops at least) process of configure and make install commands.

Configuration of the carbon aware location is via a simple per-machine or per-user text file:

# Location to request carbon intensity data for
location uksouth

Although a different location can also be specified for each script (e.g. if it is accessing a server based in a different location).

It runs on Unix servers, macOS and even on Windows, using WSL.

Easy to build upon

While circa was intended just for running CPU intensive scripts at a better time, thanks to its ease of use, you can even use it to help you pick when to charge your laptop:

ca -d 2h 8h osascript -e 'display notification "Why not plug in your charger?" with title "Lower Carbon Energy Available"'

This will pick the best 2 hours in an 8 hour working day to prompt you to charge your laptop.

3. Carbon Aware SDK

Circa is compatible with both the web API and the command line interface (CLI) versions of the Carbon Aware SDK.

You can configure it to use a local, or any remote, installation of the API:

# Url to API endpoint, without trailing slash
url https://carbon-aware-api.azurewebsites.net

Or point it to the new, redesigned CLI:

# OR full path to CLI executable
url /home/circa/carbon-aware-sdk-162-cli-redesign/src/CarbonAware.CLI/src/bin/Release/net6.0/publish/caw 

In both cases it makes use of the emissions forecasts to pick out the optimal time in the timeframe specified.

By offering the user the choice of API or CLI, both in configuration, and on the command line - circa provides a very lightweight and friendly, but surprisingly powerful, way to make use of the Carbon Aware SDK's features.

4. Impact

Circa's designed to be super-easy to pick up as a systems administrator or devops professional.  It's therefore hoping to be (or inspire other similar solutions to be) widely adoptable - changing the question from 'why should we invest time in this' to 'why aren't we doing this?'

While the individual carbon savings from time-shifting a single update or report on a computer or server are pretty small - the gain from making many tens of thousands of these computations shiftable could be huge.  Especially on a weekly, or daily basis.

And these are just the low-hanging fruit of tasks that have just been scheduled to run at midnight or 6am by default. By making it the concept of carbon awareness easily accessible and pluggable into any sort of script, it is possible that even bigger impacts may follow.

5. Feasibility

Since circa is so smol, it's pretty much ready to go already.

There is a version 0.2 release that can be installed and used on linux, mac or windows.

It's designed to fail-safe - if the Carbon Aware data is unavailable for any reason, the command will still be run, making it very low-risk to start using in production.

To make it even more straightforward to adopt, some additional work to allow it to be installed from standard package managers (e.g. a linux .deb or .rpm or a brew keg) would probably help it be adopted even more easily.  And because it's written in vanilla C, uses common libraries and uses a standard build process this shouldn't be too complicated to do.

With some extra work, it might also be desirable to allow it to control programs that have already started running (a bit like the renice command).  This ability would allow it to pause and restart processes based on the current carbon intensity of the electricity supply.

Another future feature might be to collect statistics on the usual load and runtime of each task.  This would allow it to 'learn' using some fuzzy logic, the best window of time needed to run each one.

6. Vision

Although circa is purposefully designed to be tightly focused on doing one thing well -- waiting for the lowest carbon energy to be available in a given timeframe -- it has bigger ambitions.

In particular, by providing an incrementally adoptable approach, allowing folk to add carbon awareness to their systems in phases, it aims to provide a commonly understood way of describing how time sensitive any particular task is.

This could then easily be built upon by other software.  For example, by recognising the circa command in a task schedule (e.g. crontab) a task scheduler could be developed that used that extra information to schedule several tasks even more intelligently to optimise the overall energy use of a server - or even a datacenter.

And by being lightweight, circa can also be used in Internet of Things (IoT) setups - enabling users to bring carbon awareness to other devices in houses and businesses.

But mainly, circa aims to be familiar and friendly to folk building servers and writing scripts.  The easiest of first steps to take toward low-carbon computing.

7. More Information

Find Circa on Github at https://github.com/woodcoder/circa .

8. Video

See Circa in action on YouTube at https://youtu.be/tvJco5iEVgo . 

Image credit: adapted from It's a small world... by HarshLight CC BY 2.0