KarbonWeaver

>> 1. The Big Problems that Karbonweaver Addresses

In the fight against climate change, global tech giants have united to reduce the carbon emissions that result from data processing. To this end, the GSF Carbon-Aware SDK was created, enabling developers to adjust the timing of their code to run during cleaner periods of the day. 

Now this technology exists, it's our responsibility to maximize its impact. However, there are several hurdles that will impede the widespread adoption of the carbon-aware SDK as it is:

1. Time is money. Under business demands and pressure from higher-ups, developers will favor performance, implementation time, and execution time over being green. 
   • Developers on strict deadlines won't be willing to put time into implementing this SDK and put additional time into testing.
   • Most developers in a business setting will also be reluctant to sacrifice hours of time every time they run their program.
2. If this SDK is implemented in regions where emissions ratings are poor year-round, the impact of waiting for the local grid to go greener will be negligible. To make this viable, we need a way to maximize how many emissions are saved. 

Our project is a novel solution that addresses all three of these problems. 

>> 2. An Overview of how Karbonweaver Works

We have laid the foundations for a marketplace that connects developers with servers around the world. Instead of waiting several hours for their local grid to go green, developers can send their program to a server on a greener grid, and have it begin processing immediately. 

Our solution removes all three of the hurdles listed above: 

• Our tool acts as a black box. The developer doesn't need to write code or worry about the behind-the-scenes magic; they're shown a list of hosts that meets their needs, and they can have their program set to run with a few clicks.
• We eliminate the wait-time needed for execution by connecting servers across the world. Our tool runs programs on servers where the servers' electric grid is the greenest. 
• We maximize carbon emissions reductions on the KarbonWeaver network. Running on a dirty grid at noon vs. running on the same dirty grid at night is a small difference. Running your code on a dirty grid at noon vs. running your code on a clean grid at night? That's a big impact, both in carbon emissions and in the savings on your electric bill.

By adapting the GSF API into a marketplace framework that's fast, low-to-no-cost, and easy-to-use, we've invented the next step in letting the world adopt greener software development practices. 

>> 3. How does Karbonweaver use the GSF’s Open-Source Carbon-Aware SDK?

Our tool uses the GSF's Carbon-Aware API to get a snapshot of carbon emissions ratings for all clients (the developers) and hosts (servers) registered on our website. 

We then analyze this data to pair the clients on the dirtiest grids with the cleanest grids on the server; this approach maximizes the reductions in emissions.

Clients are also given various other options to filter where they run their program: host hardware statistics, host location, host electrical costs, and host emissions ratings.  

>> 4. What is the Scale of Karbonweaver’s Impact on Global Carbon Emissions?

The principal demographic for our project ranges from large businesses that crunch analytics data all day, to academic institutions that conduct machine learning research. Our demographic also includes developers and researchers with limited access to powerful hardware. 

These two demographics use processing power on different scales:

Businesses and universities will be much more energy intensive. While they will naturally be fewer in number, they perform much more rigorous work that leaves a larger carbon footprint. Given the volume of work they do, our tool can provide moderate to large reductions of carbon emissions in this first market segment. 

On the other side of our demographics, we have hobbyists, rural educators, and independent developers. While the carbon reductions per request in this demographic may not be as large individually, they are far greater in number. Collectively, our tool provides a low-to-moderate reduction of carbon emissions in this second market segment. 

>> 5. Karbonweaver is Easy to Implement in Today’s Fast-Paced Business Environments (and Notes for the Future)

We believe our project is the next evolution of the GSF's mission statement: Building a trusted ecosystem of people, standards, tooling, and best practices for a greener future. 

Our project brings all of these aspects together.

It has a low barrier to entry (since clients save money on not consuming electric from their own grid, clients only need to cover the electricity that the host server consumes). 

One thing that differentiates the KarbonWeaver project from others is that we lay the framework to make things economically stable for both developers and hosts. 

For this aspect of our solution, we drew inspiration from energy distribution markets in the US- an economic model that has been well-proven over the past decades to be stable and efficient.

We need developers to have access to host servers to maximize emissions reductions, but why would a host want to pay the electric bill for clients' work? Our marketplace solves this by compensating hosts for the electric that they consume. 

We initially thought that measuring the energy consumption of running a program would be theoretically impossible due to the halting problem- You can't tell if a program will end, so you can't tell how long it will run nor how much energy it will consume. 

This is the second factor that differentiates our project from the competition: We overcame this technical aspect of the problem in a novel way by calculating power consumption on the fly using a custom PowerShell script.

As we continue developing the KarbonWeaver project, there are several next steps that we want to take before bringing this to market:

The first step is revisiting our power measurements. So far, we have used a PowerShell Script and an open-source hardware monitoring tool to assess the power usage of a single program running on the host device. Our power readings are based solely on the CPU's package readings, and this worked perfectly as a proof of concept! However, it does not yet account for power draw from GPU usage or storage read/writes, and that is important for use cases like machine learning which make ample use of CUDA cores on graphics cards. Luckily, our hardware monitor also captures information for GPUs (integrated graphics are already counted in the CPU metrics). This would be a simple addition. 

The second improvement we'd like to make is in our integration of our power monitoring script and our website. We have discussed different approaches to this: we think the best solution is to create a local program that tracks the total power usage (from the hardware monitor) and partial power usage of the program (from our script), and then send both metrics to our website for computation.

Additionally, we'd like to expand our database to list local electric prices, emissions ratings, and location as a way for developers and businesses to make informed decisions about where they send their data and how much money they will save. It would also be useful to create filters and sorting columns for each metric, so clients can choose the greenest solution that suits their business needs.

We have also discussed the possibility of implementing a "priority bid" system that lets clients pay a premium for access to a certain server, or to send their program to the front of a server's queue so it finishes first. This will be for clients who want to minimize their execution time even further; this follows the same model as energy distribution markets in the US (power plants bid prices to the local distributor, and the distributor takes power from the most favorably priced plants first. Similarly here, host machines on the KarbonWeaver platform will take requests from higher bidding clients first.) 

Additionally, we're looking forward to polishing the host side of our app. For the sake of our prototype (and under limited time constraints), we decided it was best to focus on the implementation of what the client would see. Because of this, the host side of the website needs to be finished, and we need to build out the registration pages as well.

We developed this project on Windows 10 laptops, and we want to expand our software to work with newer and older hardware and operating systems.

With these changes in mind, our team has a clear vision for the future of this project. And we are always open to feedback and new ideas! To further illustrate our confidence in bringing this idea to market, please let me introduce you to who we are as a team:

My name is Phillip Williams. I'm 20, a sophomore at Duke University, and I am my team's coordinator. Besides managing this project and pursuing a degree in Electrical Engineering and CS, I also balance a job as a VR tour guide, and a second job as a dev/editor for a local VR studio. 

Our web lead is Mr. Aryan Mathur. This man is gifted and passionate in his pursuit of sustainability in engineering. He works as a TA for Duke's CS201 class, leading the next class of our school's finest. For our project, Aryan built the website from the ground-up and took charge of the backend and databases. 

Our front-end lead and co-lead in our video production is Mr. Ben Matz. In addition to working as a TA for Duke's CS201 class, he assembled the whole front-end of our website and made it look beautiful. He was the star of our film and took responsibility for the scripting and presentation. 

We have a diverse and talented team that has accomplished a great deal within the past few weeks between our jobs and classwork (over the entire hackathon, we each could only commit <20 hours of work per person- including the time spent learning new languages). 

Given the impressive amount of work we've accomplished in such a limited time due to our schedules, I have confidence that we could have a production-ready application within the year if given the time and resources. The generous funding from the GSF would allow us to fully dedicate ourselves to implementing and iterating upon the prototype we present to you here; We want to make a difference!

>> 6. This is Our Vision for Karbonweaver’s Future. 

Our end product, as we envision it, would be widely accessible to developers around the world. Our point-and-click user flow is intuitive and takes the burden of implementing the API off of the end user. Additionally, we have optimized execution time, which clears another obstacle from the path of the GSF API's widespread adoption. 

A secondary benefit of our application is granting underrepresented groups (those without money and in rural areas) access to better hardware for education. Additionally, our tool can also be used by users in high electric cost grids to run their programs in low-cost grids, and then reduce their electric bill. 

These advantages are strong incentives for learners, developers, academic institutions, and businesses to adopt our tool in their day-to-day workflow!

>> 7. Have a Look Under the Hood!

Please view our code and progress here! https://github.com/Aryan27Mathur/DukeCarbonHack 

We also have a demo here on our replit: https://replit.com/@AryanMathur4/DukeCarbonHack#main.py