EMO-Lens

Links

• https://github.com/BaronLiu1993/HawkHacks

Inspiration

Kobe Bryant. 

What it Does 

Through voice recognition, the device turns on begins to record your interactions. During the interactions the camera attached to the glasses is tracking the facial emotions while also capturing speech input through a microphone. From these two inputs, our Arduino is able to process the information and display the detected emotion on the LCD. On the software side, the interaction is saved and marked down as a memory for the user, which they can revisit in the future. As well, a sentimental analysis is performed based on the detected speech in the conversation and the analysis is available for the user to read.

How We Built It

Our frontend was built with React which we connected with two Tensorflow JS libraries to help take detect speech patterns. From there, we connected the data with OpenAI to provide our users with a sentiment analysis. In addition to this, we used the FaceAPI library to detect users faces and diagnosed their emotion respectively. 

On the hardware side our webcam and microphone are wired to our Arduino that sends the information back to the computer to process.  After processing, a detailed sentimental analysis is stored on our website and the Arduino receives the detected emotion which displays on the LCD.

Challenges We Ran Into

Ragious.

Accomplishments That We're Proud Of

We're really happy with the overall functionality of our project. Connecting the hardware with the software wasn't an easy task and required lots of time on our teams end. Additionally, we're proud of being able to use three AI libraries for our project. 

What We Learned

Control Ragious. 

What's Next?

In terms of next steps, our team wishes to improve the wearability (flexible adjustments) of the glasses so that it can tailor to people of all different sizes. 

As well, the LCD can only print so much information and it would be nice if we could have a larger screen to be able to display even more helpful text to the consumer. 

Adding in facial recognition into our project so that it can save the faces of all your friends and later recognize them for future conversations.