Skip to main content

GameFace lets you control a computer with your face - yes, a feel-good AI story

AI can be a force for good. Hear me out. We saw proof at Google I/O 2023.

While many of us spend time worrying about how artificial intelligence chatbots like ChatGPT might take our jobs or set off World War II, AI-based solutions are solving real-world problems, and sometimes, even helping people bounce back (and more) from a tragedy.

After the Google I/O 2023 keynote finished blasting us with impressive AI-based product and software updates and a collection of new hardware that includes the exciting Google Pixel Fold, Pixel Tablet, and Pixel 7a, Google AI Advocacy Lead, author, and some-time AI artist Laurence Moroney took the stage during the developer's keynote and presented a story that probably prompted more than a few "I'm not crying, you're crying" reactions in the audience.

From the ashes

Two years ago, quadriplegic game streamer Lance Carr lost everything in a fire, which actually started during his livestream. Carr, who has muscular dystrophy, lost not just his Colorado home but all the assistive equipment he used to play video games live on Twitch. Worse yet, the custom equipment was outdated and mostly irreplaceable.

For Carr, who has no control of any part of his body below his shoulders, controlling his PC to play and stream games would be impossible. 

Google's Project GameFace, however, has changed all that. Instead of trying to buy and retrofit assistive gaming equipment, Google and Carr settled on a novel idea: using his face to control his PC and the games.

According to Moroney, Project GameFace uses an off-the-shelf webcam and Google's MediaPipe, a system that ties together multiple AI ML models (he didn't specify which ones) to build a live mesh of 468 points on any face. Those points are converted into telemetry for mouse clicks and movement. MediaPipe has also previously been used to create virtual avatars.

This is all tied to an interface where you can map expressions and head movement to computer input, which means instead of one-size fits all, Project GameFace can be customized for how you want to control the action. And the system is so precise, Carr can use it to write his first name in cursive.

Project GameFace

(Image credit: Google)

As Carr notes, he's now controlling his computer by making funny faces. In a video on the project, Carr explained that because of muscular dystrophy, he's weaker now than he was when he was a six-month-old. "Muscular dystrophy takes, this actually added.  The first time I gained something in a physical sense."

Google has no plans to directly productize Project GameFace, but it has open-sourced all the code and posted it on GitHub. The project page warns that the "project is not intended for human life-critical decisions," although what's more important than gaming? Ask our friends at TRG, and they'll tell you, "Nothing!"

The other bit of good news is that, while Project GameFace is constantly tracking facial movements, it's not storing images or any data for facial recognition.

So, the next time someone tells you that the obvious end game of all AI is global destruction, pause for a moment and think about or maybe even mention Carr. Where there's utility and purpose, there are possibilities. In truth, AI has as much potential for good, as it does for harm. 

Maybe we just need to point it in the right direction. Maybe.



Comments

Popular posts from this blog

The latest Apple TV 4K test lets you watch four sports streams at once

Apple is trying something new with the latest beta version of tvOS 16.5: the option to watch up to four simultaneous streams at once. Right now it's limited to live sports streamed through the Apple TV app on the Apple TV 4K , specifically MLB Friday Night Baseball and the MLS Season Pass. A multi-view option was spotted in the tvOS software last month, but the code was hidden and not enabled. MacRumors reported that the feature would be enabled this weekend, and beta testers have since been able to use it. As yet multi-view hasn't been officially announced by Apple, but it's expected that tvOS 16.5 is going to be pushed out in its final form within the next month or so. WWDC 2023 is around the corner as well, when we should be hearing about the next major updates for Apple's various operating systems – including tvOS 17. How it works Over at 9to5Mac there's a hands-on demonstrating how the multi-view feature works, and it's pretty much as you would expe...

Quantum computers are fast becoming cheaper and smaller — and they could be coming to a data center near you very soon

IonQ claims we’re closer to widespread enterprise quantum computing deployment as it lifted the lid on two rack-mounted models that can be deployed on-premises.   The startup has built the fourth-generation #AQ35 IonQ Forte Enterprise and fifth-generation #AQ64 IonQ Tempo, both of which are designed to be deployed in enterprise and government data centers. It’s also said it is deploying two quantum computers to the US Air Force.  While revealing these two models, IonQ co-founder and CTO Jungsang Kim said quantum computers are already in use by enterprises to churn through machine learning workloads. This, he added, suggests we’re much closer to readily available and affordable machines. Priming enterprises for a quantum future “We believe in the enterprise-grade quantum computing, which is where it can be something of value for enterprises, can happen in the next few years as we build powerful enough quantum computers that can actually do things that classical computers w...

Nvidia RTX 4080 GPU could get cheaper with a new version – but don’t get your hopes up

Nvidia’s RTX 4080 is purportedly getting a new spin on the GPU which could reduce the cost, but any price reduction will likely be very minor, sadly, if it happens at all. Tom’s Hardware flagged up this rumor – and treat it with caution, as with anything from the ever-spinning mill – that originated from HKEPC (a tech site in Hong Kong), claiming that while the current RTX 4080 graphics card is built on the AD103-300 chip, Nvidia is going to use a slightly different GPU in the future, namely AD103-301. There’s now more evidence this is actually happening, Tom’s points out, courtesy of a graphics card maker, Galax, which under its RTX 4080 product details lists the GPU as ‘AD103-300/301’. Furthermore, VideoCardz , which also picked up on this, informs us that Gainward, another card maker, has also listed the updated GPU variant AD103-301 in its product specs. With two separate third-party graphics card makers mentioning this new spin on the GPU in their specs, it seems pret...