Skip to main content

GameFace lets you control a computer with your face - yes, a feel-good AI story

AI can be a force for good. Hear me out. We saw proof at Google I/O 2023.

While many of us spend time worrying about how artificial intelligence chatbots like ChatGPT might take our jobs or set off World War II, AI-based solutions are solving real-world problems, and sometimes, even helping people bounce back (and more) from a tragedy.

After the Google I/O 2023 keynote finished blasting us with impressive AI-based product and software updates and a collection of new hardware that includes the exciting Google Pixel Fold, Pixel Tablet, and Pixel 7a, Google AI Advocacy Lead, author, and some-time AI artist Laurence Moroney took the stage during the developer's keynote and presented a story that probably prompted more than a few "I'm not crying, you're crying" reactions in the audience.

From the ashes

Two years ago, quadriplegic game streamer Lance Carr lost everything in a fire, which actually started during his livestream. Carr, who has muscular dystrophy, lost not just his Colorado home but all the assistive equipment he used to play video games live on Twitch. Worse yet, the custom equipment was outdated and mostly irreplaceable.

For Carr, who has no control of any part of his body below his shoulders, controlling his PC to play and stream games would be impossible. 

Google's Project GameFace, however, has changed all that. Instead of trying to buy and retrofit assistive gaming equipment, Google and Carr settled on a novel idea: using his face to control his PC and the games.

According to Moroney, Project GameFace uses an off-the-shelf webcam and Google's MediaPipe, a system that ties together multiple AI ML models (he didn't specify which ones) to build a live mesh of 468 points on any face. Those points are converted into telemetry for mouse clicks and movement. MediaPipe has also previously been used to create virtual avatars.

This is all tied to an interface where you can map expressions and head movement to computer input, which means instead of one-size fits all, Project GameFace can be customized for how you want to control the action. And the system is so precise, Carr can use it to write his first name in cursive.

Project GameFace

(Image credit: Google)

As Carr notes, he's now controlling his computer by making funny faces. In a video on the project, Carr explained that because of muscular dystrophy, he's weaker now than he was when he was a six-month-old. "Muscular dystrophy takes, this actually added.  The first time I gained something in a physical sense."

Google has no plans to directly productize Project GameFace, but it has open-sourced all the code and posted it on GitHub. The project page warns that the "project is not intended for human life-critical decisions," although what's more important than gaming? Ask our friends at TRG, and they'll tell you, "Nothing!"

The other bit of good news is that, while Project GameFace is constantly tracking facial movements, it's not storing images or any data for facial recognition.

So, the next time someone tells you that the obvious end game of all AI is global destruction, pause for a moment and think about or maybe even mention Carr. Where there's utility and purpose, there are possibilities. In truth, AI has as much potential for good, as it does for harm. 

Maybe we just need to point it in the right direction. Maybe.



Comments

Popular posts from this blog

TechRadar's Year in Review: AI revolution, smartphone evolution, and visions of the future

If I had to sum up 2023 in one word it would be AI. Okay, technically that’s two initials, but let’s not quibble – because there’s no doubt that it provided the year in tech with its main story arc. Artificial intelligence was everywhere this year, as every company in the world gawped at the massive impact of ChatGPT, and decided they wanted to jump aboard that bandwagon. You could barely move for new chatbots, new image generators, and new AI-enhanced features in every device released in 2023. And there’s no sign of that changing in 2024; in fact, we expect it to be an even bigger deal. Elsewhere, phones and laptops did much the same as they had the previous year – well, except where they had new AI-enhanced features – so it was in the virtual reality space where bigger changes were afoot. If the Meta Quest 3 was impressive then the Apple Vision Pro looks to be genuinely game-changing, and we can’t wait to see what impact it has in the next 12 months. Of course we also saw plenty ...

Microsoft and Sony enter "binding agreement" to keep Call of Duty on PlayStation

Microsoft has announced that it has reached a "binding agreement" with Sony, pledging to keep Call of Duty on PlayStation following the acquisition of Activision Blizzard.  With Microsoft having won its anti-trust case with the US Federal Trade Commission , its planned acquisition of Call of Duty developers Activision Blizzard has taken a big step closer to becoming a reality. In order to alleviate the concerns of monopoly-building that got Microsoft in legal hot water to begin with, the head of Xbox, Phil Spencer, announced yesterday via Tweet that it has entered into an "agreement to keep Call of Duty on PlayStation."  This agreement will no doubt come as a relief to fans concerned that Microsoft's acquisition plans might result in Call of Duty becoming an Xbox exclusive, following in the footsteps of future Bethesda releases. Since Microsoft's acquisition of Bethesda caused the studio to break its tradition of offering titles cross-platform for upcomi...

This Stardew Valley-like farming sim plays at the bottom of your screen so you work and play at the same time

Stardew Valley and Animal Crossing are among some of the best choices out there if you're looking for a game to get swept up in for hours at a time. But, if you're a fan of management sims and a multitasker as well, this upcoming farming sim will likely pique your interest.  Developed by Mister Morris Games, Rusty's Retirement is an idle-farming simulation game that runs at the bottom of your gaming monitor, allowing you to focus on other tasks like working while you tend to your crops. Set to release in Q1 2024 on Steam , the idle sim puts you in the shoes of a wholesome robot as you grow crops and use them to produce clean and energy-efficient biofuel. Having gathered this "vital robot power source," you'll then need to sell your harvest to the other machine inhabitants of your settlement. This, in turn, will let you reinvest the money into expanding your farm. If you're a terrible procrastinator (like me) this might not sound like your cup of tea...