Nvidia wants to release AI-controlled eye contact feature

Recently, Nvidia launched a new version of “Nvidia Broadcast,” which is deep-learning and AI-powered software that can do noise reduction, background removal/replacement, camera framing, and now even eye contact. With this function, the user will look like they’re permanently looking into the camera, even though in reality they might be looking somewhere else. This should help keep eye contact with the other person while on a video call. The feature is currently still in the beta phase.

AI has been in the headlines a lot lately, and for good reason, as this space is advancing at a stellar speed. Apps like Dall-E or Stable Diffusion create art or text, sometimes with mediocre results, but sometimes with pretty impressive ones. On the text side, ChatGPT produces legible writing that many fear will spell the end of journalism.

What is Eye Contact?

The idea behind Eye Contact is quite simple: when you’re attending a webcast or meeting, you often look away from the camera. In fact, there’s a good chance you’re always looking away from the camera because it’s at the top of the screen and the things you want to see are on the screen below the camera. But then someone thought it would be amazing if there was a way to look like you were looking at the camera without looking at the camera. And with that, Eye Contact was born.

What if you could train an AI model on faces and teach it to correct images where someone isn’t looking directly into the lens? You take millions of images, labeled accordingly, feed them into the network, and the result should be an amazing tool – at least that was the original idea.

Implementation problems

Simple in theory, but a little tricky in practice. Nvidia has been talking about its Eye Contact feature for over a year, and it’s only now being released in beta. The differences between countless faces around the world likely make it difficult to solve the problem and so far the results are rather mediocre.

One of the things early testers noticed was that the live video feed often oscillated between looking at the camera and looking at a different spot, even though the focus stayed in the same place. Maybe that’s intentional, too, because if someone were staring straight at the camera for an entire video chat, that would be pretty weird, wouldn’t it? 

How useful is the new feature?

What’s harder to say is whether this type of effect is even a real benefit. If you want to look like you’re looking at the camera, you should probably learn to look at the camera instead of creating a program to do it for you.

Fixing human errors with artificial intelligence might lead to adopting bad habits instead of simply showing a little more poise and presence. Nvidia Broadcast with Eye Contact is now available for RTX owners to try out. Right now, any RTX GPU from the past four years can support this feature.

 

Header: Nvidia