When Apple introduced Face ID in the iPhone X last year, I predicted that it was only a matter of time before someone figured out a way to use the data collected by the Face ID cameras to create a legitimate motion capture tool for mobile filmmaking. That day is here with Live Face, an app that captures your face, and your body to create animated characters.
“Out-of-the-box, LIVE FACE is probably the easiest facial mocap system of all to use. Reallusion have taken iPhoneX facial mocap to the next level. With iPhoneX and the new LIVE FACE plugin, iClone users can now produce high quality facial motion capture direct from the phone to any 3D character in iClone.” – 3D artist Mike Sherwood
The iPhone X is already a hit with mobile filmmakers, but now, they can not only get into a run and gun style of film production with their mobile phones, but they can use that phone to do motion capture and expand their repertoire into the animated genre thanks to Live Face. Created by Reallusion, the Live Face app takes the iPhone Face ID tracking cameras that are utilized for popular “animoji” messages and live streams them via USB or WiFi to a workstation running animation software like CrazyTalk Animator 3.
- The TrueDepth Camera projects and analyzes more than 30,000 invisible dots to create a precise depth map of the face.
- Analyzes more than 50 different muscle movements to mirror expressions.
- Highly sensitive to the details, like rotating eyeballs and tongues.
- Works with different skin colors and extreme light conditions.
- Strong A11 Bionic chip to totally release a computer’s CPU resource, sending analyzed mocap data instead of full image frames.
- Runtime Calibration: on-the-fly calibration without pausing the face tracking process.
- Improved Stability: less likely to require mocap data cleanup.
- Supports both WiFi and USB connection.
“LIVE FACE has absolutely changed the way I animate the characters for my children’s animated Series, The Adventures of Nkoza and Nankya!! I am able to capture in realtime my daughter and son who voice act the characters of Nkoza and Nankya, in away I never could before LIVE FACE. This has taken iClone to a whole new animation level”, says creator and director Solomon W. Jagwe.
Moreover, through Reallusion’s iClone Motion desktop software, the iPhone can also become a full body motion capture tool capable of animating the actor’s complete performance, not just their facial reactions. iClone Motion Live was released earlier this summer, and it takes data from motion capture gear for face, body and hand movement without the need of expensive motion capture camera arrays. All you need is the essential gear that provides a set amount of data points is required and is essentially plug and play.
Companion motion capture desktop apps like XSens, Noitom, Rokoko, OptiTrack and Leap Motion can translate the data streaming from the iPhone X Face ID camera into realtime motion capture dots that can then be applied to animated characters through animation software like iClone Character Creator or Daz 3D.
No word on if Adobe supports the feature through Character Animator yet, but you can bet they have the same idea. Perhaps we’ll see it during Adobe Max later this year? Here’s hoping. Until then,it’s easy to see just how powerful that the iPhone’s Face ID camera is. It’s not just for unlocking your phone anymore, it’s for serious animation and CGI motion capture work.
The Live Face App as a free download from the iOS App store. Great to play with, but you’ll also need the iPhone X LIVE Facial Mocap plugin for iClone Motion Live, which is downloaded at the Reallusion website here. To apply the data onto your character, Reallusion also has an iClone Live Face Profile, but that’ll set you back $399. The good news is, there’s a try before you buy option.
Hat Tip – PVC