Facebook did something this week that wasn’t terrible (yet)

Just a heads up, if you buy something through our links, we may get a small share of the sale. It’s one of the ways we keep the lights on here. Click here for more.

If you grew up in the ’90s as I did, you probably remember the rise of FMV video games, like Mortal Kombat, where pre-recorded video clips were blended with sprites and 3D models to make something magical happen. It appears that some of Facebook’s staff also fondly remember this time, as they’ve created an AI system that can create controllable video game characters from real-life videos.

Called Vid2Game, it uses a dual system of neural networks to create the animations. Pose2Pose first maps a current pose from the video, and a single-instance control signal to the next pose, and Pose2Frame which passes both the current and next pose onto an output frame, with any background you want. The animation can be controlled via a joystick or any other control method, and the system can also position those created characters in dynamic backgrounds.

The animations created by the system are still slightly jerky if you compare them to the ingame animations of any recent AAA game title

The tech is only going to get better from here though, so fluid, life-like animations and movements won’t be long in coming. Maybe if a hybrid approach adding pre-recorded animations based on mocap recordings would make this system near-perfect.

Does that mean we can essentially scan ourselves into any 3D video game as the playable character? The team at Facebook’s AI Research labs says not quite, but it does “pave the way for the next generation of realistic video game graphics.” They also mention that characters can be extracted from YouTube-style videos to then be placed in virtual worlds or augmented reality programs. From video star to videogame star, eh?

I could see this finding a good home on another Facebook-owned platform, Oculus. The ability to create VR avatars to then use in VR chat applications, or even in games, would be a great use of the tech.

What do you think of Vid2Game? Interested in seeing more from this? Let us know down below in the comments or carry the discussion over to our Twitter or Facebook.

Editors’ Recommendations:

• Apple is working on giving you the same experience on iOS and macOS
• T-Mobile finally launches its new banking service, T-Mobile Money
• The consumer version of BlackBerry Messenger is shutting down in May
• 2019 iPhones are getting a much needed two-way charging upgrade
• YouTube Music is now free to listen to on Google Home smart speakers, but there’s a catch

Joe Rice-Jones

Maker, meme-r, and unabashed geek with nearly half a decade of blogging experience at KnowTechie, SlashGear and XDA Developers. If it runs on electricity (or even if it doesn't), Joe probably has one around his office somewhere, with particular focus in gadgetry and handheld gaming. Shoot him an email at joe@knowtechie.com.