Filming Movies and Commercials in the Metaverse

One of the metaverse creators I profiled in Chapter 6 of The Metaverse Handbook was Joe Hunting for his ingenuity in being the first filmmaker to shoot a movie entirely in a metaverse environment, specifically VRChat. Joe’s documentary We Met in Virtual Reality was selected for the 2022 Sundance Film Festival and is now available on HBO Max.

The metaverse is largely pegged as a social environment for gaming, entertainment, and productivity.

But I think we’re overlooking the potential for the metaverse to offer film and commercial sets, as well as general content creation studios.

The Tech

Horizon Worlds, Spatial, and Roblox, among other sandbox environments, all offer the ability to design spaces exactly as you want.

Filming in the metaverse is a little trickier as you must rely mostly on screen-capturing your first-person POV. Screen captures don’t provide a lot of optionality in dimension or resolution without a little extra work.

However, we’ll see more developers create tools for better filmmaking in the metaverse.

For example, VRCLens is an Avatar add-on virtual camera for VRChat which comes with many interesting camera features like controlling the depth of field at multiple focal lengths, autofocus, image stabilization, drone footage auto-lock, and more. There are also green screen controls, anchoring camera controls, and filters. This is what Joe Hunting used to film his documentary.

The Use Cases

I was watching YouTube the other day when I was delivered an ad from the national law firm Morgan & Morgan. It caught my attention because it looked like they shot the entire thing in Roblox:

Ultimately, I determined that they didn’t film the commercial in Roblox, as I’m sure there’s a Roblox licensing agreement preventing free-reign over using footage commercially. But theoretically, you could design a Roblox set to record commercials, TV shows, or sitcoms.

Furthermore, I’ve already seen a metaverse-based vlogger, known as SWISSVERSE, who vlogs his experiences in Decentraland. Xanadu was a brief YouTube series that followed a character in an entirely CGI-created world.

Jay Haynes shot a short comedy film called “The Horizon Hipsters” (below) and then shared his main takeaways from the process of metaverse filmmaking here. Basically, the benefits were that it cost nothing, there were no unexpected variables, and he was able to film with people from around the world. The drawbacks were that there were limited lighting options, camera angles, expressiveness, and actions.

Namoo is a narrative poem come-to-life as an animated virtual reality experience created with Oculus’s VR animation tool “Quill.” It was selected for the 2021 Sundance Film Festival.

My Take

The metaverse as a commercial, content, and film set is an idea we’re only just beginning to explore. Zuckerberg just announced many upgrades to their avatars to make them more realistic and expressive through face-tracking, which should ultimately lend itself to better metaverse-based content creation.

I think it was particularly great to see a law firm trying something different with its image.

Overall, the metaverse is an ideal destination for content creation because it provides complete control over the environment and eliminates unexpected variables.

• Sets – Any location can be created. We’ve already seen the proliferation of this with modern CGI. Major blockbusters like Avatar, Aquaman, The Great Gatsby, and Gravity were filmed almost entirely in front of green screens. The metaverse builds on this concept.
• Crew – This won’t eliminate crews, but it will change their makeup mostly to programmers and designers.
• Cast – Celebrities are still the main draw of movies, so I don’t think this changes much. However, it could change how they’re represented.

I see a lot of opportunities for designers to start building build city-scale environments, like studio sets, that can be rented out at any time. In other words, now is the time to position services around set design in Horizon, Spatial, etc.