If it weren’t for technology trailblazers in film, we would still be cranking machines to view silent “moving pictures.” The world would be a drastically different place if we never heard the omniscient voice of James Earl Jones or the curses of Samuel L. Jackson in a film. Needless to say, cinematic storytelling would be incredibly limited without the trailblazing, tech-savvy titans among us.
One of those titans is virtual reality creator Eliza McNitt. Inspired by scientific wonders and discoveries, Eliza fuses science with narrative storytelling, and has developed quite the resume because of her unique vision. The Chromista director has collected awards and support from festivals like Sundance and the VR Grand Prix at the Venice Film Festival, academic institutions like the MIT-Lincoln Lab, and even the United States Armed Forces. Not to mention, her breakout film Spheres was acquired for a history-making seven-figure deal. We asked Eliza to give us pointers about the development, production, and post-production processes of VR filmmaking.
Consider this your introduction to a new chapter in the history of film, and a crash-course to help you take the first steps towards making your first VR experience.
FTW Takeaway: A good story is just as important in VR filmmaking as it is in traditional filmmaking.
Eliza McNitt: VR is about building a world that unfolds all around you. But when you are developing an experience, the core of the narrative is always the story. With all of my work, I start first with the story and then build the world and—in the case of VR—the universe around that.
When I was creating Spheres, I had the honor of collaborating with Protozoa and a team that included Executive Producers Darren Aronofsky, Ari Handel, and our producer Dylan Golden. The first question they asked me is, “What is the hero’s journey?” My immediate thought was that it’s VR, so there is no hero’s journey. But I realized that I was wrong. The hero is the user of the experience. Suddenly, the entire intention of the interactivity fully transformed for me. I connected each beat and interaction of the story to the hero's journey.
For example, in the second episode of Spheres, the viewer first watches the birth of a star. Initially they have what is called “six degrees of freedom” which is the ability to walk and look in every direction. Suddenly in the experience, a black hole sucks in the viewer, and so we created a moment where they lose the freedom to move. They are suddenly constricted, trapped in the gravitational pull of a black hole. We wanted to viewer to have a visceral connection to this story beat. It was really important that the science became something you could feel. So this was an example of using interactivity as a tool to enhance the story.
"The keys to storytelling with any new technology are to have an idea that you are deeply passionate about and to surround yourself with people that can help you bring that world to life."
Don’t be intimidated by new tech. Embrace it and take your films to the next level.
What is so exciting about VR as an emerging medium is the constant evolution of technology. One month you have controllers to interact with the experience, and the next you can suddenly interact with the blinking of your eyes. The pressure of the constant changes in technology keeps VR creators on their toes, pushing us to innovate and to create incredible storytelling experiences.
Developing interactive VR experiences involves unique planning methods.
As I develop a VR experience, I storyboard, just like how you would for a traditional film. I like to plot out the main elements of the story. But, I also have to consider the fact that I am building a world that fully encompasses the user. I have to try my best to visually illustrate all the interactive points of action as well. For Spheres , we mapped out the entire flow of movement for the black hole episode in my producer Jess Engel's kitchen. We acted out movement and taped the motion of the viewer onto the floor, as if setting up a physical dance between the user and the cosmos.
The equipment that you use depends on your story, the effects you aim to achieve, and the emotions you want to evoke.
I recently directed a commercial experience called “Drip Drop” for RYOT and P&G with Chromista. The experience reveals the stories of four individuals who didn’t “see themselves” in the creative industry, yet took action to great effect. The experience begins in a mysterious void where digitized versions of four people are floating. Viewers can bring each person to life by blinking, thus prompting an interview describing the adversity each has faced in the creative community.
In the experience, the user is submerged under the ocean, and you have to stare at different characters who demand the user’s attention. If you stop staring at them and blinking your eyes, they float away and their voices are not heard. To make that experience, I used the HTC Vive's blinking technology to allow you to advance the storytelling with your eyes.
We also shot the experience using two Phantom cameras as well as a Depthkit rig to capture the images of he people. Then we transported them and made the characters appear as if they were floating in a dark void under the ocean.
Sometimes, you won’t need to shoot with cameras.
The world of Spheres was created with CGI in a real-time game engine called Unity, so we didn’t need to use cameras or shoot anything on a set. Instead, I closely worked with an amazing team of developers that my French producing team, Atlas V and Arnaud Colinart, brought on board. Our development team at Novelab and our VR technical director, Clément Chériot were the heart of the production. They brought an expertise in game and interaction design in real-time that helped to shape the immersion of the world of Spheres.
Plan accordingly—post-production is a challenge.
When we had to shoot on a tight timeline for “Drip Drop,” we processed the depth footage immediately. While we were shooting, in between set-ups, I edited the interviews together and selected the takes of the Phantom footage that I liked. It was the most intense experience, but I had to do it because it would have been prohibitively expensive to transport all the data back to France for post. I had to make a lot of real-time decisions on the spot, which is a process I don’t recommend for anyone that wants to maintain their sanity.
There’s a lot of potential to collaborate with traditional filmmakers in the VR space.
For “Drip Drop” where components were filmed in live action and then transported into Unity, I collaborated with a DP that specializes in Depthkit named Cory Allen and worked with Julie Kirkwood, the legendary DP who shot Destroyer, a film by Karyn Kusama that stars Nicole Kidman. It was surreal to be able to fuse the traditional world of cinema that I so deeply love with the exciting, uncharted world of VR.
Sometimes you will have to give direction that you wouldn’t usually give on a traditional film set.
When I was working on “Drip Drop,” I wanted to make each character appear as if they were floating underwater. So I had people sit on a rotating seat, surrounded by fans. Then, I had them throw their arms and hair in the air as we shot them with Phantom cameras. A woman named Ashley Robicheaux, who I went to college with, helped choreograph the movement. The movement in real-time actually looks totally ridiculous. But afterwards, it all came together to create a beautiful moment of extreme grace, and the people really appear to be floating in the experience. That was one of the weirdest moments I have ever had to direct. It was this very unusual, dynamic, and exciting dance between choreography, lighting, and technology.
There are no limits to what can be made in VR. Do your research, but feel free to break from reality.
What is so beautiful about VR is that there is no formula for how to craft an experience. For Spheres, we were venturing into uncharted territory. No one has ever seen inside a black hole. I spent hours with scientists from Columbia University trying to understand what falling into a black hole as a star would look like. They told me all these incredible details of how, in theory, a star would be spaghetified, ripped into thousands of pieces, and then fall down to this flash of light called a singularity which is a point where space and time cease to exist. Then, they just turned to me and said, “You know what, just make it strange.” Making VR experiences is about making something you feel is strange and beautiful. Ultimately, we just really wanted to bring science to life in a way that we have never experienced before.
Be passionate and work with good people. Good vibes make for good VR.
The keys to storytelling with any new technology are to have an idea that you are deeply passionate about and to surround yourself with people that can help you bring that world to life. I feel so fortunate that I got to collaborate with an incredible team including Jess Engel, Dylan Golden, Arnad Colinart, the team at Protozoa, Darren and Ari, Atlas V, Clément Cheriot and Novelab who really helped and supported me to tell a story that bridged both the world of traditional cinema and this new evolving medium. VR is so boundary-pushing, and it’s exciting to experience the whole process of creation with a group of incredible collaborators.