The Ultimate Beginners Guide to Virtual Reality Storytelling

As we are approaching the last couple days of our Indiegogo campaign, I wanted to post another great article on storytelling for VR from UploadVR. Please help us make the VR Experience we want to bring to you a reality, so we can give you the best art possible.

We’ve released a new perk this week called the Virtual Story Incubator. Skye will have a 1:1 hour long session with you to talk all things VR. It’s a great chance to exchange ideas and learn from each other. Check it out here.

If you can spread the word on social media, please do so. Again, thank you for all the support and encouragement. Without you none of this would be possible. Click here to support the project. 

The Ultimate Beginners Guide to Virtual Reality Storytelling

by UPLOADVR • APRIL 16TH, 2016

What is VR?

Virtual Reality [VR] is a magical alternate universe where all of your wishes come true and where pain and suffering don’t exist. Just kidding. Ok, seriously VR is the use of computer technology to create a simulated environment. Unlike traditional user interfaces, VR places the user inside an immersive experience. Instead of viewing a screen in front of them, users are “immersed” and able to interact with 3D worlds.

Short and sweet: in VR we create worlds and invite individuals to experience them. This means that in VR you’re not just an observer but a participant in the story with an option to influence the story itself. That’s pretty rad!

Who is this for?

This guide is for everyone who is dipping their toes into Virtual Reality storytelling and filmmaking. This includes VR game developers, filmmakers and amateur enthusiasts who are discovering the freedom of the new medium.

I won’t go into the theoretical aspects of filming because that’s not the domain or the scope of this eBook. I also won’t talk about how to choose the right equipment, camera system, editing processes or which software to use for post-production. Instead, I’ve gathered the best practices and tips that the most successful filmmakers used on their projects. I’ll strictly focus on creating a good story.

It’s important to understand that we are all experimenting and learning about VR. For that reason, you won’t find many books on the subject. All the progress and learning is currently happening in the real world examples.

This eBook’s goal is to give you a boost and understanding what VR can bring to the table. It’s a new medium that gives you as much or more power and creativity than traditional filmmaking. But as Spidey said: “With great power comes great responsibility.” I wish you can avoid the pitfalls that can set you back in your next awesome VR feature film.

I’d also like to thank you for taking the time to read this. VR has been making waves since the start of 2016, but for people to grasp it and have a positive experience from it, they need good content. That’s where you come in. By reading this you set yourself on a path of storytellers in the new era. You’re making history already!

Virtual reality was once the dream of science fiction; but the internet was also once a dream, and so were computers and smartphones. The future is coming”. — Mark Zuckerberg

Types of VR

Virtual Reality comes in different forms. Either Computer Generated Images (CGIs) and display live images from the physical or real world. There are Heads Up Displays [HUDs], or Heads Mounted Displays [HMDs] that can superimpose CGIs onto the real-world . This function is often referred as mixed or Augmented Reality.

The 360-degree video, also known as 3DVR and Stereoscopic VR, use multiple cameras that capture the image from 360 degrees. There are some that would argue that 360 video is not really a VR. Technically they are correct. A standard 360 video is just a flat equirectangular video displayed on a sphere. Think of it like the face of a world map on a globe, but with VR, your head is on the inside of the globe looking at the inner surface. As you move, the head tracking on your device moves with you, giving you that feeling like you are inside the scene.

360 video is usually augmented with stereoscopic 3d which adds another level of immersion by adding depth between the foreground and background. This is achieved by shooting the scene with two lenses side by side, that give you a feeling of different vantage point per eye. It can appear strange if not done correctly, but great if done right. With stereoscopic 3D in VR, that depth information has to be overlaid and mapped to the sphere. Because of parallax between cameras, this can be tough to achieve. Any minor flaws or “stitch seams” in the footage are magnified in 3D. These anomalies occur in different places per eye — which makes it uncomfortable to watch or even promote headaches, eye strain, and motion sickness.

The goal of 360 and VR video is the same — total immersion of the viewer in the created world. Once the viewer feels comfortable in the created scene, we can serve him with a story.

What do I need to know about VR Storytelling vs Traditional Storytelling?

Virtual Reality is now firmly established. The technology that we have been craving for is now at our fingertips; We have the ability to spread the word and show the people the freedom of the new medium, and we can make them comfortable with VR to use the right way. This means establishing new routines and especially showing them what they can do in the world you have created for them.

In the old days, we only had a pantomime and a black and white camera standing in front of the theater. Then we learned to use the camera, panning it around, trying out different shooting techniques and points of view [POVs], zooming into people’s faces, cross-cutting, multiple locations, etc. Thus, the cinematography was born.

VR to a film is what film is to pantomime. It will completely change the game when we figure it out. The POV and cinematography break the 4th wall that separates the actors, and the audience goes away. That’s why in VR you have to find creative solutions on how to guide your audience.

The viewers of VR are not just passively looking at a flat screen. Suddenly, your viewer is inside that “frame”, freely looking around and within the next few years, they’ll be able to move around too. You can get an idea how impactful these connections are.

This unique sensation of “being inside ” is called presence.

Chris Milk, founder of VRSE and VRSE.Works calls VR the ultimate empathy machine.

“VR is difficult to explain because it’s a very experiential medium. You feel your way inside of it. It’s a machine, but inside of it, it feels like real life, it feels like truth. And you feel present in the world that you’re inside and you feel present with the people that you’re inside of it with.”

For storytellers, your first thoughts might be cramming all the traditional knowledge you learned in film school into a new medium. This can lead to negative consequences because you’ll soon find out that VR goes beyond traditional filmmaking rules and techniques. With that said, it’s important to remember that the stories don’t change, but the way we tell them in VR will change.

Before you start filming… I mean creating stories

There seems to be a confusion if VR filmmaking is possible. Most of the confusion comes from not understanding the difference between 360 videos and VR. The simple differential factor is one element — Interactivity.

In reality, your actions (and inactions) have consequences. The decisions you make change the world around — or your experience of it — sometimes in profound, unexpected ways.

These changes are not scripted unless you believe in some kind of outcome.

Virtual Reality should then conform to the same rules. The decisions in VR should change the constructed world in an unexpected way. This means the VR world should change without predetermined scripts.

PRINCIPLES:

Hundreds of VR projects are being developed by various artists and studios; I’ve collected few core principles, or should we say axioms, that you will have to take into consideration before you even start shooting.

They are:

  • Creating a great story is going to be a battle of compromises.
  • Think of yourself more as an influencer then a director. It’s all you can do, because you’re not showing your audience just a frame but a world.
  • The core of VR is experience and presence. As a director, you’ve got the huge responsibility to keep your viewer engaged. Your viewer can’t switch the channel or look the other way — the VR device is mounted onto his or her head, and something major has to occur for him or her to pull the HMD off.
  • VR Storytelling is a lot like being in a video game.

Tip 1: Ease it in

The first goal we have to achieve with the viewer is making him feel comfortable. Oculus Story Studio recommends 30-second introduction video. This will give the viewer time to adapt to the headset first and familiarize themselves with the new medium.

Lost

This “settling in” period is enough for most people to adapt to the environment and relax. In Lost, you start in a darkened space. A pinpoint of light appears: it’s a Fi the Firefly. The Firefly buzzes around from left to right. It’s placed there with an intention for the viewer to follow it around and get used to the movement in VR. You soon realize you’re looking at a forest, and then the movie’s opening credits start as light strings swell in the background.

First-time viewers are used to watching flat screens; most likely they will just stare straight forward. You can use narrative voice or visual cues to encourage them to look around. Give your viewer a permission and encourage their curiosity.

Tip 2: Guide the Viewer’s Attention… Or not!

You will have to decide beforehand how do you want to treat your viewer. Would you like to guide him through the story and lead her where to look? Should we discourage a viewer from looking somewhere else?

Give the viewer complete control of the situation? For directors, relinquishing control is a completely new concept. If a viewer really wants to spend the entire time looking up at the sky, even though the narrative is happening in front of him, the director of a VR movie has to accept this. Cinema has plenty tricks in the bag to tell stories and guide viewers attention.

You can use different lighting or sound cues, a focal point of a character’s POV onscreen, or even verbal/action cues. The important fact to remember is that our gaze is attracted to two things: movement and faces.

The double edged-sword of control is it may feel inauthentic. Since VR is the medium about your viewer’s freedom, I suggest you embrace it. Instead of spending a considerable amount of time and energy attempting to control something that you fundamentally can’t focus on delivering a world of wonders — a unique experience that makes people feel.

Tip 3: Presence

Total immersion is often the end goal of VR storytelling. The first obstacle you need to overcome is a functional technology. This means you have to prevent or repair the stitching issues; use the highest quality picture possible and the right sound equipment. The last thing you want is to cause headaches or nausea to you viewers and thus ruining the chance to show him your world.

Second, you have to trick the viewer’s brain the VR world is “real” by figuring out a way to trick the mind to accept things that are clearly not real. Remember Kevin Spacey’s line from The Usual Suspects?

The greatest trick the Devil ever pulled was convincing the world he didn’t exist”.

Montreal-based studio Felix & Paul had an “a-ha” moment on one of their first projects. They brought a non-professional actor, an older Chinese woman with gentle personality, the lady was walked into the church and sat down next to the camera. After few seconds, she looked at the camera as if was a person. That 20-second scene blew people away. They couldn’t stop thinking about the woman!

Use characters that look directly at the viewer. The intimate connection evokes authentic emotions of the actor and the viewer. This is a perfect example how VR has the power to break through the 4th wall. 

Henry

There are situations where you don’t want the viewer to be the center of attention; especially action scenes like high-speed car chases or martial art performances, where it makes sense to use different perspectives.

Tip 4: Pacing

A sudden shift from flat screen frame to a 3D immersive world can be confusing and overwhelming at first. If you’ve done a good job with settling in your viewer and making him or she feel comfortable you’ve completed the first goal.

Oculus Story Studio is using the ritual of “Settling in & Setting the Scene”. In “Lost”, they used a firefly which flies a bit to left and right, just so the audience get used to the concept of looking around. The firefly is the only thing they see before the mysterious forest appears.

Your viewers, most likely, aren’t used to VR films and for most of them, this could be their first VR experience. Because of such a huge jump in information density from tv screen to immersive environment, think about how you’d gently ease them into the story.

You may decide to move the camera in your story but be very careful with an undulating movement, since it can cause sickness.

There’s a thin line where the story is interesting because there is a lot of things going on at once, beware of overwhelming from sheer information overkill. If there’s just one thing going on, the scene will feel empty and inauthentic. VR should emulate a real world environment. Oculus calls this “Spatial Story Density” — a three-dimensional storytelling with the space around us.

Remember our first principle, we will have to make compromises.

Tip 5: Conditional Storytelling

In “Mass Effect”, a popular Sci-Fi Role Playing game, we are put in the role of Commander Shepard’s point of view. We are responsible for our spaceship, the SSV Normandy, its crew members, and choosing which mission to tackle. As we progress through the game, we are forced to make difficult decisions. The decisions carried huge consequences — a death of a crew member, genocide of a certain race, even the destruction of the whole planet. 

Mass Effect. Make a choice and face the consequences…

In VR, we have the ability to give the same power of decision making to our viewers. How much power we delegate is entirely our decision. These decisions can be thought through and mediated. For example, our viewers come to a checkpoint where the action stops and are forced to make a decision to continue. Choose a path — will our viewers take the red pill or the blue pill?

Will you take the red or the blue pill?

But we can go a step further and predetermine an expected reaction. VR already have the capability to go beyond the usual content experience; It can simultaneously engage visual, auditory, and haptic (touch) senses. Virtual space within the story can respond to speech and gesture inputs from the viewer. This might be technically much harder to achieve, however, as it simulates reality much better than thought breaks in the previous case.

Let’s assume you are in a scene where the actor and our viewers are in a passenger seat driving in the a car. The street seems empty and the driver speeds up. As the car approaches a pedestrian crossing, we notice a child running down the street. If the driver doesn’t hit the brakes, an accident will occur. If we scream, the driver will try to stop the car, but if we fail to notice the child or decide not to react, the outcome will be different.

Tip 6: Experiment

VR is an exciting sandbox where we find endless tools and tricks to play with. The tips so far are not the rules of filmmaking, but rather suggestions. However if our tips were the rules, we should break or bend them any way you can.

Henry Ford’s famous statement was: “If I’d asked my customers what they wanted, they’d have said a faster horse.” Instead of making the horse faster, he built a car. Something that hadn’t even existed in people’s minds.

I dare you to be as bold in VR storytelling in the same way. Make mistakes because we will either learn something that doesn’t work or we’ll find an ingenious solution. Charles Goodyear accidentally invented vulcanized rubber when the combination of rubber and sulfur dripped on the hot stove. Much to Goodyear’s surprise, the rubber didn’t melt but actually hardened. This accidental product is used on almost every vehicle today — it’s your tires.

Tip 7: Test, Test, Test

Product and services are always tested before release into the public. So should your VR creation. Since we have established a developmental l relationship with our work, we are not the right person for testing our design. We might be too critical or forgiving of our creation.

Ideally, we should show our work to random people that we don’t know since they have no vested interest in outcomes and won’t stroke our ego.Their observations, feelings, and especially reactions watching our creation are pure gold.

This expression is usually a good sign.

Recommended Reads

FOLLOW THIS PERSON FOR MORE VR STORYTELLING GOODNESS:

Jessica Brillhart — Principal Filmmaker for VR at Google

From setting the stage, introduction to moments and fundamental questions about Virtual Reality Storytelling. Brillhart is one of the first creators to follow and learn from.

Neel Kshetramade— Provides Insightful and interesting theories about Storytelling in VR.

Oculus Story Studio Blog — Lessons learned from the Oculus creative minds

Nick Bicanic — Composition/Focal lengths in VR

Good advice on technical aspect and shooting on the field.

Katy Newton & Karin Soukup — The Storyteller’s Guide to the Virtual Reality Audience

A super useful guide to engaging your audience in VR storytelling, and understanding where and why we look at certain places.

Get in Touch

Hope these few tips will help you make absolutely amazing creations. I am super curious about your VR videos. If you have in mind creating “Choose Your Own Adventure” style stories in VR, I strongly recommend signing up with Viar360, since this would be one of the main features.

Get this guide in a printable PDF document

Note: This post was originally written on Medium by Dejan Gajsek and has been posted here with the author’s full permission. 

4 Lessons Virtual Reality and 360 Filmmakers Can Learn From The Theater

Check out my article on storytelling for VR and 360 Video on Upload VR. It was just published this morning! :)

Also don't forget to show us some love for our Indiegogo campaign: www.foolvr.net/IGG We are offering a VR story Incubator perk with non-other than myself!

4 Lessons Virtual Reality and 360 Filmmakers Can Learn From The Theater

by SKYE VON for UPLOADVR • JUNE 12TH, 2016

As many classically trained filmmakers struggle to make the transition to virtual reality, they are given the advice to turn to the theater and use its approach not only regarding production, but also regarding storytelling. And there are good reasons for it.

Recently, I had the opportunity to see the play Mount Olympus: To Glorify the Cult of Tragedy at the Wiener Festwochen (Vienna Festival), which is an annual cultural festival that takes place each year for five to six weeks in May and June in Vienna, Austria. Mount Olympus was not only exceptional in delivery, execution, and art direction, but it also pushed the boundaries of the conventional theater format and storytelling. Instead of choosing one of the many Greek Tragedies, the director Jan Fabre opted to tell all in an impressive 24-hour stage production. Yes, you read it correctly, 24 hours. Even though the audience was allowed to come and go as they pleased, the goal was to transfer the physical exhaustion of the performers and the act of glorifying tragedy viscerally onto the theater-goer. And I think it succeeded at this brilliantly. But the reason I am writing about it today is that it also offered an opportunity for me to reflect on how theater and storytelling for theater is the closest existing structure storytellers can reference and work off when approaching storytelling for VR and 360-degree video.

Every Character is a Hero

As I was sitting in the audience, I was reminded about how theater calls for each character on stage to have a full storyline for the whole time they are on stage, since during a play the director cannot control where the audience will choose to look or which character they will decide to follow. Of course, the director can utilize such tools as sound or visuals to draw the audience to a particular point, which is also an instrument for use in VR and 360 Video. But even still, there is no guarantee if the audience will respond to these signals and that is what the director should prepare for. This goes as well for VR, which is so very different from film.

Read More: The Ultimate Beginner Guide to Virtual Reality Storytelling

In film where the director utilizes cuts to control what the audience sees and who it follows, the hero is often the only character who is given a full story arc. In some cases, a supporting character or two will also have their own arc to complete, but the other supporting roles only serve to push the main characters’ journey of discovery forward and to completion. But since it is different in VR and 360 Video, it is imperative that the writer lets go of the concept of the single hero or protagonist. Each character will need to be developed fully for the time they are on screen.

Take Your Time When Telling the Story

Aside from French Cinema, not one second is wasted or dallied with when it comes to film and the cinema. Each moment on-screen counts and is as precious as gold. But when it comes to theater, for the reasons mentioned above, it is the overall scene that matters, not each single moment. So what you will notice in theater is that its pace is much slower than that of film. This allows for the audience to look about the stage, catch up with the action and even follow it if they miss a beat.

Because even in theater there is a main storyline that the director wants his audience to experience. And if this is your goal in VR and 360 Video, then it is also important to adopt a slower pace for it as well. Slow down the unfolding of events and let each scene breath, then you can trust your audience to find their own story.

The experimental theater experience  Sleep No More  

The experimental theater experience Sleep No More 

Trust Your Audience

And discovery is a vital component to keep in mind regarding storytelling for VR and 360 Video. It is actually the most important. VR and 360-degree video asks of us as storytellers and directors to allow our audience agency and participation. This is something classic theater does to a lesser degree, but experimental theater is built on. In Mount Olympus the audience was allowed to come and go at their own will, while in the New York experimental theater smash hit Sleep No More, another experimental theater experience to reference, which mimics VR and 360-degree video even more closely, the participant is left to roam, is sometimes pulled from room to room and from floor to floor in the reimagined building of the McKittrick Hotel to discover the classic story of Macbeth.

Both make the audience active participants in how they will experience the story, leaving each to walk away with an entirely unique experience. This is what we should strive for as VR content creators. But this asks of us to relinquish control and trust the audience to find their own story, which I understand is difficult for filmmakers since many of us have spent years harnessing the control freak within us because that is what is needed when it comes to movies.

Sleepy Hollow storyboard for their VR experience

Sleepy Hollow storyboard for their VR experience

The Audience is Not Only a Character, But Also a Co-Director

The beauty of allowing the participant to discover the story is that the experience will not only be utterly unique for each and every participant, but also for each and every time they partake. Making it something that we will want to enjoy over and over again. This applies maybe less so to 360 Video since it only offers one level of agency to look around in 360. Virtual Reality, however, permits for more active interaction because it utilizes things like game engines, which can enable spatial interaction and manipulation. Just like in Sleep No More where the 4th wall was completely torn down because the action moved from room to room, so also VR can we remove any barrier between the world of the experience and the user.

It is this which is one of the biggest promises of Virtual Reality, especially regarding monetization of this medium. Within my circle of friends some have gone to see Sleep No More not only twice, but multiple times, some as many as ten times, which is incredible since a single performance costs a hefty $100. This shows us that we have an audience ripe and hungering for unique and actively engaging experiences. This also means there is no better time for VR.

So approach each character as a hero and develop his her storyline thoroughly, take your time, trust your audience to find the story, and let them become active agents and co-directors of the story. If you do all this, then you will have successfully cracked storytelling for Virtual Reality and 360 Video. And as we expand into Augmented Reality (AR) these principles apply all the same.

Article contributed by Skye Von. Skye is a director, writer and VR creator based in NYC. Starting in theater back in London many eons ago, she feels like she has come full circle to be telling story again in a 3-dimensional space. The potential she sees for VR is endless, and she can’t wait to take everyone to a place called the Metaverse. Currently she is working on the VR experience Falling Out of Love, which is reimagining the music video for the band Aloud. Skye was selected as the 2016’s IFP/Indiegogo Fellow at the NY Media Center for this very project with her team. She’s a member of Association of Women Directors (AWD), of New York Women in Film & Television (NYWIFT) and of Film Fatales. She once shared a Jeep from Timbuktu to the Sahara with Henry Rollins.