Hello VR friends, in this article, I will describe the predominant design philosophy that has driven our VR work to date. It’s a philosophy that has resulted in producing primarily instructional-based VR experiences for the purposes of teaching, learning, and research. I will describe what I see as flaws in this philosophy and I will propose a new approach that might influence our work going forward. This new design philosophy will favor producing assessment-focused experiences over instructional-based experiences.
I recently finished reading a book called Experience on Demand, by Jeremy Bailenson, who is a professor of communication at Stanford University and the founding director of the Virtual Human Interaction Lab. This book chronicles some of professor Bailenson’s extensive research in virtual reality. Bailenson has been working with VR technology since the early 2000s but he is also somewhat of a historian and references work that was done as far back as the ’80s. The point being VR research is not new, however, what is new is that for the first time virtual reality technology is now small and affordable enough for mass adoption to take place. While previously VR equipment could only be found in labs like Bailenson’s for a cost upwards of $30k, now anyone can own an HMD (head-mounted display) for under $300.
My experience with VR is small compared to Bailenson’s. While I have been developing interactive media for over 22 years, including educational game design, I have only been working with VR since 2014. I was excited to read his book to see if any of his theories, findings, and philosophies around VR development were similar to mine. It turned out they were! The book is largely stories about some of the VR experiences his lab has developed, with some brief reporting about research findings around those experiences. While the book does not offer any concrete advice on what makes for successful VR, it does provide some guidelines. It’s these guidelines where I found the most commonality between what Bailenson’s lab has done at Stanford and the work I try to do and encourage at Penn State.
Here are my 7 biggest takeaways from the book:
- While VR might be more engaging than traditional instruction, getting people interested and caring about knowledge, actual knowledge transfer shows only minor gains over traditional, far less expensive methods.
- When you directly experience something the way you can in VR, you see it in a different light.
- “VR is about exploration; storytelling is about control.” Using a technique called “Discovery Narrative” is more effective than traditional linear storytelling.
- What makes a good VR is when the entire environment surrounding a viewer is important to the experience. The experience should require a user to explore by either turning their head or moving around.
- “Ask yourself: Does this need to be in VR?” By considering all other options first, you can design better VR experiences.
- At all costs, don’t make people sick. Framerate over graphics, always!
- VR experiences should be supplemental to other learning methods, not a replacement. Experiences should be short in duration; even twenty minutes is a long time in VR.
Working for teaching and learning with technology at Penn State University, it’s my job to discover ways of integrating VR in a classroom. Most of the work we have done so far focuses on the instructional side of teaching and learning. We primarily have been trying to find ways of using VR as a knowledge transfer tool. After reading Bailenson’s book and learning about some of the experiments and studies his lab has done, I thought a shift in our design philosophy might be beneficial.
The shift I am proposing is to change our current “design an instructional experience” mindset to a “design an assessment experience” mindset. This is not a new approach. It is a method I’ve used over and over when designing educational games. With VR experiences, we have drifted away from assessment and measurable metrics as the foremost design considerations. (Even though for research this is ultimately what we are looking for and in some ways have been tacking on at the end.) I believe there has been a rush to cram instructional content into a headset and then simply examine if people respond well to it. After being involved with the creation of several underwhelming instructional-based VR experiences and after reading Bailenson’s book where he concludes from multiple studies that knowledge transfer from VR experiences has only minor gains over traditional methods of teaching, it’s time for our design philosophy to move on and create richer, more meaningful user experiences.
While thinking about this shift in design philosophy, a few (possibly research-worthy) questions came to mind:
What Separates Instructional-Based Design And Assessment-Based Design?
Instructional-based design is filled with a temptation to create content not suitable for VR. If we always carefully consider takeaway #5 from Bailenson’s book: “Does this need to be in VR?” we should realize most instructional content can be more effective and more efficiently delivered by other means. Users can quickly identify aspects of VR experiences that will not require them to have a headset on. Wearing a headset is uncomfortable. To ask users to listen to a lecture, or read large amounts of text, or participate in any other poorly conceived VR content can result not only in distraction but in rejection of the purpose of an experience.
There is also a temptation to cram too much content into instructional-based VR experiences. Remember takeaway #7: VR experiences need to be short, definitely not exceeding twenty minutes. The visceral reaction to a VR experience feels more like a rollercoaster than a long slow train ride to most users. It’s a difficult proposition to expect a user to absorb a great deal of information in a short amount of time, especially if the content does not clearly justify itself as appropriate for VR.
The goal of designing assessment-based VR experiences over instructional-based designed experiences is to avoid using a screwdriver to hammer a nail. We want users to care about experience, discover on their own why they are participating in it, and realize it wouldn’t be the same if the experience happened any other way. VR is a chance to experience situations, not be fed with information. Properly designed assessment-based VR will allow instruction to happen more organically. It should allow users to explore while being observed and measured in real time, as opposed to being told information and tested later.
Bailenson points out that VR experiences should be supplemental to other learning methods. Assessment-based VR can fit nicely as a bookend to other learning. Provided as pre and post-tests, a VR experience can provide users with measurable tasks and expose where challenges lie. These challenges, which may be unique to every one, can be addressed with feedback and instruction outside of VR before letting them try an experience again. Not only will this validate a VR experience but also the instruction that accompanies it. A well-designed VR assessment experience is an instructional experience that disguises a lesson in an activity suitable for the technology.
Can VR Be The Most Effective Assessment Tool Ever Developed?
Maybe. I don’t like to prognosticate too much about the future with these technologies. I would rather focus on what we have today and what we can do with it. But it’s not hard to imagine that five, ten, or fifty years down the road, VR might be capable of creating experiences undisguisable from reality. If this is true, then combining a VR experience design with a game design can result in assessment methods that will give participants an unrivaled measurement of realistic experience and feedback. Imagine a police training that could assess when officers are ready to handle hostile situations because they had performed well in virtual ones.
From Bailenson’s experiments, we know VR can make users care more, even feel more empathy, and that direct experience leads to seeing situations in a different light. With hyper-realistic VR, it will be possible to give people experience in any situation, allow them to fail safely but make them feel as if though the consequences are real. The experience, then, can be evaluated, provide feedback, and repeated as many times as needed until an acceptable level of performance is achieved.
Can VR Be Effective Enough At Convincing Users They Are Present In A Moment That They React As Though They Are?
If VR experiences can make users feel present in a moment, then they might also make them feel the consequences of their actions at that moment are real. Obviously, there are real consequences to traditionally written exams that come in the form of a letter grade. But, in real-life scenarios, examinations of knowledge don’t always come in written form and consequences are rarely revealed as a letter grade. VR experiences can give users a heightened understanding of the impact and the importance of the knowledge they are being asked to display beyond an evaluation of themselves.
Displaying knowledge or skill is one thing; but can you do it in a moment that matters? Understanding, feeling, and caring about the consequences of your actions add pressure to any examination. I’m a golfer. I can tell you there is a big difference between playing golf with my friends and playing in a tournament. The psychological difference between casual performance and pressure-filled performance is real. It can affect one’s ability to both mentally a