January 2018 - March 2018
A Social VR Usability Study @ with
Understanding how Novice Users perceive and navigate a Social VR space
How easy/difficult is it for novice participants to navigate the social VR space (Rec Room) for the first time?"
01
Project Background
As companies are increasingly creating content for virtual reality devices, we are seeing a rising interest to engage socially with others in VR whether in games or productivity apps. This project was a summative usability study, based on the social VR game Rec Room**, to explore how novice VR users navigate a social digital space.
**Rec Room is a Social VR space that allows users to meet other players in the virtual environment. Users can interact with other players in a variety of ways; they can communicate verbally through a microphone, play games with a sportive goal together, or interact physically in the VR space, e.g. giving out high-fives.
The Team
My Role
Jian Zheng, Dmitry Levin
...and me 🙂
Mentored by Angela Sharer, Lead User Researcher @HTC
Pre- and Post-Study Interviews + Leading Usability Sessions (3 of 8 interviews + sessions)
Tools
Camtasia, Rec Room (VR Game)
Some of the key questions we wanted to explore were:
-
Are the various navigation options and affordances easy to discover and use?
-
Are participants able to create an avatar they are satisfied with?
-
How comfortable are participants in interacting with others in VR?
02
Research Methods
Pre-/Post-STudy
User Interviews
Secondary Research
In-Person Usability Tests
03
The Participants
Who we were looking for:
-
Savvy tech users of any age/gender group that have not used VR before (or a minimum of 1-2 times).
-
Early to midlevel adopters of new technologies (preferably no late adopters).
Our participants profiles:
-
7 participants with little to no VR experience
-
Current social media users
-
5/7 participants played little to no video games
-
Age range: 19-36
-
Gender: 3 Male | 4 Female
High level of interest in VR: average of 4.28 out of 5 point scale where 5 is very interested
Participants' game avatars from the usability study
04
Key Tasks
For the purpose of this study, we set up a new user account for each participant during the in-person usability test. Each participant was guided through a short tutorial. The user starts in the Dorm Room, where the avatar customization takes place, and where the user can see different challenge boards. Each player is then prompted into a paddleball game with another real user. After that, users are free to explore various game spaces independently.
Users can navigate the Rec Room space using a watch menu; the game spaces can also be reached going through different doors in the Rec Center, a central location of the game where users can interact with other users between games or play mini games like basketball.
For the purpose of this study, we asked participants to complete the following six tasks in the VR program Rec Room:
Task 1: Log into Rec Room
Please log into Rec Room with the user <username>.
Task 2: Discover and use the game tutorial
Now that you’re logged into Rec Room, explore as you would at home.
Task 3: Discovering and customizing an avatar
Now you want to change how your avatar looks.
Task 4: Join a game and manipulate an object
Join a game that you'd like to play in Rec Room.
Task 5: Interact with another User
Find another game you’d like to try where you can play with another user.
Task 6: Navigate back to the avatar customization
You’re interested in going back to change your avatar’s look for a different game, find a way to do this.
05
Key Takeaways
-
Participants like to explore the VR space independently.
-
A tutorial should be flexible and explain the controller functions and navigability well.
-
Cyber bullying can have a strong negative impact on the overall VR experience.
-
Not all users are comfortable talking to other users, or interacting with other users in VR via a microphone.
-
Users with a competitive edge seemed most interested in interactions with other users in VR.
06
Findings, Impact & Recommendations
After analyzing the data gathered during the usability test, we concluded several key findings (by task) for our study, which we presented to the team at HTC during a 60 minute presentation with Q&A.
In the presentation, we outlined each finding in association with the key task performed, provided design recommendations based on our findings and the severity of the issue and shared additional findings with the audience.
Some examples of high severity issues we discovered were:
Forced step-by-step instructions within the tutorial feel constraining and overwhelming.
-
5 out of 7 participants were frustrated at the lack of flexibility within the tutorial and wanted to explore the space on their own.
-
4 participants were overwhelmed by the audio instructions.
-
2 participants wanted to end the tutorial sooner, but were not able to discover how.
-
Recommendation:
-
Design a contextual tutorial, rather than a structured onboarding.
-
Let the users decide if they want to enter into a tutorial.
-
Add a skip and/or exit button, to let the user go at their own pace and end at any time.
Storage and presentation of items are unintuitive, causing confusion and delay when customizing avatars.
-
4 out of 7 participants were delayed because they were not expecting items to appear behind them.
-
It was not intuitive that items were stored in empty drawers. Displayed items were hard to discover for some participants because they exited the drawers very quickly, flying behind the participant and out of the participant’s field of vision.
-
It was difficult for the participants to turn around and select an item.
Recommendation:
-
Display all available customization options in front of the user, instead of behind.
-
Design a more natural setup of items associated with opening a drawer, for example display items in the drawers or on a hanger.
VR enables others to abuse communication channels and cyber-bully players, resulting in an unpleasant experience.
-
5 out of 7 participants felt uncomfortable having to endure verbal and physical abuse by other players.
- They did not know how to block the users and tried to run away.
- They commented that it made them more fearful of VR and less inclined towards wanting to join social VR environments.
Recommendation:
-
Offer tips on how to interact with other users, including how to block abusive players.
-
Add a “report user” option to the menu.
-
Enable users to quickly exit any negative encounters.
-
Include a training on identifying inappropriate behaviors and how to take actions.
