Attention Harvest
Creative VR Experience
Thesis project in MFA Design and Technology program at Parsons School of Design - 2017
Applicable Platform and Media: HTC Vive & Steam VR
Developing Software: Unity and Maya
Programming: Coded with Unity C#
About
Attention Harvest is a VR experience to critique the ubiquitous reality which nearly all social media and internet companies steal and sell their users' attention(time) to make profits. This first-person narrative experience lies in revealing one of the truth of the digital world - our life is fully entangled with distracting advertisements and disturbing information without us being aware of it.
Project Philosophy
In this virtual gaming world, it tracks not only the location of the player's gazing point but also the duration of their gazing time on each object. Furthermore, it saves those tracked eye movement data into a database. In a later time, it recalls those data to render intrusive objects to block player's view to impede them and delay their playing time intentionally. The disruptive interactions intend to simulate the concept of attention exploitation, of which the constant time-stealing from social media products to their every users by delivering commercial promotions onto their rich media service. This satirical experience aims to inspire and encourage players to reflect on the exploitative reality they are living with after played with it.
Project Documentation
Concept Development
Inspiration
This is an image contains 36 FB posts that I captured from my personal account in a random time. After I put them together, I realized 9 out of 36 posts are sponsored advertisings, which means one-quarter of the served contents are unattended information.
I realized that every new medium - from Television to Internet companies has attained commercial viability and soon becomes very rich by turn themselves into an advertising platform. The attention merchants managed to reach us in previously unexploited times and places. Our attention is under the assault from commercial solicitation in virtually every waking moment.
Domain Map
Applied VR Interactions
Why VR -- Since I am making one project to simulate the sick reality now we are living in, VR is the perfect medium to create one experience to allow user immerse in. Besides, it is one simulation that the player diving in one exploitative world when the player wears the headset interacting with the virtual world, exactly as the project criticized - each player is living under the reality filled with endless and latent attention exploitation.
Why Gaze tracking -- Simulating the situation that the attention merchants have the ability to access netizens' online data no matter what we searched or what we saw online. It seems that the attention exploiters can achieve into individual's sight. Therefore, I use the gaze tracking to build the database for the following user interaction in this VR scene.
Why real-time interaction -- For collecting and analyzing user behavior and interaction first and give out the immediate feedbacks based on the previous data tracking. All of those for providing the player one scene that truly generates by and interacts with the player himself/herself.
Developing Process
Storyboard
I drew the storyboard to clarify the experience narration and did the user testing to see if it is reasonable to deliver the concept.
User Testing
Testing the storyboard
Testing the scene setting
User Testing Feedback
-
The game mechanic should be more simple since the design intention is elaborating the data surveillance and attention exploitation at this age. I paid most attention to complete the tasks instead of comprehending the experience narration.
-
More operating instructions needed for user-friendly. VR still is a cutting-edge medium, it is not easy to convey complicated thoughts via the unfamiliar platform in the first try. Possessing beneficial guidance is critical for delivering the concept thought.
Developing Process in Unity
The Final VR Experience
Documentation of the Gaze Tracking Interaction
Once the game experience started, the game engine can detect if the player's is gazing on any objects by detecting the center point of the player's game view, and it saves those tracked eye movement data into a database for generating real-time objects to interact with players. For example, if users looked at certain objects, the game engine will render a virtual screen playing corresponding "advertisements" in the players sight view, to force them to watch, hence disturbing their playing experience. The longer they play with the game, the more "advertisements" they see in the game. The intention of the applied interactions is to mimic the massive promotions delivering in using social media products.
The Diagram of Gaming Experience
Game Workflow
The Real-time Interaction Setting
• Each player has 100 coins in the beginning, but the player will lose coins if they watch on the detected objects.
• After 30 seconds playing time, the distracting item will be automatically loaded according to the back end ranking, in order to impede the player in the experience.
• When the interfering ads playing, no exploration or interaction allowed until it finishes. There are two rules about the interfering ads:
1. the loading ads relevant to objects which the player paid more attention to in previous experience.
2. the loading frequency is increasing with time passing
No matter the player finish the task first or lost all the coins in this visual space, the project will reveal the detailed watching time and how much coins spend on each ad representative, for revealing the outcome of data surveillance to users.
Playing Outcome
VR Interactive Demo
Teleport effect
Game task
Impeding object
2017
Exhibition at the NYC Media Lab Summit
