Attention Harvest

Interactive VR Experience

Thesis project in MFA Design and Technology program at Parsons School of Design - 2017

  Applicable Platform and Media: HTC Vive & Steam VR

Developing Software: Unity and Maya

Programming: Coded with Unity C#

  

About

Attention Harvest is a VR experience to critique the ubiquitous reality which nearly all social media and internet companies steal and sell their users' attention(time) to make profits. This first-person narrative experience lies in revealing one of the truth of the digital world - our life is fully entangled with distracting advertisements and disturbing information without us being aware of it.

Project Philosophy

In this virtual gaming world, it tracks not only the location of the player's gazing point but also the duration of their gazing time on each object. Furthermore, it saves those tracked eye movement data into a database. In a later time, it recalls those data to render intrusive objects to block player's view to impede them and delay their playing time intentionally. The disruptive interactions intend to simulate the concept of attention exploitation, of which the constant time-stealing from social media products to their every users by delivering commercial promotions onto their rich media service. This satirical experience aims to inspire and encourage players to reflect on the exploitative reality they are living with after played with it. 

​Project Documentation

Concept Development

Inspiration

This is an image contains 36 FB posts that I captured from my personal account in a random time. After I put them together, I realized 9 out of 36 posts are sponsored advertisings, which means one-quarter of the served contents are unattended information. 

I realized that every new medium - from Television to Internet companies has attained commercial viability and soon becomes very rich by turn themselves into an advertising platform. The attention merchants managed to reach us in previously unexploited times and places. Our attention is under the assault from commercial solicitation in virtually every waking moment. 

Domain Map

Applied User Interactions

The Diagram of VR Interactive Mechanism

Why VR -- Since I am making one project to simulate the sick reality now we are living in, VR is the perfect medium to create one experience to allow user immerse in. Besides, it is one simulation that the player diving in one exploitative world when the player wears the headset interacting with the virtual world, exactly as the project criticized - each player is living under the reality filled with endless and latent attention exploitation.

Why Gaze tracking -- Simulating the situation that the attention merchants have the ability to access netizens' online data no matter what we searched or what we saw online. It seems that the attention exploiters can achieve into individual's sight. Therefore, I use the gaze tracking to build the database for the following user interaction in this VR scene.

Why real-time interaction -- For collecting and analyzing user behavior and interaction first and give out the immediate feedbacks based on the previous data tracking. All of those for providing the player one scene that truly generates by and interacts with the player himself/herself.

Developing Process

User Testing

Testing the Storyboard

Testing the scene design

Refined Interactive Solutions Based on the Feedback of User Testing

User 1: The game mechanic should be more simple since the design intention is elaborating the data surveillance and attention exploitation at this age. I paid most attention to complete the tasks instead of comprehending the experience narration.

 

User2: More operating instructions needed for user-friendly. VR still is a cutting-edge medium, it is not easy to convey complicated thoughts via the unfamiliar platform in the first try. Possessing beneficial guidance is critical for delivering the concept thought.

The Diagram of Gaming Experience

Clarify the detailed workflow 

Developing Process in Unity

Storyboard

I drew the storyboard to clarify the experience narration and did the user testing to see if it is reasonable to deliver the concept. 

The Final VR Experience

Documentation of the Gaze Tracking Interaction

Once the game experience started, the game engine can detect if the player's is gazing on any objects by detecting the center point of the player's game view, and it saves those tracked eye movement data into a database for generating real-time objects to interact with players. For example, if users looked at certain objects, the game engine will render a virtual screen playing corresponding "advertisements" in the players sight view, to force them to watch, hence disturbing their playing experience. The longer they play with the game, the more "advertisements" they see in the game. The intention of the applied interactions is to mimic the massive promotions delivering in using social media products.

Setting of real-time interaction

Each player has 100 coins in the beginning, but the player will lose coins if they watch on the detected objects.

After 30 seconds playing time, the distracting item will be automatically loaded according to the back end ranking, in order to impede the player in the experience. 

 

When the interfering ads playing, no exploration or interaction allowed until it finishes. There are two rules about the interfering ads: 

1. the loading ads relevant to objects which the player paid more attention to in previous experience. 

2. the loading frequency is increasing with time passing

 

 

No matter the player finish the task first or lost all the coins in this visual space, the project will reveal the detailed watching time and how much coins spend on each ad representative, for revealing the outcome of data surveillance to users.

Playing outcome

VR interactive demo

Teleport effect

Game task

Impeding object

Exhibition at the NYC Media Lab Summit 

Contribution

The player’s behavior tracking and real-time rendering are for exposing the truth which of almost all platforms are accessing user’s online data and behavior to analyze their preference, in order to target them as various audience group to deliver ads in their offered service. Attention Harvest uses an interactive and immersive way to narrate and critique this true reality to its audience, to recall people to reflect the phenomenon of which individual’s cyber life under the control of the exploitative business model.

2017

COPYRIGHT ©2020 XIAOMENG TANG, All RIGHTS RESERVED.