Floating in the sky - VR experience

• First person interactive story
• Lonely main character grab an umbrella and fly into the sky
• He found that the world is still beautiful and fill with hope.
• The umbrella symbols dream
• Use Sequencer to run the narrative story

Storyboard

1. Wander on the street
2. Hold and open the umbrella
3. Fly high to the sky
4. Floating from city to country
5. Fly through the forest and birds
6. Land and pass the umbrella to another lonely person
7. A lot of colorful umbrellas floating in the night sky, symbols we should face our dream and feel free to float away to our desire imagination of life

Interactions

Looking at a point trigger certain animation and generative objects

Rotate your head to see around the scenes

I get an asset from market place that is a slum city to test for my umbrella mechanic at the mean time.

Slum-City – assets in the market I found

Set the pawn with Camera to control the motion

Get Serial port signal from ESP8266 then send map it to speed, send osc signals

Camera Control & Unreal Blueprint

I create a camera on pawn and send the reference to the Umbrella Object, which contains an OSC receiver to get the value from ESP8266. The value will add local offset to the umbrella object, and for each event tick, I calculate the delta between the camera and the umbrella, let actor camera approach umbrella’s position gradually.

Use Osc Receiver to get speed value from ESP8266

I add a sound ball in the sky with the background track I made, it have great effect that when you approach certain height in the sky, you will heard the background track become louder and louder.

Current Progress – Control camera follow the motion of umbrella

Hardware Testing

For the second stage, I change my sensor to pressure sensor which allows me to get a feel of holding the umbrella. I think it really have a great difference when the sensor type and hardware match the user experience.

Use Pressure sensor to control the floating movement

Use Arduino Mega breakout to transmit the sensor value to MaxMSP through serial port

Sound Design

The sound track of this work is divided in to four sections - Wandering in the city, Doubting life, Grad the umbrella and fly into the sky and arrival at the dream land, I use String, Bell and Flute to compose a warm melody which express the feeling of the flying experience.

Update and improvement

Before the showcase, I update the control module in MaxMSP, use stream and median method to make the accelerometer value smoother. The signal received from Arduino Serial, then run through a slide object, than pass through zl.stream and zl.median, then convert to proper scale, finally send into unreal to control the flying motion of the umbrella.

I also rebuilt the Mountain landscape, add beach and use the ocean project plugin to make the sea. Besides, I put several event trigger e.g. the dead whale, the tallest tree on the mountain to let users explore the world more freely.

The Sea and the island

The updated and sorted blueprint of the floating umbrella

Demo Video