Week 2 - Multimodal Storytelling

What is multimodal Storytelling?

• Let’s break it down, multi – ok lots or many

• Modal this is how, technique or technology used

• So multimodal storytelling is storytelling working with different forms of media and techniques in order to tell a story, lets looks at some examples...

Thresholds

Thresholds by Mat Collishaw is VR photography exhibition. This immersive experience transports viewers back in time to a 19th-century setting, specifically recreating Talbot’s 1839 exhibition at King Edward's School in Birmingham.

Using VR headsets, participants can experience the exhibit with historically accurate details, from the architecture and exhibition setup to atmospheric elements like ambient sounds and flickering candlelight. The installation goes beyond traditional VR by incorporating sensory elements like temperature and even “ghostly” virtual moths, enhancing the illusion of time travel. Its a extremely fun concept and ties well into the time travel themes of this project.

Bandersnatch

Bandersnatch is an interactive film by Netflix, released as part of the Black Mirror series. Bandersnatch lets viewers make choices for the main character, who is a young programmer in the 1980s adapting a choose your own adventure book into a video game. Throughout the story, viewers are given options that influence Stefan's decisions, leading to multiple possible endings.

Research

For the research section of this lecture, our lecturer Emily suggested we go onto the MIT - Docubase website and look for at some examples that are relevant to our project or simply just interest us! Whist scrolling through the website I stumbled across Papers Please, a game that I remember being highly popular on YouTube back in the day.

Soundtrack

https://www.youtube.com/watch?v=xaaxfSwV_e8

One thing that immediately stood out to me was the Papers Please original soundtrack. It perfectly captures the bleak, oppressive atmosphere of the game's fictional Eastern Bloc country, Arstotzka. Created by composer Lucas Pope, the music is sparse and solemn, dominated by dark, droning brass and military-style percussion that evoke a sense of duty, struggle, and the weight of bureaucracy.

Storytelling

The storytelling in Papers Please is subtle and layered, drawing players into the world of Arstotzka through a combination of gameplay, visual storytelling, and moral decision. The story unfolds through the daily interactions and events you encounter as a border inspector, which makes the player feel like an integral part of the world.

At the core, Papers Please tells the story of your life as an immigration officer at a border checkpoint. The game progresses through daily shifts, with each day bringing new issues, policies, and characters. This approach builds immersion by slowly adapting the player into the routines of the job. It’s a simple, repetitive, but it starts to reveal more about Arstotzka and its people as you continue.

One thing I found interesting, is Papers Please use of “micro narratives.” As different characters approach your desk, each carries their own story. For example, you might meet a mother desperate to reunite with her children across the border, or a smuggler with a forged passport who offers you a bribe. These brief interactions allow the player to make choices with real consequences, where bending or strictly enforcing the rules could mean life or death. Because your decisions affect whether these characters succeed or fail, each small interaction feels significant, making you question your role in their lives and Arstotzka’s wider, authoritarian regime.

I find the storytelling within Papers Please extremely interesting. When it comes to my own project I believe the use of "micro narratives" could work really well with my ideas presented in week 1. For example, I have mentioned using items as a form of storytelling before, however, I could also incorporate NPCs who would tell their own story within the fictional world. The goal is to make the game feel alive, like everything that player does matters.

Oat The Goat

Oat the Goat is an interactive online storybook game developed by New Zealand's Ministry of Education to teach young children about empathy, kindness, and making positive social choices. The game is designed as an engaging way to help children navigate social scenarios where they may encounter bullying or unkind behaviour. Players help Oat decide how to respond in various situations, allowing them to experience the outcomes of kind versus unkind actions. I found the game extremely charming as we played through a small portion of it in the lecture.

"Your choice matters"

While writing about Bandersnatch, Papers Please and Oat The Goat, I started to think about "Your choice matters" games as a whole, as well as the ways in which they create immersion. By allowing players to make decisions that directly impact the storyline, characters, and the world, as well as giving players a sense of control and agency, "your choice matters" games create a great sense of immersion. When players see their choices reflected in how the games narrative unfolds, they feel personally invested, as if they are crafting their own journey. This dynamic involvement, where each choice carries meaningful consequences, encourages players to think carefully about their decisions, deepening the immersive experience. What I am trying to say here, I believe this type of storytelling would be fun to explore within this project. There is a lot of potential here in not only immersion but also in exploration of storytelling itself.

Eko Studio

Similarly to Twine, that we have looked at during week 1, Eko Studio is a platform for creating interactive, branching narrative videos. It allows creators to build stories where viewers can make choices that affect the outcome. Eko Studio allows creators to design nonlinear storytelling experiences without complex programming skills. The platform supports video, audio, and dynamic transitions which I found quite useful!

Update: It's with a heavy heart that I have to share this news - Eko Studio is no longer a thing. Bummer, I know. And because of that, I've lost access to the project I was working on there. (I seriously thought I had saved it, but I guess not!) Lesson learned - make sure you've got your files backed up in at least four different places. Better safe than sorry, right? Well at least I got a screenshot of the project I was working on, consisting of ominous stock footage and branching narratives. It is such a shame Eko Studio shut down, I was really looking forward to its features.

Testing

Towards the second part of the lecture I have decided to experiment a little bit in Roblox Studio. I had an idea of creating the games environment on Roblox as a place holder of what the game could look like. I have little to no experience using Roblox Studio, however, I figured it would be easier to experiment on rather than using big boy programs like Unity (at least for now).

In the end, I managed to create a hallway environment that manages to roughly represent what I have envisioned. Its simple, yet effective at fulfilling its simple purpose. I am happy with how it turned out, I found the lighting feature extremely fun to mess around with within the Roblox Studio. The whole process reminded me of building Legos in a way, assembling shapes, having all of them fall into place.