In preparation for my presentation next month at University of Rochester’s Decoding the Digital conference, I’ve finally mapped out (with some help from reddead.wikia.com) the narrative structure of Red Dead Redemption. I wanted an easy way to show the nested structure of the story, and having seen a few Prezi presentations, I thought I’d give that a try. You might have seen the swooping, twisting, motion-sickness inducing form of presentation before, and I worry that it may become so overused that its the twenty-teens’ version of “death by PowerPoint.” But I think in this case it does a difficult job very well. Edit: hmmm I can’t seem to embed the presentation into wordpress, so here’s a link instead:
I’m excited because my first conference paper proposal has been accepted, and it gets financial support to help me go deliver it. So in September I’m off to the University of Rochester, NY for their Decoding the Digital conference. I thought I’d share the abstract here. Now, of course, I have to write the paper.
Abstract
The creators of digital narratives, in the form of computer games, are experimenting with form as they explore story telling in virtual spaces. Different approaches to so-called “open world” games all succeed in creating emotionally engaging diageses, three-dimensional virtual story spaces around which the player can wander with apparent freedom.
Cultural heritage institutions, including museums, built heritage, historic and ancient sites and heritage landscapes, have long been telling stories in three dimensions. Where it’s done well, visitors to those sites can immerse themselves in stories that they co-author as they make choices about what to look at first and subsequently and how deeply they want to explore individual points of interest.
Digital content creators have long had the opportunity to learn from heritage interpretation (Carliner 2001, Sylvester 2013), but what can cultural heritage institutions learn from computer games?
This presentation reports on early research comparing narrative approaches in digital games and cultural heritage institutions. Using case studies of open world games such as Red Dead Redemption, Dear Esther, and Skyrim, the presentation identifies different narrative techniques, structures and emotional triggers and seeks comparators in a number of UK cultural heritage sites. Highlighting the relative strengths of the digital and real-world media, the presentation discusses how cultural heritage sites might adapt some of the techniques of game narrative, including structure and music, to interpretive use. The results of an evaluation of a digital ludic interpretation case study, Ghosts in the Garden, at the Holburne Museum, Bath, illustrate the discussion.
The presentation concludes by setting out the plan for further research, including an exploration of adaptive narrative and the narrative braid (Hargood et al, 2012), and experiments with more considered use of music to trigger emotional responses at heritage sites.
In his book, Designing Games, Tynan Sylvester says:
If we look around, we find interactive narrative everywhere. Museums and art galleries are interactive nonlinear narratives where visitors explore a story or an art movement in a semi directed, personal way. Ancient Ruins and urban graffiti tell stories…
These interactive forms – museums, galleries, real spaces, and life – should be our first touchstones as we search for narrative tools. These older forms address our most fundamental challenge: creating a story that flexes and reshapes itself around the player’s choices, and deepens the meaning of everything the player does.
So here am I, trying to discover what museums and cultural heritage can learn from digital game design, and I find the lens turned back on me and my cohort. Sylvester points game designers towards the heritage industry (and elsewhere) in an attempt to dissuade them from focusing on cinema as the sole source of narrative instruction. My own studies arise from wanting to look beyond linear storyforms (text, film) which exert a strong influence on interpretation designers looking to engage visitors’ emotions. Games do a great job of getting players to care about watching mathematical algorithms choose what colour each pixel on a screen is. So thanks (on behalf of my profession) for the kudos, Tynan, but I think the learning can be two-way.
So, lets look at what Sylvester calls “the basic emotional triggers.”
Emotion through learning
Not just any old learning though. “If a lesson is obvious,” he says “there’s not much buzz in finally getting it because it was always fairly clear.” Instead, he advocates a moment of insight, where everything that has come before “clicks into place and reveals the shape of the whole.”
Emotion through character arcs
This is what film can do so well, engaging the audience’s empathy with one or more characters, as they face internal conflicts, grow and change.
Emotion through challenge
And this is the emotional trigger that we most readily associate with games,testing the the player’s dexterity and pattern learning before rewarding him or her not just with a sense of accomplishment, but progression within the game. But Sylvester argues that it is not as essential to games as it might appear, and cites Dear Esther as a game that can “create powerful emotions without players struggling.”
Emotion through social interaction
There’s still a perception of gamers as solitary types with no friends, but of course most games, not just team sports, are ways of bringing two or more people together.
Emotion through acquisition
As Sylvester acknowledges, gambling games are all about acquisition, and computer games often simulate the acquisition of wealth (or simply points). Of course gambling works in two ways, and the bitter emotions of loss shouldn’t be disregarded.
Emotion through music
Ah! This is one I’ve already discovered (and it occasions a warm internal glow as I read about it, my own little “emotion through learning”), though I’ll admit its not exactly rocket science. I like how Sylvester talks about it though:
And music is wonderfully subtle – even more than most emotional triggers. Nobody ever gives it the credit it deserves because nobody consciously pays attention to it during play. But even though the conscious mind is oblivious, the unconscious is still processing the music into a continuous flow of feeling. You can tell because music is easily separable from the rest of the experience. Listen to a game soundtrack by itself, and you’ll feel much of what you felt during play. Play the game in silence, and you’ll be surprised at how hollow it feels.
Emotion through spectacle, and, Emotion through beauty
In my own mind, these two are conflated, beauty being a form of spectacle. But Sylvester separates out the beauty of “a sunset over the ocean” from the spectacle of “a slow-motion dive to dodge an incoming rocket.” He also offers a warning not to over use either.
Emotion through environment
Touching upon people’s preferences for one environment other overs (which he suggests is at least in part, culturally conditioned) Sylvester also talks about the contrast between two environments. In one game, set in the tunnels of a post-apocalyptic Moscow underground, he describes a how play moved to the ruined, frozen, surface and the powerful effect it had on him as a player.
Though most would call Metro 2033 a shooter or RPG, I wouldn’t, because I don’t think its about shooting or roleplaying. I think its about discovering how a place like that makes you feel.
Emotion through newfangled technology
“The first few games with any new graphics, animation, or physics technology get an emotional rise from certain players,” but Sylvester warns new technologies can result in a reduction in the quality of games, as the designers learn how to best use the new capabilities available to them. As an example, he cites the introduction of CD-ROMs as a method of delivery, which in turn produced games with a lot of dreadful full-motion video cut-scenes.
Emotion through sexual signals
Another emotional trigger that comes with a health warning. Though easy to use, its heavy handed, just as spectacle and beauty can be (maybe it more properly belongs in a subset of those). “For the more serious or broadly targeted games, it’s often not worth being tasteless” he concludes.
You’ll notice, that change is a common factor for a lot of the above, and indeed, Sylvester declares that “the bedrock principle behind all emotional triggers is change”.
So, how does that list look to us in the the cultural heritage industry? Emotion through spectacle/beauty is something that my own organisation recognises. The full corporate name of the National Trust does after all continue with “for places of […] natural beauty.” Respondents to the Trust’s most beautiful places, like Ightham Mote do record a higher emotional impact than elsewhere. But does the National Trust as an organisation rely on beauty too much, as Sylvester accuses some games designers of doing? Good museums know how to use spectacle too, by creating “wow” moments often at the threshold of galleries. These can involve impressive exhibits, multimedia “shows”, interpretive “set design” or even the design of the spaces itself, such as the Great Court at the British Museum.
The impressive Great Court “wow moment”, at the British Museum (linked from British Museum website)
The Great Court being a free to enter space, is often used as a place to meet friends, and cultural heritage attractions are fully aware of their role as places for social interaction. Just like games, a heritage visit can be some you do alone, but the majority visit with others, partly to use the place as a backdrop, or springboard, for their own interactions.
Interpretation professionals are all about learning too. But are we too keen to reveal the whole story? Do we sometimes make it so easy to learn that our audience miss the thrill of discovering something for themselves?
Not every site has the sort of character that can take a narrative arc, but of course there are loads of sites that are associated with famous personalities, or where the lives of the less famous can be uncovered. Fictional, or archaeologically inferred characters, can be created with very clear character arcs. Real people’s lives are often messier, but internal conflict and change are often part of of what makes people famous, so emotionally engaging narratives can be woven from the threads of their lives. When the Imperial War Museum’s Cabinet War Rooms became the Churchill War Rooms, the emotional pull of one character was recognised. However, his life was full, and the part of the exhibition focussing on Winston Churchill describes so many character arcs that the emotional power of the story is diluted.
These triggers then: spectacle; beauty; social interaction; learning and character arcs, cultural heritage sites should be reasonably confident in delivering. There are also though some from Sylvester’s list which cultural heritage interpretation uses less confidently.
Challenge and acquisition are interpretive tools mostly confined to “children’s” or “family” trails and activities. At their most basic, such a trail will present a young visitor with a number of items to spot and check off against a list, a very simple game mechanic in which the child acquires sights (and possibly a small reward such as a badge, if they managed to fill every checkbox). More sophisticated activities, including interactives which are often short computer games, may present more challenges and of course at some sites there can be playful physical challenges.
A popular navigation challenge aboard Cutty Sark
Many cultural heritage sites are, by their very nature interesting environments. But, again because of their nature, it can be difficult to manipulate the environment to trigger emotional responses. Historic sites can’t be altered easily, and many sites are composed of just one environment type so its difficult to play with the contrast of two or more environments. Where the opportunity does exist though, the impact of transition between two environments can be increased with careful routing.
Visitor attractions prefer tried and tested technology to cutting edge newfangled technology. Technology needs to be robust and reliable to suffer the attentions of hundreds, or thousands (or hundreds of thousands) of visitors with varying degrees of manual dexterity. Technology in museums also tends to have a longer shelf-life than faddish games, so interpreters don’t want invest heavily in something that will turn out to be “sooo last year”.
Generally too, cultural heritage strives to appeal to as wide a cross section of the public as possible, so using sexual signals to manipulate emotion isn’t common. That said after an appropriate warning at the start, the sexual content of the British Museum’s Life and Death at Pompeii and Herculaneum exhibit had me grinning.
And, as I’ve discussed before, music is not well used at most cultural heritage sites.
Sylvester points out the challenge of working with emotions, because its so difficult to understand how emotional triggers work. He cites a number of experiments which demonstrate that, when asked to describe an emotional response, or the reasons for that response, people find it hard to say why they feels something. One famous example, by Dutton and Aron in 1974, seemed to show that young men thought they were attracted to a female researcher when in fact their pounding heart was due to being on a scary rope bridge.
Of course, as Sylvester goes on to say, this emotional misattribution makes music a great tool with which to manipulate how people feel. He describes what he calls “the Leonard Cohen Gravitas Moment” in TV drama:
It comes at the start of the third act of the show, when things are bad and it looks like all hope is lost. The dialogue stops, and a soulful or catchy song—often something Leonard Cohen-like—swells as the camera slides through a montage and a voiceover discusses the theme of the show. Viewers feel refreshed and contemplative. But they misattribute these feelings to the story when they actually come from the song.
I’m thinking about music. Which is slightly scary for me, as I’m not very good with music. I have no sense of rhythm, I’m not tone deaf, but I do struggle to tell the difference between notes, and though I enjoy singing, people around me don’t enjoy my singing. This might have something to do with two of my favourite musicians being Bob Dylan and Shane McGowan whose own singing voices are a matter of some division among critics.
But while I may not be terribly qualified to think about music, I have become aware of how important music is the storytelling that occurs in some of the most applauded video games. When I hear the words music and video games in the same sentence I think first of the god-awful bleeps and beeps that I used to turn down in the eighties, but music in games has come a long way since then. Something I think I was only actually aware of when I started playing Dear Esther for this research. I was so impressed with how the music added to the atmosphere, and helped tell the story that I was not surprised to learn that the composer, Jessica Curry, had been nominated for a Bafta for her work on the game.
Then, when I was telling people I was planning to play Red Dead Redemption, everyone I spoke mentioned the music as an impressive feature of that game, most pulling out one particularly impressive example, which indeed take the number two spot in this list of the top twenty songs in games. This is the first time in the game that the (excellent) ambient music gives way to a very “front of mind” song, licensed from Swedish (with South American roots) singer Jose Gonzales. The simple fact that so many people talk about this moment in their appreciation of the game indicates that the music contributes to an emotional, memory creating, response in the player.
We talked about this at work on Wednesday, briefly mentioning the way music is used in the Bowie exhibition at the V&A, and the hope that the experimental opening of Leith Hill Place might include an innovative soundscape. However we concluded that cultural heritage doesn’t use music enough in interpretation, and where it does, it doesn’t do so that imaginatively. My boss said she might be up for sponsoring an innovative (and repeatable elsewhere) use of music in interpretation. (So if anyone out there has an exciting ideas that would fit in a National Trust property around London and the South East, get in touch!)
It definitely seems to me that if I’m planning to learn from how games tell stories, I can’t ignore music. But I have some questions that need answering, and I think these are questions occasioned by the broad range of cultural heritage sites that my organisation, the National Trust, looks after though they would also apply elsewhere.
Many examples of music and sound in interpretation occur through headphones. This tends towards an insular, individual experience. Lots of people enjoy audio guides but many people seek a more social learning experience in museums. How can places use sound and music in a more open, participatory manner? (This is one of the questions the Ghosts in the Garden tries to address.)
Similarly, many people visit outdoor locations in part to enjoy the sounds of being in the open air. Can we design musical experiences that make space for, or even amplify some of the ambient sounds that may be occurring around the listener in the non-virtual world?
Lots of the music we hear in cultural heritage interpretation is bought off the shelf – existing recordings, licensed or borrowed from royalty free collections. Occasionally (for Ghosts in the Garden, for example) new recordings are made of music historically connected with the site. More rarely (I’m aware of a piece created especially for Ham House last year, and another in development at Mottisfont) have new pieces of music been commissioned to help tell the story of site. Why doesn’t this happen more often?
Luckily I don’t have to try and an answer these questions on my own. I’ve already met colleagues at the university that are already asking similar questions. The At Home with Music project has already worked at National Trust sites, and this post from Ben Mawson suggests he’s recently been dealing with exactly the same narrative frustrations that started me on this research. I’m hoping I can enlist their help, and that they don’t mind my singing.
So much easier when you know where the save function is! I completed Dear Esther, and managed to do some comparative different routes to the final scene too. Notice the singular there, you can’t change the ending of this story, or at least I’ve not discovered an alternative ending. And now I can read the other online comments about the ending, it doesn’t appear anyone else has either.
I won’t spoil it for you, other than to say the final scene is a satisfactory ending to an atmospheric, poetic, allegorical story.as to what was going on I’ve not much to add to the suppositions in my previous post, other than to wonder, at the last moment whether Esther and Donnelly were one and the same all along. Oh, and Paul might not be a drunk driver, rather a battered and unreliable car…
The Save function did allow me to explore the narrative structure in more detail, and what I observed in my previous post was confirmed, though there might be diversions from the main narrative path, they are but small distractions that will loop back to put you somewhere close to where you left the main story. You might hear a different narrative segment depending on the paths you take, but it won’t change your constructive interpretation that much, and you are also likely to hear the same segment on different routes through the story.
The strengths of Dear Esther are its atmosphere and sense of place. Which isn’t surprising, as that was the aim of its creator(s).Dan Pinchbeck, the game’s original author was researching presence (see below) when he wrote it. I didn’t comment before on the exquisite soundtrack, both effects and music. When there’s no music playing the sound of the wind, the water and the echo of your own footsteps, maybe even snatches of a (your?) voice, all contribute to a suitably spooky atmosphere for a game wherein you are on a deserted island where somebody has been lighting and AWFUL lot of candles. But when the fabulous music kicks in your engagement in the scene suddenly kicks up a gear.
I don’t know much about presence yet, but it comes from research into telepresence. The ISPR introduce presence as “as a sense of ‘being there’ in a virtual environment and more broadly defined as an illusion of nonmediation in which users of any technology overlook or misconstrue the technology’s role in their experience.” I think I felt one jolt of that while playing Dear Esther (which isn’t a critiscism, I hope, as I was almost always aware of keeping my finger on the “w” key to walk around the island). I don’t want to say too much about when I felt it, for fear of spoiling the moment for anyone else who reads this and wants to try the experience themselves. Suffice to say it involves an unexpected visit to the M5, a real place in very unreal circumstances. In fact the unreality of the situation amplifies the emotional engagement with the scene, much as the musical score does in other scenes. For a moment I stopped breathing, forgot where or who i was, and let my emotional surprise take over. For a moment only, less than a second I’m sure, before I remembered to locate the “w” key and press forward to investigate the scene…
Its funny isn’t it, that apparently ADDING to the mediation – music, magical takes on reality – seems to have to opposite effect to what one might presume, it can distract you from the medium, and immerse you in that narrative. Well, I say its funny but I expect better minds than mine have already unpack and explained that effect. I’ve got a a lot a reading to do, starting with Pinchbeck’s academic work, now that I’ve played his game.
I’ve been trying to find my way around Dear Esther today. It’s a narrative experience that first appeared as a mod for Half-Life (a First Person-Shooter), and which was then released as a stand-alone “game.” I’m told it takes two or three hours of play to complete, but I’ve been stymied today by the thing that gets me in FPS games – motion sickness. (Which is weird because I never suffer from it in the real world.)
So the game is taking me longer than two hours. That said, I’ve already begun to pick at the story structure, and I’m finding it more linear than I expected.
You start the game on the shore of a deserted (Scottish?) island, by an abandoned lighthouse. As you start, you hear (and read) a voice over (you?) with subtitles, introducing you to the Island and the mysterious Esther (maybe you are Esther?). I’ve restarted the game enough times to discover that there a number of different statements that you hear as you start. The atmosphere as you explore the island is very spooky, without (so far at least) any antagonists out to get you. If fact, it seems that you don’t interact in way way with the island except by walking around it, Occasionally when you reach certain points, you hear another voice over, so far I’ve heard about Ester’s birth, a car crash, Donnally – wrote a book that the narrator has stolen from the library and brought with him to the Island, and Paul, who may be the drunk driver that caused the crash. But I’ve noticed already that the environment isn’t quite the sandbox I’d hoped for – frustrating dead ends and impassible rocks or fences, guide you down a single path, being fed bits of narrative as you go.
What I’ve not yet worked out is are the narrative voice overs pinned to particular places? Or is there some algorithm that factors in time spent playing and what you’ve heard so far when it chooses what voiceover to feed you next?
The Interpretation Game 