Personalising the heritage visit

One the things that my external examiner pointed out during my viva is that I had not put in enough about personalisation. A number of the articles that she recommended I look at for my corrections address that issue and this is one. Not, Elena & Petrelli, Daniela. 2018. Blending customisation, context-awareness and adaptivity for personalised tangible interaction in cultural heritage. International Journal of Human-Computer Studies 114: 3-19. doi: 10.1016/j.ijhcs.2018.01.001.

I think one of the reasons why I had somewhat skimmed over the subject in my thesis is, coming out of a career based on live interpretation, which is essentially a conversation, personalisation is built into the way I think about heritage. My work in the PhD might be summed up as trying to ensure that conversation takes place even when there isn’t an excellent live interpreter working there. Of course, my examiner, Daniella is coming at the same aim, but from a different direction, and so she does not take personalisation for granted as I am wont to do. This paper however is a useful resource looking at it from the HCI point of view and I think there will be a number of quote from it and a summary among my corrections. But which quotes? What is summarised? As you might already have guessed these posts are a first pass at answering those questions. A Bit of practise as it were before creating a restructured version of my thesis. So, the paper starts off asking what personalisation actually means in this context:

“‘Personalisation’ is a broad term that encompasses three types ofsystem behaviour: adaptability (also called customisation, the term we use hereafter) offers users a number of options to set up the application/system the way they like it; context-awareness is the ability of the system to sense the current state of the environment and to respond accordingly; adaptivity implies the system maintains a dynamic model of the on-going interaction and dynamically changes its own behaviour to adapt to the changing situation.”

I can see why they chose to call adaptability customisation, adaptivity sounds far too similar and might be confusing. Of course one factor we have to consider is that although people do make heritage visits on their own, the vast majority are as part of a group; self-organised like couples, families, or friendship groups; or organised groups such as school visits and coach tours. As the authors point out “Research that directly addresses the social dimension is still limited” but they point the way to studies that look at conversation around a context aware-table, and sharing tables around a group, among others. However it is important to include this social dimension in any consideration of personalisation, which is something I did at Chawton – the choice there were made by the visiting “group” even if, sometimes that ‘group’ consisted of one person.

The paper of course starts with some case studies of similar work, including the Italian trenches soundscape I looked at a couple of days back. In another project at the same museum uses a “pebble” with NFC capability that activates media when places in certain places around the museum. When the visitor leaves, the pebble’s journey is read and a personalised postcards printed for the user to take home. In the Hague, a similarly NFC enabled system has the user place replica objects in “an interactive ring” which plays media from a choice of three different viewpoints (two military and one civilian). A third project, The Loupe, uses a phone disguised as a magnifying glass to present AR media. My problem with all these is an HCI one, two these systems force the users to learn a new interface, placing the pebble or replica in a certain spot to activate media that seems unintuitive, especially in environments where conditioned behaviour often precludes touch, picking things up, or even putting things on museum surfaces. On the other hand, the authors do make a point later that “A synergy can be created with tangible and embodied interactions to increase visitors’ awareness they are building their own visit path.” And I must admit that when the personalisation is invisible, the visitors do not perceive it. However my evaluation of Ghosts in the Garden suggest that even when tangible interactions are involved, the visitors may still not be aware that their experience was personalised.

But leaving my issues aside. There is some really good overview stuff in this – including a table that summarises some of the factors to consider when personalising interpretation. This includes: “stable” visitor factors – like age and disability, interests and Learning preferences; factors related to the current visit – motivations, fatigue, visit history and available time etc.; the type of tracking – two in this table, proximity tracking and interaction with objects; the location – indoor or outdoors, layout, noise etc.; and the content – the media, the story.

The team brought together 25 participants in a co-design workshop (curators , computer scientists and engineers) and they came up with a classification of features by the type of personalisation they support. The first group includes features that depend on content and are activated by “customisation preparation”: is this about on-site visits or virtual visits? Is it indoors or outdoors? What are constraints – is there power and wifi? The next set is decided by the curator or interpretation staff, Most of these come under “customisation preparation” too: what is the heritage topic?; The media type?; the genre of the text?; The thematic threads?; the supported visitor profiles? the type of group? Then what is the structure of the narrative, for example a story or a Q&A? Finally what is the structure of the visit? For example, is it guided, free exploration, or a treasure hunt?

One curatorial decision that falls into context awareness is does the interaction involve augmented objects, an if so wha are input and output abilities of those objects? were I the curator on this project I would look for forms of content awareness which do not rely on objects, even though some I have written about elsewhere are fun. But that move me into a set of context awareness features that are modelled by the system itself (according to the authors): user location, proximity to exhibits, proximity to other users, and the current state of the exhibits. To give the experience the adaptivity it needs, the he system will also use data about the shorty of the individual interactions with the space/objects and delivered content – just as in my Chawton experiment the system selected content based on what had been shared with the user before.

Finally comes the customisation choices, chosen, or course by the visitor and based on their motivation and expectations. They might have been given the opportunity to express interest in topics and narrative threads, as I did (somewhat clumsily) with my Clandon prototype. And, as at Chawton, the expected a duration of the visit is a factor (though I suggest it is less an active choice of the visitor, and better modelled by the system). Of course another factor that is totally out of the control of anyone other than the visitor is what the visitor thinks the type of visit is – they might be coming for an emotional reason, or social, or fun or for learning.

The paper concludes that “fully automatic adaptivity, where the system takes all the decisions on what to present to which visitor, when and how, may not be the best solution” and argues that therefore what curators (or interpreters) value as most meaningful should be the driver of of the personalisation model. I agree, but with the proviso that if the intent is top emotional engage the visitor, many heritage stories don’t do the job well enough. The authors say “This requires a radical rethinking of how personalisation in cultural heritage manifests itself and the role curators and visitors play” and I think think that my thesis might contribute to that rethinking.

Simulating ideology in storytelling

The Story Extension Process, from Mei Yii Lim and Ruth Aylett (2007) Narrative Construction in a Mobile Tour Guide

Another great piece from Ruth Aylett, this time from 2007. Here, she and collaborator Mei Yii Lim are getting closer to what I’m aiming for, if taking a different approach. They kick off by describing Terminal Time, a system that improvises documentaries according to the user’s ideological preference, and an intelligent guide for virtual environments which take into account the distance between locations, the already told story, and the affinity between the the story element and the guide’s profile when selecting the next story element and location combination to take users to. They note that this approach could bring mobile guides “a step nearer to the creation of an ‘intelligent guide with personality'” but that it “omits user [visitor] interests”. (I can think of many of a human tour guide that does the same). They also touch on a conversation agent that deals with the same issues they are exploring.

This being a 2007 conference paper, they are of course using a PDA as their medium. Equipped with GPS and text to speech software, a server does all the heavy lifting.

“After [an ice-breaking session where the guide extracts information about the user’s name
and interests], the guide chooses attractions that match the user’s interests, and plans the shortest possible route to the destinations. The guide navigates the user to the chosen locations via directional instructions as well as via an animated directional arrow. Upon arrival, it notifies the user and starts the storytelling process. The system links electronic data to actual physical locations so that stories are relevant to what is in sight. During the interaction, the user continuously expresses his/her interest in the guide’s stories and agreement to the guide’s argument through a rating bar on the graphical user interface. The user’s inputs affect the guide’s emotional state and determine the extensiveness of stories. The system’s outputs are in the form of speech, text and an animated talking head.”

So, in contrast to my own approach, this guide is still story lead, rather than directly user led, but it decides where to take the user based on their interests. But they are striving for an emotional connection with the visitor. So their story elements (SE) are composed of “semantic memories [-] facts, including location-related information” and “emotional memories […] generated through simulation of past experiences”. Each story element has a number of properties, sematic memories for example incude: name ( a coded identifier); type; subjects; objects; effects (this is interesting its lists the story elements that are caused by this story element, with variable weight); event; concepts (this that might need a further definition when fist mentioned); personnel (who was involved); division; attributes (relationship to interest areas in the ontology); location; and, text. Emotional story elements don’t include “effects and subjects attributes because the [emotional story element] itself is the effect of a SE and the guide itself is the subject.” These emotional memories are tagged with “arousal” and “valence” tags. The arousal tags are based on Emotional Tagging, while the valence tag “denotes how favourable or unfavourable an event was to the guide. When interacting with the user, the guide is engaged in meaningful reconstruction of its own past,” hmmmmm.

So their prototype, a guide to the Los Alamos site of the Manhatten project, the guide could be either “a scientist who is interested in topics related to Science and Politics, and a member of the military who is interested in topics related to Military and Politics. Both guides also have General knowledge about the attractions.” I’m not convinced by the artifice of layering onto the interpretation two different points of view – as both such are being authored by a team who in their creation of the two points of view will, even if striving to be objective, will make editorial decisions that reveal a third, authentic PoV.

When selecting which SE to tell next, the guide filters out the ones that are not connected to the current location. Then “three scores corresponding to: previously told stories; the guide’s interests; and the user’s interests are calculated. A SE with the highest overall
score will become the starting spot for extension.” The authors present a pleasingly simple (for a non-coder like me) algorithm for working out which SE goes next. But the semantic elements are not the only story elements that get told. The guide also measures the Emotional, Ideological story elements against the user’s initial questionnaire answers and reactions to previous story elements and decides whether or not to add the guide’s “own” ideological experience on to the interpretation, a bit like a human guide might. So you might be told:

“Estimates place the number of deaths caused by Little Boy in Hiroshima up to the end of 1945 at one hundred and forty thousands where the dying continued, five-year deaths related to the bombing reached two hundred thousands.”

Or, if the guide’s algorithms think you’ll appreciate it’s ideological perspective, you could hear:

“Estimates place the number of deaths caused by Little Boy in Hiroshima up to the end of 1945 at one hundred and forty thousands where the dying continued, five-year deaths related to the bombing reached two hundred thousands. The experience of Hiroshima and Nagasaki bombing was the opening chapter to the possible annihilation of mankind. For men to choose to kill the innocent as a means to their ends, is always murder, and murder is one of the worst of human action. In the bombing of Japanese cities it was certainly decided to kill the innocent as a means to an end.”

I guess that’s the scientist personality talking, perhaps the military personality would  instead add a different ideological interpretation of the means to an end. As I mentioned before, I’m not convinced that two (or more) faux points of view are required when the whole project and every story element that the guide gets to choose from are already authored with a true point of view. But in many other aspects this paper is really useful and will get a good deal of referencing in my thesis.

Trapped in the bunker

The debrief session after the experience. Photo taken by Adam.

Last weekend at Geek 2017, I played Sarcophagus, a Nordic LARP (Live Action Roleplay). LARP, as we know it today, grew out of the international popularity of Dungeons and Dragons and other tabletop roleplaying games. It didn’t take long, in the late seventies and early eighties, for people to start making costumes out of blankets and swords out of camping mats and gaffer tape, to take the game away from the table and into the real world. Indeed, I played in one of the first commercial LARPs, Treasure Trap at Peckforton Castle.

People still do that of course, and the weapons have become better made, safer and more “realistic” in their design (with in the spiky aesthetic of fantasy illustration). LARP is not limited to high fantasy genres either. In the nineties there was an explosion of gothic LARP in the US, with people playing communities of Vampires. Any world can be recreated in LARP form, even the real one.

The popularity of LARPing in Scandinavia, led to style/variant with its own name, Nordic Larp (note, in this form, Larp has become a word in its own right, no longer an acronym). This style has gained an international reputation for attempting something more than recreating fantasy adventures, exploring its possibilities as an art form in its own right. Similarly, the form often eschews external, procedural adventures in favour of exploring internal, emotional struggles.

Thus it was that I, and a dozen or so other players (though we’ll return to the ideas of games and players shortly), signed up to a five hour experience that would involve us being locked up in a nuclear bunker with no hope of escape. The players ranged from Larp virgins to experienced Larpers from Belarus, a Canadian musician, lecturers and students. (Three of us are doing PhDs.) We weren’t in that bunker the whole five hours though, and given that I’m writing this, you’ll understand we were let out. But this isn’t like an Escape room game, where we have limited time to solve the puzzles and find a way out, neither is any of us expected to win by becoming the king of a post-apocalyptic society. (Though, as we’ll see, my character might have thought so.) In fact its arguably not a game at all. Our facilitator, artist Adam James, kept correcting himself when he used the word game, explaining that his Larping mentor disapproved of it. So we are not players in a game, but rather players in an improvised drama, and not just the players but the audience as well. Indeed, Adam defined larp as an artform where the participants and the audience are one and the same. (Which is something Robin Laws used to say about tabletop roleplaying games, though he’s had to drop that definition as the streaming of such games on Twitch and YouTube has become more and more popular.)

The object of Sarcophagus is not to escape or win, but to explore the five stages of grief, as modeled by psychiatrist Elisabeth Kübler-Ross in her book On Death and Dying. First all though we had to work out who we were.

The preparation took a couple of hours in itself, with Adam explaining Larp, running us through some improvisation exercises and helping us create a character for ourselves. To begin with, he littered the briefing room with pictures, some large (A3) some small. Some abstract, some figurative, some photographs, some drawings. Adam encouraged us to walk among them, pick them up, and see if any of them inspired thoughts about who we might be.

I latched upon a couple, a medieval drawing of a ploughman and a dead bird which may have been trussed into some sort of totem, and imagined the pastural post-apocalyptic fantasies of my youth, things like John Christopher’s Tripods and Sword of the Spirits trilogies, the seventies BBC series Survivors, and Riddley Walker (which is set very close to where we were playing). What about a character who’d actually looked forward to the end of the world, I thought, a character who’d foolishly imagined a clearing away of everything what was wrong with the world, and a return to simpler, purer times. Of course he would be disappointed, Adam had made it clear that we’d know we wouldn’t even survive long enough to see the nuclear winter, let alone a new Garden of Eden.

Talking about it with other players, I imagined that my character might be rather reactionary, not necessarily well liked. Then Adam passed around a “hat” with little slips of paper listing the jobs we might have had before we found ourselves in the bunker. We found out we were hairdressers, waiters, sailors, stockbrokers and gym teachers. I pulled out “politician” and everything fell into place. I didn’t once mention the word UKIP once, indeed I painted myself as someone who, like Churchill, had moved between parties, but everyone knew exactly what sort of politician I had been. We also got to pick a flaw out of the hat. I ended up an alcoholic, though frankly I felt politician was enough of a flaw.

Then we split into two groups. One of people that wanted close connections with other player’s characters (for example, sisters), and a slightly smaller group who preferred thinner, looser connections. I went with the second, thinking many will have seen my politician on television, but few would know me well. We workshopped our connections, the gym teacher had taught my children, the security guard was someone I ignored on the way into the gated community to lived in, neighbour to the stockbroker, and the waiter was the rebellious son of wealthy donors to my political campaign.

Some more improvisation followed, this time walking as our characters, in different situations. Then after a break, and some meditation on what the world would be like if we died today, we met up outside the Bunker for one last improvisation workshop before going into (in this case) as reconstruction of a WWII Anderson shelter. Adam asked us to find a position to start then read out a short introduction:

11:27—Radio and TV broadcasts are interrupted by breaking news. Brussels and Copenhagen have been hit by large explosions.

11:31—Unconfirmed sources report that the blasts may be nuclear explosions.

11:33—Similar explosions are reported in Paris, Stockholm and Dublin.

11:37—A black helicopter lands on the roof of the prime minister’s office in Oslo London.

11:38—Associated Press confirms that Copenhagen has been hit by a nuclear explosion.

11:40—The air alarm goes off in London. Most people haven’t heard the news and just think it is an exercise. A few run to the shelters.

11:41—TV and radio transmissions are jammed.

And we begin. I won’t go too much into what happened, every version is different. Suffice to say, my politician tried to build a power-base in within the group, before properly realising the hopelessness of the situation, and even then trying to get people to like him – I’d managed to keep my bottle of booze (creme soda) secret until, hopeless, I shared what was left to curry favour. The only scripted moments were that introduction above, the lights going out in three phases before ending in darkness, and one event that one of the players is previously given guidance on timing. Note the importance even in this mostly improvised story, of having kernels – events that happen in order, even if its only the lights failing and the (spoilers!) event.

Afterwards there was a (vital) debrief session, almost a decompression chamber. We had time to get happy again, and use to the light of day and discussed how our story developed. Adam observed that we hadn’t quite entered the depression stage when he had to bring it to its conclusion, and I thought we hadn’t quite had time. I felt my character had been bargaining to the last, and was teetering on the brink of depression when Adam came through with the torch and rounded us up. I was about to say something to that effect when Adam let slip that we were meant to spend four hours, not two in the bunker.

Due to constraints of the day, we always knew we were only going to have two hours, but I would have liked to have played the longer stint if it were possible. And of course it is! If I get to play again, it will be a totally different experience. I won’t be a politician again, maybe I’ll be a historian, or hairdresser, and the stor(ies) will be, even if the lights still go out and “the event” happens as before.

I even thought about playing it in the bunker that the National Trust owns, on Orford Ness. Now, that would be cool.

Adventures in Moominland

In the evening after I visited the V&A, I’d managed to bag the last ticket for the last entry slot that day of the exhibition at the South Bank Centre. I’m a Moomin fan, having read all the books when I was a child. I’d already seen much of what was on show at the public Library in Tampere, Finland, which is the guardian of most of the Tove (best pronounced something like “Toover”) Jansson archive. The South Bank exhibition, Adventures in Moominland, takes advantage of the Tampare collection moving to a new home there in May to borrow part of the collection for a similar presenation to last year’s very successful Wondercrump World of Roald Dahl. Just like that successful family exhibition, this one also uses the author’s work to explore the life of the author.

Tove Jansson’s life was not entirely happy, she grew up during civil war in Finland, with loving parents whom she loved and respected in return. But while she was a progressive left winger, her father sided with the Fascists. Not only that, she had a hard time coming to terms with her lesbian sexuality. These are some quite challenging concepts to share with an audience as young as might visit this show, but the curators and interpreters did a very good job of it.

I wouldn’t be surprised if this exhibition sees less actual children than last year’s. While Dahl’s popularity has endured with every generation, in the UK Moomins grabbed the imagination of a good part of my generation in the seventies, and had brief surges of popularity with a couple of later children’s television series, but it seems to me that they are best known by people of my age, and not so quickly recognised by younger generations (except, I’m willing to bet, by the children of older fans – I know I read all the books to my own children).

The group assembled for my tour were, apart from one young fan, all adult. It’s harsh of me to say its what I expected – it was after all the last tour of a workday in term-time, so even the home-schoolers will have likely gone home. (And I bet, given the lack of school in the novels, that there is a correlation between homeschooling parents and Moomin fans, but I digress.) The format followed Wondercrump’s successful formula. An introductory talk from a host who warned of scary dark experiences ahead, but reassured us that though the Moomins often had scary adventures, they always ended happily. “Apart from Moominvalley in November” I said. Then she handed over to a guide, a young woman dressed in such a style as to resonate with Jansson’s illustrations of the Mymble, Fillyjonk and Toft, etc., without actually trying to be one.

The guide led us in and as with Wondercrump, we discovered she was playing a two-hander with a disembodied voice. This time, it was Sandi Toksvig – guess Danish is a bit like Finnish. So so we progressed with these two guides, recorded and live, through spaces that evoked Sniff’s cave, Snufkin‘s tent, the woods of Moominvalley, a raft like that in Comet in Moominland, Moominpapas lighthouse island, and the Moominhouse itself. In each space the guide and exhibits focussed (mostly) on one of the books, and explained what each book had to reveal about Tove’s life. Even the youngest reader will recognise (even if they might struggle to put it into words, as I did when I was seven or nine or whatever) that the novel series (there were a couple of picture books too, but I only found those as an adult) start out as outward facing proceedural adventures but become more inward looking, dramatic, and psychological with each publication. It can put some young readers off the later books, but those who persevere have their first introduction to existentialism.

The South Bank adventure doesn’t follow the books in order, but structures the story about Jansson coming to terms with her sexuality and acknowledging her love for her life-long partner Tuulikki. Moominland Midwinter thus becomes the climax of the exhibition’s story. In the novel young Moomintroll wakes up early from hibernation, in strange new snow-covered world, which he doesn’t like at all, until he meets Too-ticky, the girl who shows him its wonders. I admire the curators’ resistance to chronology in favour of a more satisfying emotional journey, preceding Midwinter with the loneliness of Moominpappa at Sea, which actually came eight years later.

Maybe wisely, they avoid the last novel Moominvalley in November, altogether. As a young reader, this was the most important work for me. Taking place after the Moomins have left for the lighthouse island in Moominpappa at Sea, the other inhabitants of the valley move into their house and try to recreate their life, waiting for them to return. But they never come back. Jansson’s mother had died while she was writing this final novel, and she is said to have said she “couldn’t go back and find that happy Moominvalley again.” I can’t quite explain the emptiness that fills my chest, even as as an adult, as I remember finishing the novel as a child. I am convinced it was a vital moment, maybe the very first step in my journey to being a grown-up.

So, I wrestle with some dissatisfaction that the experience didn’t feature my favourite work, while at the same time being impressed with the effectiveness of their story construction. I was also even more impressed with the “mixed media” approach of exhibit, lighting effects, audio commentary and sound and live guide. In a way its a more scripted version of what I’m trying (and currently failing, it feels) to do at Chawton.

I feel it might be a technique that other places (and yes I’m thinking of the National Trust places I work with) should experiment with.

Resonance: Sound, music and emotion in historic house interpretation

Just drafted an abstract for my Sound Heritage presentation:

This presentation explores what computer games can teach us about emotional engagement in cultural heritage interpretation. Beginning with a model of emotional affect drawn from the work of Panksepp and Biven (Panksepp, 2012), Lazarro (Lazarro, 2009), Sylvester (Sylvester, 2013)and Hamari et al (Hamari et al., 2014), it reveals how music especially has become a versatile emotional trigger in game design.

Drawing on the work of Cohen (Cohen, 1998)and Collins (Collins, 2008)eight functions that music has in games:

Masking – Just as music was played in the first movie theaters, partly to mask the sound of the projector, so music in new media can be used to mask the whir of the console’s or PC’s fan.

Provision of continuity – A break in the music can signal a change in the narrative, or continuous music signals the continuation of the current theme.”

Direction of attention – patterns in the music can correlate to patterns in the visuals, directing the attention of the user.

Mood induction; and,
Communication of Meaning- the nice distinction here is between music that makes the user sad, and music that tells the user “this is a sad event” without necessarily changing the user’s mood.

A cue for memory – The power of the music to invoke memories or prepare the mind for a type of cognitive activity is well recognized in advertising and sonic brands such as those created for Intel and Nokia.

Arousal and focal attention – With the user’s brain stimulated by music s/he is more able to concentrate on the diagesis of the presentation.

Aesthetics – The presentation argues that all too often music is used for aesthetic value only in museums and heritage sites, even if the pieces of music used are connected historically with the site or collection.

As an example, the presentation describes a project to improve the way music is used in the chapel at the Vyne, near Basingstoke. Currently, a portable CD player is used to fill the silence with a recording of a cathedral choir, pretty, but inappropriate for the space and for it’s story. A new recording is being made to recreate about half an hour of a pre-reformation Lady Mass, with choisters, organ and officers of the church, to be delivered via multiple speakers, which will be even more pretty but also a better tool for telling the place’s story.

With a proposed experiment at Chawton House as an example, we briefly explore narrative structure, extending the concept of story  Kernels and Satellites described by Shires and Cohan (Shires and Cohan, 1988)to imagine the cultural heritage site as a collection of narrative atoms, or Natoms (Hargood, 2012), both physical (spaces, collection) and ephemeral (text, video, music etc.). Music, the presentation concludes is often considered as a “mere” satellite, but with thought and careful design there is no reason why music can not also become the narrative kernals of interpretation.

 

COHEN, A. J. 1998. The Functions of Music in Multimedia: A Cognitive Approach. Fifth International Conference on Music Perception and Cognition. Seoul, Korea: Western Music Research Institute, Seoul National University.

COLLINS, K. 2008. An Introduction to the Participatory and Non-Linear Aspects of Video Games Audio. In: RICHARDSON, J. A. H., S. (ed.) Essays on Sound and Vision. Helsinki: Helsinki University Press.

HAMARI, J., KOIVISTO, J. & SARSA, H. Does Gamification Work? — A Literature Review of Empirical Studies on Gamification.  System Sciences (HICSS), 2014 47th Hawaii International Conference on, 6-9 Jan. 2014 2014. 3025-3034.

HARGOOD, C., JEWELL, M.O. AND MILLARD, D.E. 2012. The Narrative Braid: A Model for Tackling The Narrative Paradox in Adaptive Documentaries. NHT12@HT12. Milwaukee.

LAZARRO, N. 2009. Understand Emotions. In: BATEMAN, C. (ed.) Beyond Game Design: Nine Steps Towards Creating Better Videogames. Boston MA: Course Technology / Cangage Learning.

PANKSEPP, J. A. B., L. 2012. The Archaeology of Mind: Neuroevolutionary origins of human emotions, New York, W. W. Norton & Company.

SHIRES, L. M. & COHAN, S. 1988. Telling Stories : A Theoretical Analysis of Narrative Fiction, Florence, KY, USA, Routledge.

SYLVESTER, T. 2013. Designing Games – A Guide to Engineering Experiences, Sebastolpol, CA, O’Reilly Media.

Mulholland on Museum Narratives

Working on the narratives for the Chawton Project, I’m taking a break and catching up on reading. Paul Mulholland (with Annika Wolff, Eoin Kilfeather, Mark Maguire and Danielle o’Donovan) recently contributed a relevant first chapter to Artificial Intelligence for Cultural Heritage (ed Bordoni, Mele and Sorgente).

Mulholland et al’s chapter is titled Modelling Museum Narratives to Support Visitor Interpretation. It kicks off with the structuralist distinction between story and narrative,  and points to a work I’ve not read and should dig out (Polkinghorne, D. 1988 Narrative Knowing and the human sciences) as particularly relevant to interpreting the past. From this, the authors draw the “narrative inquiry” process which “comprises four main stages. First, relevant events are identified from the historical period of interest and organised into chronological order. This is termed a chronicle. Second, the chronicle is divided into separate strands of interest. These strands could be concerned with particular themes, characters, or types of event. Third, plot relations are imposed between the events. These express inferred causal relations between the events of the chronicle. Finally, a narrative is produced communicating a viewpoint on that period of history. Narrative inquiry is therefore not just a factual telling of events, but also makes commitments in terms of how the events are organised and related to each other.” Which is as good and concise a summary of the process of curatorial writing as I am likely to find.

There’s another useful summary paragraph later in the document. “When experiencing a museum exhibition, the visitor draws relationships between the exhibits, reconstructing for themselves the exhibition story (Peponis 2003), whether those relationships are, for example, thematic or chronological. The physical structure of the museum can affect how
visitors perceive the exhibition narrative. Tzortzi (2011) argues that the physical structure of the museum can serve to either present (i.e. give access to the exhibition in a way that is independent from its underlying logic) or re-present (i.e. have a physical structure that reinforces the conceptual structure of the exhibition).” Tzortzi there is another reference I’ve not yet discovered and may check out.

What the paper does not do however, is make any reference to emotion in storytelling. the authors seem to leave any emotional context the the visitors’ own meaning making. The chapter include a survey of current uses of technology in museums, and academic experiments including virtual tour guides and opportunities to add the own interpretations and reminiscences, as well as web-based timelines etc.

But, digital technology gives us the opportunity (or need) to break down cultural heritage narratives even more, and an earlier (2012) paper by (mostly) the same authors, Curate and Storyspace: An Ontology and Web-Based Environment for Describing Curatorial Narratives describes a system for deeper analysis. (Storyspace turns out to be a crowded name in the world of writing tools and hypertext, so eventually the ontology and Storyspace API became Storyscope). The first thing that the ontology brings to the table is that

a curatorial narrative should have the generic properties found in other types of narrative such as a novel or a film

So the authors add another structuralist tool, plot, to the story/narrative mix. “The plot imposes a network of relationships on the events of the story signifying their roles and importance in the overall story and how they are interrelated (e.g. a causal relationship between two events). The plot therefore turns a chronology of events into a subjective interpretation of those events.” But using the narrative inquiry process “the plot can be thought of as essentially a hypothesis that is tested against the story, being the data of the experiment.”

I like this idea. But its worth distinguishing between the two uses of the word “interpretation” in cultural heritage. The first use, familiar to my archaeologist colleagues, describes the process of building an understanding of of aspect of the past from the available evidence. The second, more familiar to my museum and heritage site colleagues describes the process of explaining the evidence to non-professional visitors. At its very best, the museum/heritage site form of interpretation will resemble and guide visitors though the process of inquiry that builds an understanding of the evidence on display. But most of the time the second form of interpretation more closely resembles storytelling. That’s not a fault or failure of my museum/heritage site colleagues, most visitors are time poor in story rich environments. But digital technology has the potential to allow museum and heritage site interpretation to more closely resemble the first use of the word.

What digital technology offers, is the opportunity for brave curators to offer alternative plots, or theses, and test them in a public arena, rather than just through a peer review process. Or even to create plots procedurally by following the visitors’ path of attention between objects, maybe discovering plots the curator had not imagined.

The two experiments that the authors describe go someway towards this, by their dry ontology misses an emotional component. The event ontology could surely include an authorial opinion on whether the narrative element suggests a simple emotional reponse (even as simple as hope or fear) but instead “If the tag represents an artist, then events are used to represent, for example, artworks they have created, exhibitions of their work, where they have lived, and their education history.” Dry, dry facts… There is the tiniest nod towards, if not emotion per se, the some sort of value in their brief discussion of theme:

Theme is also related to the moral point of the story. This could be a more abstract concept, such as good winning through in the end, which serves to bind together all events of the story.

Given that they say “Narratives are employed by museums for a number of purposes,
including entertainment” they haven’t given much time to what makes narratives engaging. There is hope however. In their conclusion, they do say “Other narrative
features such as characterisation and authorial intent could potentially be
foregrounded in tools to support interpretation.”

 

Interactive story beats

In my exploration of interactive storytelling I’ve concentrated on computer games, because I’m exploring the digital delivery of story. But I’ve already decided that for my experiment at Chawton next year, I’m going to “wizard of Oz” it – use actual people instead of trying to write a computer program to deliver the interactive narrative.

I’ve been thinking about the issues around that. People are natural storytellers, though some are better than others, so I have a double edge problem. As I recruit and train people to be my “wizards of Oz”, I need to train the poor story-tellers to be better, and weirdly, I need to train the great storytellers to be worse! My reasoning is this, I want to prototype what a computer might do, there’s little or no experimental value in simply enhancing a great storyteller’s natural ability with some environmental bells and whistles. So part of what I’m trying to learn is about how to systematize (is that a word? It’ll do) story.

I’ll explain about Kernels and Satellites of course, but I need (I think) some sort of simple system of identifying how different story elements might fit into the emotional journey the visitor is going to take.

So, I’m reading Robin D. Laws‘ Hamlet’s Hit Points. Laws is a game designer but mostly of tabletop, or “pen and paper” role-playing games (though he has written for some computer games too). This book attempts to systematize (I think it is a word) story, with an audience of role-playing gamers in mind. I think it may be useful for me, because it attempts to train the Game Master of such games (the “referee” who, together with the players, makes the story) to be aware of the emotional impact of each scene or action (which he calls, using a screen-writing term, “beats”) on the players, and better choose which element to serve up next to keep everyone emotionally engaged. Tabletop Roleplaying Games must be the most interactive, responsive, stories ever created. In a way, my “wizards of Oz” will be like a Game Master, not telling a story they prepared earlier, but working with their visitors to create a story on the fly, but keep it emotionally engaging.

In a handy short opening chapter called “How To Pretend You’ve Read This Book” Laws explains “With its system of beat analysis, you can track a narrative’s moment-to-moment shifts in emotional momentum. Beat analysis builds itself around the following very basic fact:

Stories engage our attention by constantly modulating our emotional responses.”

Sadly though, I can’t get away with reading just this chapter. It’s only later that he actually shares the classification of beats that he uses in his analysis.

Hamlet’s Hit Points Icons and Arrows by Gameplaywright LLP and Craig S. Grant is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

He begins with two types that he says will make up the majority of beats in any story, Procedural and Dramatic beats. Procedural beats move the protagonist towards (forfilling the audience’s hopes) or away from (realizing the audience’s fears), his practical, external goal. Dramatic beats do the same for the protagonist’s inner goals. “We hope that the beat moves him closer to a positive inner transformation and fear that it might move him towards a negative transformation.”

Laws talks a lot about hope and fear. In fact he simplifies the audience’s emotional response to every beat (which he describes as its resolution) as being a movement towards one of these poles. I’ve got fear on my nascent emotional affect and affordances diagram, its one of Panksepp’s primal emotions, but I’m not yet sure where hope sits – I wonder, is it in care?

In both types of beat, Laws describes two parties, the petitioner, who wants the thing, and the granter, who must be negotiated with. Dramatic beats are mostly actual verbal negotiations, procedural beats might also be fights, tricks, races or other challenges.

From the way Laws describes them, I’d expect that most kernels in a story are likely to be one of these two types of beat. And the other types are more likely to be satellites. He lists:

Commentary – “in which the protagonist’s movement towards or away from his goal is momentarily suspended while the author underlines the story’s thematic elements.” Laws uses Mercutio’s Queen Mab speech in Romeo and Juliet as an example.

Anticipation – which “create[s] an expectation of coming procedural success, which we look forward to with pleasure.” The example here is “Popeye has eaten his spinach. (any given episode of Popeye)”

Gratification – “a positive emotional moment that floats free from the main narrative. They often appear as rest breaks between major sequences. A musical interlude often acts as a gratification beat (unless it also advances the story, as it frequently does in musical genre).”

Bringdown – the opposite of gratification. “Jerry Lundergaard’s car alone in a desolate parking lot, is completely iced over after his father-in-law bars him from a promising business deal. (Fargo)”

Then Laws offers us three “informational beats”:

Pipe – “A beat that surreptitiously provides us with information we’ll need later, without tipping the audience to its importance.”

Question – “introduces a point of curiosity we want to see satisfied […] a question usually resolves as a down beat.”

Reveal – “provides the information we were made to desire in a previous question beat, or surprises us with new information. In the latter case it might come out of the blue, or have been set up with one or more pipe beats laying the groundwork for the surprise.” The example he uses is the Revelation that Bruce Willis’ character in The Sixth Sense is dead. “We tend to be more engaged by exposition when it has been teased to us by a prior question, or can clearly see its impact on our hopes and fears.”

(Laws explains that literary fiction makes much use of question/reveal cycles to manipulate emotion, rather than the procedural / dramatic beats that fill genre fiction and thrillers.)

Laws goes on to analyse three scripted narratives in full, Shakespeare’s Hamlet, and the films Dr No and Casablanca, but that’s not what I’m discussing now, though having recently rewatched Casablanca as part of my children’s continuing cinema education, I was  interested to read his analysis of that. It is worth pointing out, however, that the “curve” of a story like Casablanca is inexorably downward. Laws compares the maps his analysis creates with “the classic chart you may recall from secondary school literature classes” (which I’ve touched on before) and notes that the lines his analysis creates are “flatter overall. It tends to resemble a stock tracker measuring the progress over time of a slowly deflating security […] Even stories that end happily […] tend to move downward over time.” He explains that narratives build up fear with numerous incremental steps, before sudden uplifting moments of hope. So in most stories, there are simply more down beats than up beats, given that the up beats are more intense. I think there is also a point that Laws misses, in many of those narrative curves the absolute value of emotional intensity is being measured, with no thought as to whether the emotion is hopeful or fearful.

So, is all this useful to me? Well I think at the very least I think I can get my “wizards of Oz” to think about up beats and down beats, and make sure not to pile on too many down beats in a row without the occasional up beat. Whether or not heritage interpretation lends itself to procedural and dramatic beats, there is definitely room for question/reveal beats, and it could be argued that too much interpretation goes straight for the revelations without asking the questions or laying the pipes first. So I think it is something that may prove useful.

“Breaking” Chawton’s story

I’ve been disassembling Chawton House Library’s guidebook and handout down into Natoms, as the very first stage of my project there. Natoms are not my idea, but a concept from Southampton colleague Charlie Hargood. However, for my purposes I’ve distinguished between Persistent natoms (or P-natoms) which are physical and thus, for the a heritage visitor, stay in one place and do not change, and Ephemeral natoms (or E-natoms) which are not tied to a particular place or even form, they are any media which can be digitally transmitted and shared, such as text, video or music.

To break the guidebook that Chawton supplied down, I’m currently using Scalar, for reasons I explained in an earlier post. It took a little time for me to work out how I should best use it, and to be honest I think I might come unstuck when I start distinguishing between kernels and satellites, but right now I’m getting into the flow of it, so before I went too far, I thought it would be good to share.

First of all I wanted to to some of the P-natoms in place. These are the rooms that our visitors will be exploring. It was very easy to list the rooms, and soon I had that list of rooms, each tagged as a P-natom, looking like this in Scalar’s useful “connections” visulisation:

But I had a slight crisis of confidence about how best to illustrate the links between spaces, or as Bill Hillier puts it, the permeability. Scalar allows simple hyperlinks, but doesn’t visualize them, or make them reciprocal. It also has a Path function, but that is only one way, really for stringing natoms together into a long-form text. So in the end I went for the program’s tagging function, which is quite sophisticated in distinguishing between items “being tagged” and “tagging”. Once I’d tagged the transitions betwene the listed spaces, my visualization looked like this:

I added the gardens at this point. I plan to keep my experiment within the walls of the manor house itself, but some doubt, or fear that I wasn’t future-proofing the model, made me put them in, as the exit form the gift shop.

These spaces are by no means all the P-natoms. More of the the collection will go in later. Next however I wanted to try adding some E-natoms. And that meant starting by breaking down the guidebook text. The first part of the guidebook was an essay, a brief chronological history of the place. Broken down into its constituent parts, and tagged as E-natoms, this is what that essay looked like:

As you can see, at this stage there is no connection between the the new E-natoms and the P-natoms, or between the story and the place. However, already some (but at this stage surprisingly few) of the e-natoms were suggesting engaging stories: which I’ve tagged “Literary Women” and “Jane Austen”. But look what happens when you start adding in parts of the collection, in the spaces in which they are displayed, and their associated stories. To two P and E-natom “solar systems” start to join together:

This is after the collection in just one space, the entrance hall, has been broken into the natoms. Its going to get a lot more complex as work progresses.

Chawton

A dreary day to photograph a fine building, but the meeting made up for the weather!

Just a quick note today to reflect on the meeting I had this morning with Gillian Dow, Executive Director of Chawton House Library. This place has been preying on my thoughts since I visited for the last Sound Heritage workshop. In fact, somebody (my friend Jane and her colleague Hilary) had suggested last year that it might be the perfect place to try out my Responsive Environment ideas. But my visit for Sound Heritage made me think more and more that they were right.

• The place has many interesting stories but ones that can conflict with each other. Do people what to know about it’s centuries as a residence for the Knight family, its connections with Austen, and/or its modern day research into early female writers?
• It’s a place that hasn’t been open to the public long (this year its its first full season welcoming days out visitors) and is still finding it’s voice.
• Its relatively free of “stuff” and has modern display systems (vitrines and hanging rails), which means that creating the experience should not be too disruptive.
• It has pervasive wi-fi (the library’s founding patron Sandy Lerner, co-founded Cisco systems) which will make the experiment a lot easier and cheaper to run, even though I’ve decided to Wizard of Oz it.

So today I explained my ideas to Gillian and, I’m pleased to say, she liked them. We’ve provisionally agreed to do something in the early part of 2017, before that year’s major exhibition is installed. I brought away a floor plan of the house, and I have just this moment received a copy of the draft guidebook, so I can start breaking the story into “natoms”. It looks very much like its all systems go!

I have to say I’m very excited.

(But right now, I’m meant to be taking the boy camping so, I’ll leave it there…

Scalar

Today, I’ve been thinking about a story-telling tool to help me “break” the stories of a site for the experiment I’ve been rethinking. I think I’m looking for something that meets the following criteria:

• Collaborative – more than one author
• Handles all sorts of media types
• Includes tags
• Allows the author(s) to see and manipulate the links between “Natoms” in networks
• Ability to turn some tags into narrative order (to create my “kernels”)
• Works with ontologies (eg OWL)
• Complies with data standards (eg RDF)

(There may be more criteria – can anyone suggest any?).

So, first of all, I thought Twine, which can do the Network thing, and the Kernel order thing, and I’ve even “broken” a story with it. But its not collaborative (well, it may be in version two) and doesn’t handle different media easily – I did get it to handle music (just about), but its not easy.

So then I got into the sort of software academics use to build their data networks. But the learning curve on most of them looks pretty steep. Then I remembered hearing about Scalar. Scalar looks quite interesting. It’s built to make interactive e-book multimedia dissertations really, but it might do the job I need. It works with OWL and RDF, uses, tags, makes narrative paths and does some very pretty network visualisations. Its definitely collaborative and handles lots of media types.

What I haven’t worked out is whether I can make the paths conditional (which I think I need) and wehther it can publish to a stand alone file, or whether it requires a web-connection. If it does require a web connection, then I can’t use it during the experiment, because I’m not likely to have we access anywhere where the experiment takes place.

Anyway its worth a deeper look, and maybe a play-around with.