Supporting Content Creation for Games Through Assistive Technologies Dr. Michael Katchabaw Department of Computer Science The University of Western Ontario

2 Outline n Introduction n The Content Creation Gap n Study 1: Storytelling in Games n Study 2: Music in Games n Concluding Remarks

3 Introduction n Modern computer and video games would be sophisticated yet rather uninteresting and unimpressive pieces of software without all of their luscious content n This content defines their worlds, characters, atmosphere, events, and most aspects of the games that players directly experience n It can come in many forms: artwork, music, sound effects, story, dialogue, motion capture data, and so on …

4 Introduction n Fortunately, modern games tend be relatively flexible and extensible n There are many opportunities for people to be involved in creating content for games n This includes professional developers all the way through to the end user community comprised of players and hobbyists n Unfortunately, barriers ultimately limit expression by content creators regardless of where they fit in …

5 The Content Creation Gap n Content creators often do not have the technical expertise to integrate their content into games on their own n Technical practitioners often do not have the creative expertise to create such content in the first place n Consequently, there is a gap between content creation and its place in games

6 The Content Creation Gap

7 The Content Creation Gap: The Current Solution n Traditionally, programmers are used to bridge this content creation gap n By supporting content creators with their technical expertise, content works its way into games …

8 The Content Creation Gap: The Current Solution

9 n Problems with this approach ultimately limit its effectiveness: n Increased costs, both in terms of budget and scheduling resources n Content creators lose creative control over their content n Communication errors between content creators and programmers impact the overall quality of the experience

10 The Content Creation Gap: Another Solution n Another solution is to provide additional training to content creators to increase their technical expertise n This, in effect, would allow them to clear the content creation gap on their own …

11 The Content Creation Gap: Another Solution

12 The Content Creation Gap: Another Solution n This approach has its problems as well: n The training processes themselves might be expensive and time consuming n Not every content creator will be able to successfully acquire this technical expertise n In doing this, you might lose some of the benefits of specialization when it comes to development

13 The Content Creation Gap: A Better Solution n A better solution can be found through the creation and use of software tools that do not require technical expertise n These assistive technologies would allow content creators to ply their craft directly in games, using modes of interaction that are natural and familiar n This would allow content creators to overcome the content creation gap too …

14 The Content Creation Gap: A Better Solution

15 The Content Creation Gap: A Better Solution n Unfortunately, these assistive tools and technologies do not generally exist at the moment, or are fairly immature n But, interesting research is currently under way to address these issues and provide them for use n In doing so, many benefits to content creation for games can be realized

16 Study 1: Storytelling in Games n Storytelling is widely recognized by many as an important element of modern video games n For games that are dependent on their story elements, the level of quality required for their story at least equals the need for quality in graphics, audio, and other aspects of the games n Getting story content into games requires technical assistance, however

17 Study 1: Storytelling in Games n Assistive technologies would be helpful n Writers could be given a simple and natural way of expressing their stories n Automation could then prepare this content for presentation within a game with little or no human intervention n This is the intent of our own work towards a Reusable Scripting Engine n Our current focus is on story content for cinematics or cut-scenes, as opposed to in-game story elements

18 Storytelling in Games: Story Scripting n To automate the presentation of story elements from a story, the original story needs to be scripted in such a way that it can be easily acted out on-screen n This scripting must identify characters, dialogue, stage directions, setting, and other elements common to traditional dramatic pieces n This scripting will also need to be precise and formal enough for it to be easily processed and understood by the software automating the presentation

19 Storytelling in Games: Story Scripting n Fortunately, the Text Encoding Initiative (TEI) has developed an XML-based specification for marking up all different kinds of texts, including dramatic pieces n Since XML is not necessarily the most natural platform for storytellers writing stories, several tools have been developed according to TEI guidelines for importing existing works or writing them from scratch

20 Storytelling in Games: Story Scripting n In the end, we could not completely follow TEI guidelines, and had to derive something with some modifications and extensions n TEI guidelines are incredibly detailed and require information that does not quite make sense in our target application n Several elements in the TEI guidelines are not formal or precise enough yet for our purposes n We needed additional elements to link content and assets into story scripts

21 Storytelling in Games: Story Scripting

22 Storytelling in Games: A Reusable Scripting Engine Director Stage Manager Script Reader Authoring Tools Script User Display Writer Reusable Scripting Engine

23 Storytelling in Games: Experiences to Date n Initial experiences involved reenacting scenes from movies and television shows n Examples include the Princess Bride and various episodes of The Simpsons n This demonstrated the general usefulness of our approach n Recent experiences involved replicating cut-scenes and cinematics from existing video games …

24 Storytelling in Games: Experiences to Date Scene from Replication of Trauma Center: Second Opinion

25 Storytelling in Games: Conclusions n Our Reusable Scripting Engine is an example of an assistive technology to support content creators for games n Using our software, together with appropriate authoring tools, we can effectively assist writers in creating and integrating their story content into games

26 Study 2: Music in Games n There are many recognized outstanding issues in music for games n Static compositions that are looped or repeated throughout a (long) game can become quite tiresome n Music is, at times, disconnected from the events unfolding in the game, particularly in open ended, non-linear games n Licensing music or hiring talent can be prohibitively expensive, particularly for small or medium sized enterprises

27 Study 2: Music in Games n Adaptive automatic composition systems can alleviate or address these issues: n Endless new compositions can be composed dynamically throughout a game without any repetition n Music can be adapted to the current emotional context of the game n Music composed by a computer is not rights encumbered and can be more cost effective

28 Music in Games: AMEE Introduction n To provide assistive technologies to this end, we developed the Algorithmic Music Evolution Engine (AMEE) n Flexible and extensible architecture that can impose as much or as little structuring to music as desired n Can reuse existing musical elements or compose them entirely from scratch n Can incorporate emotional characteristics n Multi-level API to support a range of users with varying musical and technical expertise

29 Music in Games: AMEE Concepts n Pipeline: controls generation process n Producers: create high-level musical elements n Generators: create low-level musical elements n Pattern libraries: enables reuse of musical elements (e.g. Harmonies, Motifs, etc.) n Emotion mapper: enables incorporation of emotions (e.g. Happy, Sad, etc.) n High-level classes: model real-world entities (e.g. Musician, Instrument, Mood, Ability, Style, Ensemble, etc.)

30 Section Producer Section Producer Block Producer Block Producer Line Producer Line Producer Output Producer Output Producer Emotion Mapper Music in Games: AMEE Pipelining Section containing blocks Pipeline I-IV-V-I

31 Music in Games: Experiences with AMEE n We have only just begun to scratch the surface with what we can do with AMEE n More development and user testing (novice and expert) is still required Original Sad Despondent Happy Manic

32 Concluding Remarks n Assistive tools and technologies have the potential to bridge the content creation gap we discussed earlier n This provides content creators supports needed for more direct integration of their content into games with many far reaching benefits

33 Concluding Remarks n At Western, we are developing such assistive technologies for various types of content for game integration n To date, efforts have focused on story and music content n Experiences so far have been rather positive, demonstrating great potential for on-going research in this area

34 Questions?