1. A New Approach to the Design of Interactive Storytelling Engines TeongJoo Ong and John Leggett Texas A&M University

2. Overview Introduction and Background Introduction and Background Motivation and Scenarios of Use Motivation and Scenarios of Use Architecture of the Interactive Storytelling Engine Architecture of the Interactive Storytelling Engine Story Generation Story Generation Encoding and Evaluating the Story Components Encoding and Evaluating the Story Components Story Example Story Example Evaluation Criteria, Contributions and Conclusions Evaluation Criteria, Contributions and Conclusions

3. Overview Introduction and Background Introduction and Background Motivation and Scenarios of Use Motivation and Scenarios of Use Architecture of the Interactive Storytelling Engine Architecture of the Interactive Storytelling Engine Story Generation Story Generation Encoding and Evaluating the Story Components Encoding and Evaluating the Story Components Story Example Story Example Evaluation Criteria, Contributions and Conclusions Evaluation Criteria, Contributions and Conclusions

4. Introduction Stories are used to convey information, cultural values, and experiences Stories are used to convey information, cultural values, and experiences New technologies have constantly provided increasingly sophisticated means to tell stories New technologies have constantly provided increasingly sophisticated means to tell stories Recent trend is the convergence of technology, entertainment, and art in the computer Recent trend is the convergence of technology, entertainment, and art in the computer The ancient art of storytelling can be used to efficiently convey information in our increasingly computerized world The ancient art of storytelling can be used to efficiently convey information in our increasingly computerized world

5. Background Interactive storytelling is a major research area Interactive storytelling is a major research area Many overlapping approaches have been used: Many overlapping approaches have been used: Immersive storytelling Immersive storytelling Emergent storytelling Emergent storytelling Plot-based systems Plot-based systems Interactive authoring of stories Interactive authoring of stories Character-based systems Character-based systems These approaches differ in design paradigms and technical solutions These approaches differ in design paradigms and technical solutions

6. Background (cont.) Approaches used in interactive storytelling fall into the field of Narrative Intelligence Approaches used in interactive storytelling fall into the field of Narrative Intelligence Narrative Intelligence commonly refers to research on human narratives and storytelling, and related technologies Narrative Intelligence commonly refers to research on human narratives and storytelling, and related technologies Related research projects from this field: Related research projects from this field: CHOROS (N. M. Sgouros) CHOROS (N. M. Sgouros) Fuzzy Cognitive Maps (FCM) (E. Maffre) Fuzzy Cognitive Maps (FCM) (E. Maffre) HyperSoap (J. Dakss) HyperSoap (J. Dakss) Terminal Time (M. Mataes) Terminal Time (M. Mataes)

7. Overview Introduction and Background Introduction and Background Motivation and Scenarios of Use Motivation and Scenarios of Use Architecture of the Interactive Storytelling Engine Architecture of the Interactive Storytelling Engine Story Generation Story Generation Encoding and Evaluating the Story Components Encoding and Evaluating the Story Components Story Example Story Example Evaluation Criteria, Contributions and Conclusions Evaluation Criteria, Contributions and Conclusions

8. Motivation Each of the different approaches used in interactive storytelling has its own strengths and weaknesses Each of the different approaches used in interactive storytelling has its own strengths and weaknesses A hybrid approach that can combine the strengths while reducing or eliminating the weaknesses of several different approaches could potentially be very important A hybrid approach that can combine the strengths while reducing or eliminating the weaknesses of several different approaches could potentially be very important Certain characteristics of the approaches will also be discarded to keep complexity and understandability in check Certain characteristics of the approaches will also be discarded to keep complexity and understandability in check

9. Motivation (cont.) Our hybrid approach proposes the use of an evolutionary algorithm and story templates to construct story components Our hybrid approach proposes the use of an evolutionary algorithm and story templates to construct story components Current system states and interaction between fuzzy rules impose constraints, modifications and evaluations on the story components generated by the evolutionary algorithm Current system states and interaction between fuzzy rules impose constraints, modifications and evaluations on the story components generated by the evolutionary algorithm The end result is a template-based storytelling engine offering flexibility and robustness in storytelling The end result is a template-based storytelling engine offering flexibility and robustness in storytelling

10. Scenarios of Use A murder mystery that can yield different conclusions depending on the reader’s interaction with computer agents using dynamically generated story components A murder mystery that can yield different conclusions depending on the reader’s interaction with computer agents using dynamically generated story components Sub-module of a computer game engine that generates story scripts dynamically Sub-module of a computer game engine that generates story scripts dynamically Interactive learning environment that presents concepts in a narrative thread that includes recombination and indeterminacy Interactive learning environment that presents concepts in a narrative thread that includes recombination and indeterminacy

11. Scenarios of Use in Education Multiple experiences (points of view) through changing plots, story lines, and character relationships while adhering to a common thread Multiple experiences (points of view) through changing plots, story lines, and character relationships while adhering to a common thread Student involvement in unfolding the story through interactive exploration of alternatives and role- playing Student involvement in unfolding the story through interactive exploration of alternatives and role- playing Encourages personalization of the story Encourages personalization of the story Children as authors of their own stories Children as authors of their own stories

12. Overview Introduction and Background Introduction and Background Motivation and Scenarios of Use Motivation and Scenarios of Use Architecture of the Interactive Storytelling Engine Architecture of the Interactive Storytelling Engine Story Generation Story Generation Encoding and Evaluating the Story Components Encoding and Evaluating the Story Components Story Example Story Example Evaluation Criteria, Contributions and Conclusions Evaluation Criteria, Contributions and Conclusions

13. System Architecture Evolutionary Programming Subsystem (EPS) Knowledge Base Fuzzy Decision Based Subsystem (FDBS) Graphical/Object Library (GOL) Integrated Authoring Environment (IAE) evaluates/modifies parameters processes/generates story components monitors/modifies parameters of rules user interactions sends combined output rendering instructions authors/interactors

14. Interactive Storytelling Engine Combines the global search and emergent characteristics of evolutionary programming and approximate reasoning of fuzzy logic to generate new story components Combines the global search and emergent characteristics of evolutionary programming and approximate reasoning of fuzzy logic to generate new story components The solution can be any valid combination of story components in accordance with the author’s rules, system states and templates The solution can be any valid combination of story components in accordance with the author’s rules, system states and templates Templates are used to describe story components and the valid contextual sets of actions, scripts, objects and agents Templates are used to describe story components and the valid contextual sets of actions, scripts, objects and agents

15. Interactive Storytelling Engine (cont.) Story Components are the representation of a particular time step in the story. They facilitate the encoding process of the Evolutionary Programming Subsystem (EPS) Story Components are the representation of a particular time step in the story. They facilitate the encoding process of the Evolutionary Programming Subsystem (EPS) Contextual Sets are used to categorize various plot units, events, scripts, actions and characters within the story components Contextual Sets are used to categorize various plot units, events, scripts, actions and characters within the story components The Evolutionary Algorithm explores the problem space using solution instances of a particular story component The Evolutionary Algorithm explores the problem space using solution instances of a particular story component The Fuzzy Decision Based Subsystem (FDBS) imposes constraints, modifications and evaluations on the solution instances with rules and variables in the knowledge base The Fuzzy Decision Based Subsystem (FDBS) imposes constraints, modifications and evaluations on the solution instances with rules and variables in the knowledge base

16. Interactive Storytelling Engine (cont.) The Knowledge Base stores user interactions, events, fuzzy rules, discrete rules and various variables to keep track of the story’s progress The Knowledge Base stores user interactions, events, fuzzy rules, discrete rules and various variables to keep track of the story’s progress In order to ensure proper development and conclusion of the story, underlying story threads must be created by the author to act as guidelines for the engine In order to ensure proper development and conclusion of the story, underlying story threads must be created by the author to act as guidelines for the engine Underlying story threads are coded as a collection of story components so that solution instances generated by the EPS can be evaluated by the FDBS Underlying story threads are coded as a collection of story components so that solution instances generated by the EPS can be evaluated by the FDBS

17. Integrated Authoring Environment (IAE) A collection of tools and interfaces for authors to create interactive stories A collection of tools and interfaces for authors to create interactive stories The collection of editing tools and library consists of: The collection of editing tools and library consists of: Editing Tools Editing Tools Agent Characters Agent Characters Action Scripts Action Scripts Story Components Story Components The EPS The EPS The FDBS The FDBS Library Library The Graphical/Object Library (GOL) The Graphical/Object Library (GOL)

18. Overview Introduction and Background Introduction and Background Motivation and Scenarios of Use Motivation and Scenarios of Use Architecture of the Interactive Storytelling Engine Architecture of the Interactive Storytelling Engine Story Generation Story Generation Encoding and Evaluating the Story Components Encoding and Evaluating the Story Components Story Example Story Example Evaluation Criteria, Contributions and Conclusions Evaluation Criteria, Contributions and Conclusions

19. Story Generation: Step 1 User interacts with the story User interacts with the story Knowledge base records user interactions Knowledge base records user interactions User Integrated Authoring Environment (IAE) Interacts Knowledge Base Modifies/Records user interactions

20. Story Generation: Step 2 The knowledge base stores parameters from previous execution of the EPS The knowledge base stores parameters from previous execution of the EPS Evolutionary Programming Subsystem (EPS) Knowledge Base Stores Parameters from previous execution

21. Story Generation: Step 3 Knowledge Base Fuzzy Decision Based Subsystem (FDBS) Provides rules/parameters for merging and checking Knowledge base provides various information and rules for the FDBS Knowledge base provides various information and rules for the FDBS Output of FDBS determines the firing strength of the various fuzzy rules Output of FDBS determines the firing strength of the various fuzzy rules

22. Story Generation: Step 4 Knowledge Base Fuzzy Decision Based Subsystem (FDBS) Evolutionary Programming Subsystem (EPS) Provides parameters and information from rules to generate story components from templates Provides templates and contextual sets used to populate story components Information from the knowledge base and FDBS are merged by the EPS to generate new story components Information from the knowledge base and FDBS are merged by the EPS to generate new story components

23. Story Generation: Step 5 Knowledge Base Evolutionary Programming Subsystem (EPS) Stores story component and other necessary data for historical record State information about the system is stored in the knowledge base to allow the user to return to a particular scene of the story State information about the system is stored in the knowledge base to allow the user to return to a particular scene of the story

24. Story Generation: Step 6 Knowledge Base Provides story component used to generate rendering instructions The resultant story component is sent to GOL to be interpreted into a series of instructions The resultant story component is sent to GOL to be interpreted into a series of instructions Graphical/Object Library (GOL)

25. Story Generation: Step 7 Graphical/Object Library (GOL) Integrated Authoring Environment (IAE) Renders various graphical objects and agents Various objects and agents are displayed in the IAE based on the rendering instructions generated in the previous step Various objects and agents are displayed in the IAE based on the rendering instructions generated in the previous step These steps are repeated until the story terminates These steps are repeated until the story terminates

26. Overview Introduction and Background Introduction and Background Motivation and Scenarios of Use Motivation and Scenarios of Use Architecture of the Interactive Storytelling Engine Architecture of the Interactive Storytelling Engine Story Generation Story Generation Encoding and Evaluating the Story Components Encoding and Evaluating the Story Components Story Example Story Example Evaluation Criteria, Contributions and Conclusions Evaluation Criteria, Contributions and Conclusions

27. Encoding Function Parameters in Floating Point Each of the floating point numbers represent a gene in a chromosome Each of the floating point numbers represent a gene in a chromosome They can encode the unknown variables in a function (z = v + w + x) or the partition sizes of a surface They can encode the unknown variables in a function (z = v + w + x) or the partition sizes of a surface 1.2342.3451.113.2110.1212.32220.982 Floating point format

28. Encoding the Chromosomes Chromosomes are created by concatenating the floating point numbers Chromosomes are created by concatenating the floating point numbers For the EPS, a chromosome consists of a series of floating point numbers For the EPS, a chromosome consists of a series of floating point numbers Each floating point number represents a particular story sequence in the template Each floating point number represents a particular story sequence in the template 1.2342.3451.113.2110.1212.32220.982 Floating point format

29. Encoding the Story Components 232… ……………………………….…… Valid sets of action and object instances Gene in a chromosome For example, a template sequence numbered 2, with two object instances will be encoded as a floating point number 2.xx depending on the set being chosen For example, a template sequence numbered 2, with two object instances will be encoded as a floating point number 2.xx depending on the set being chosen

30. Crossover Chromosome A Chromosome B  Crossover operator allows chromosomes to exchange genetic materials  Single-point and Multi-point crossovers are supported by the EPS Chosen offset for crossover Chromosome A* Chromosome B*

31. Mutation Chromosome A Chromosome A*  Mutation operator randomly changes the gene in a chromosome  Introduces variations in the population Mutation will happen to this gene After mutation

32. Fitness Evaluation Chromosome in a population Story thread given by the author which may indicate preferences or avoidance of certain actions or object instances The story components are evaluated against the story thread given by the author  The story components are evaluated against the story thread given by the author

33. Overview Introduction and Background Introduction and Background Motivation and Scenarios of Use Motivation and Scenarios of Use Architecture of the Interactive Storytelling Engine Architecture of the Interactive Storytelling Engine Story Generation Story Generation Encoding and Evaluating the Story Components Encoding and Evaluating the Story Components Story Example Story Example Evaluation Criteria, Contributions and Conclusions Evaluation Criteria, Contributions and Conclusions

34. The Three Little Pigs Story Example <storyline> </storyline>

35. The Three Little Pigs Example (cont.) Introduction Wolf’s Threat Wolf’s Attack Wolf’s Plan While the wolf is still alive or a pig is available for the wolf, these template sets provide various alternatives for the wolf to attack the pigs and the pigs’ response to those attacks Provides background description of the story to the reader. It also determines the template sets that will be used in the rest of the story by selecting a particular variant of the story The wolf actor shows up. Depending on how the pigs react, certain action instances could dictate that the pigs are paralyzed with fear, and thus eaten by the wolf which can lead to an early conclusion to the story Conclusion The wolf may have been killed or cooked by one of the pigs, or all of the pigs were eaten by the wolf, yielding a conclusion

36. Overview Introduction and Background Introduction and Background Motivation and Scenarios of Use Motivation and Scenarios of Use Architecture of the Interactive Storytelling Engine Architecture of the Interactive Storytelling Engine Story Generation Story Generation Encoding and Evaluating the Story Components Encoding and Evaluating the Story Components Story Example Story Example Evaluation Criteria, Contributions and Conclusions Evaluation Criteria, Contributions and Conclusions

37. Evaluation Criteria Quality of story is based on user’s evaluation of the events, deviation from the author’s story line, and congruence of the transitions in the story Quality of story is based on user’s evaluation of the events, deviation from the author’s story line, and congruence of the transitions in the story Responsiveness, performance and execution speed of the engine Responsiveness, performance and execution speed of the engine Stability and consistency in the knowledge base Stability and consistency in the knowledge base

38. Contributions Simple and integrated environment for authors to create interactive and recombinant stories Simple and integrated environment for authors to create interactive and recombinant stories Materials used in one story can be reused in a related story setting Materials used in one story can be reused in a related story setting A useful educational tool when rules are integrated into the engine to reflect certain physical or social phenomena A useful educational tool when rules are integrated into the engine to reflect certain physical or social phenomena

39. Conclusions The hybrid approach used in storytelling is not perfect since it lacks the ability to automatically generate story templates out of a knowledge base The hybrid approach used in storytelling is not perfect since it lacks the ability to automatically generate story templates out of a knowledge base Gives authors the power to have full control of the cyborg authoring environment and provides tools to author various aspects of the interactive stories Gives authors the power to have full control of the cyborg authoring environment and provides tools to author various aspects of the interactive stories