1. PyClarion / Mockingbird Michael F. Lynch

2. What is Clarion? A hybrid cognitive architecture developed at the Department of Cognitive Science at RPI by Ron Sun and colleagues. Four major subsystems, each supporting an upper, symbolic layer and lower, sub-symbolic layer. Models, among other things, motivation (drives and goals) and metacognition Also models (in Wilson’s version) personality.

3. Action-Centered Explicit Representation Action-Centered Implicit Representation Non-Action-Centered Explicit Representation Non-Action-Centered Implicit Representation Filtering/Selection/ Regulation Goal Setting Reinforcement Action Perception Goal Structure Drives Clarion ACS MS NACS MCS

4. What is PyClarion? Another rewrite, necessitated by limitations in the C# edition, this time in Python. Uses the PyBrain library by Schmidhuber et al for its neural net implementations. This is turn will form the basis for further work into episodic memory. Also the basis for Mockingbird, an interactive musical agent (IMA).

5. Mockingbird

6. What is Mockingbird? … an attempt to build an Intelligent musical agent (IMA) based on Ron Sun’s Clarion cognitive architecture and Doug Van Nort’s FILTER expanded musical instrument (aka “hyperinstrument”).

7. Mockingbird – What it Does Mockingbird interacts with a live performer in a free improvisation performance. Mockingbird analyzes, via FILTER, the human performer’s auditory live audio signal, and makes decisions in real time to create its own musical accompaniments.

8. Positioning Mockingbird “Tape delay” (and later digital delay) music has a long tradition, dating back to the 1950s. Builds on the musique concrète tradition of Pierre Schaeffer. Among the early pioneers: Pauline Oliveros, right here at RPI.

9. Positioning Mockingbird More recently, “scratching,” “scrubbing” and “turntablism” have been added to this lexicon. Granular synthesis Algorithmic composition Expanded instruments Mockingbird proposes to build on these traditions by incorporating the capabilities of a modern, hybrid cognitive architecture.

10. Expanded Instruments Oliveros’s EIS Van Nort’s GREIS (inspired by EIS) Van Nort’s FILTER CAIRA (Creative, Artificially-Intuitive and Reasoning Agent) Mockingbird

11. Mockingbird – Design Goals Construct a musical “agent” (a musical accompanist and improviser) – Our term: Intelligent Musical Agent (IMA) Identify and develop cognitive models of musical performance needed in such an agent. Implement these models in an actual agent, using the Clarion cognitive architecture as the “substrate.”

12. Mockingbird – Initial Goals Be able to listen to a single live performer “Learn” the contours of the performance during the actual performance From that, accompany the performer in the manner of free improvisation, one-on-one, playing back the performer’s own materials, time-delayed and transformed. That’s why it’s called Mockingbird.

13. Mockingbird – Eventual Goals Expand the capabilities to an M x N performance setting. Give the agent long-term “expertise” via machine learning, Give the agent the ability to recall past performances and the material therefrom. Develop other means of imparting musical knowledge. Give the agent the ability to play from a free- improvisation “score”. – In the manner of Baker’s First Nation’s Ley

14. How It Is Bolted Together Auditory processing (inputs) implemented with a subset of FILTER Output generation (audible signals) also implemented with a subset of FILTER Internal “cognition” supplied using the CLARION cognitive architecture… … and also the cognitive capabilities already in FILTER

15. What is Needed…

16. Python programmers… especially if you are really good with multiprocessing programming techniques. People interested in neural nets and working with the PyBrain library. People interested in cognitive architectures and working on Clarion. People interested in these sorts of issues in experimental music.

17. lynchm2@rpi.edu