A language for auralizing data

MOTIVATION  Visual input overload!  Automatic musical scoring

UPBEAT  A flexible language to auralize input data  aur al iz a tion (n):  1. the act or process of interpreting in aural or musical terms  2. the process of converting data into music  Previous attempts haven’t sounded great

Musician "StockMusician" { phrase tune = "tune.mid"; "ChangePitch" { Change Pitch by 3_semitones; } "Play" { ||: tune :||; } } EASY CODING, BETTER MUSIC  Easy to use  Imperative Data Analysis  Declarative Musical Syntax  Portable: based on Java  Sounds great!  Goal for today: react to stock ticker

COMING UP… Intro to Upbeat Peyton Sherwood(Project Manager) Upbeat Code Adrian Weller(Language Guru) Compiler Structure & Tools Fan Lin 林凡 (System Integrator) Runtime Framework Matt D’Zmura(System Architect) Testing & What We’ve Learned Miles Ulrich(Verification and Validation)

Conductor Musician “Keyboard” Musician “FunkyDrummer” TIME_SIGNATURE 4/4; TEMPO 140; Conductor { cue up = "GoingUp"; cue down = "GoingDown"; cue background = "GoingNowhere"; float param = 0.95; float threshold = 5.0; float Avg = 70.0; // weighted average Conduct(time currentMeasure) { Avg = param*Avg + (1-param)*input.currentvalue; Case { (currentMeasure==1) Add background currentMeasure; (input.currentvalue > Avg + threshold) Add up currentMeasure; (input.currentvalue < Avg - threshold) Add down currentMeasure; } Musician "Keyboard" { phrase intro = "intro.mid"; phrase bridge = "bridge.mid"; phrase UpbeatRules = "upbeatrules.mid"; phrase SongSnippet = "goingdown.mid"; "GoingNowhere" { ||: intro | bridge :||; } "GoingUp" { Change Volume to 100; // volume is always in [0,100] || UpbeatRules :||2; } "GoingDown" { Change Volume to 70; ||: SongSnippet :||2 ; } Musician "FunkyDrummer" { phrase introdrums = "introdrum.mid"; phrase bridgedrums = "bridgedrum.mid"; phrase funkydrums = "drumpatch.mid"; phrase cymbal = "cymbal.mid"; Change Instrument to 32; // drums happen to be on MIDI channel 32 "GoingNowhere" { ||: introdrums | bridgedrums :||; } "GoingUp" { || funkydrums | cymbal ||; } Cues SAMPLE CODE

TIME_SIGNATURE 4/4; TEMPO 140; Conductor { cue up = "GoingUp"; cue down = "GoingDown"; cue background = "GoingNowhere"; float param = 0.95; float threshold = 5.0; float Avg = 70.0; // weighted average Conduct(time currentMeasure) { Avg = param*Avg + (1-param)*input.currentvalue; Case { (currentMeasure==1) Add background currentMeasure; (input.currentvalue > Avg + threshold) Add up currentMeasure; (input.currentvalue < Avg - threshold) Add down currentMeasure; } Musician "Keyboard" { phrase intro = "intro.mid"; phrase bridge = "bridge.mid"; phrase UpbeatRules = "upbeatrules.mid"; phrase SongSnippet = "goingdown.mid"; "GoingNowhere" { ||: intro | bridge :||; } "GoingUp" { Change Volume to 100; // volume is always in [0,100] || UpbeatRules :||2; } "GoingDown" { Change Volume to 70; ||: SongSnippet :||2 ; } Musician "FunkyDrummer" { phrase introdrums = "introdrum.mid"; phrase bridgedrums = "bridgedrum.mid"; phrase funkydrums = "drumpatch.mid"; phrase cymbal = "cymbal.mid"; Change Instrument to 32; // drums happen to be on MIDI channel 32 "GoingNowhere" { ||: introdrums | bridgedrums :||; } "GoingUp" { || funkydrums | cymbal ||; } DATA ANALYSIS Conductor cue up = "GoingUp"; cue down = "GoingDown"; cue background = "GoingNowhere"; … Case { (currentMeasure==1) Add background currentMeasure; (input.currentvalue > Avg + threshold) Add up currentMeasure; (input.currentvalue < Avg - threshold) Add down currentMeasure; }

TIME_SIGNATURE 4/4; TEMPO 140; Conductor { cue up = "GoingUp"; cue down = "GoingDown"; cue background = "GoingNowhere"; float param = 0.95; float threshold = 5.0; float Avg = 70.0; // weighted average Conduct(time currentMeasure) { Avg = param*Avg + (1-param)*input.currentvalue; Case { (currentMeasure==1) Add background currentMeasure; (input.currentvalue > Avg + threshold) Add up currentMeasure; (input.currentvalue < Avg - threshold) Add down currentMeasure; } Musician "Keyboard" { phrase intro = "intro.mid"; phrase bridge = "bridge.mid"; phrase UpbeatRules = "upbeatrules.mid"; phrase SongSnippet = "goingdown.mid"; "GoingNowhere" { ||: intro | bridge :||; } "GoingUp" { Change Volume to 100; // volume is always in [0,100] || UpbeatRules :||2; } "GoingDown" { Change Volume to 70; ||: SongSnippet :||2 ; } Musician "FunkyDrummer" { phrase introdrums = "introdrum.mid"; phrase bridgedrums = "bridgedrum.mid"; phrase funkydrums = "drumpatch.mid"; phrase cymbal = "cymbal.mid"; Change Instrument to 32; // drums happen to be on MIDI channel 32 "GoingNowhere" { ||: introdrums | bridgedrums :||; } "GoingUp" { || funkydrums | cymbal ||; } MUSICAL SYNTAX Musician: “Keyboard” "GoingNowhere" { ||: intro | bridge :||; } "GoingUp" { Change Volume to 100; || UpbeatRules :||2; } "GoingDown" { Change Volume to 70; ||: SongSnippet :||2 ; }

COMING UP… Intro to Upbeat Peyton Sherwood(Project Manager) Upbeat Code Adrian Weller(Language Guru) Compiler Structure & Tools Fan Lin 林凡 (System Integrator) Runtime Framework Matt D’Zmura(System Architect) Testing & What We’ve Learned Miles Ulrich(Verification and Validation)

Front End Lexer Parser.up Runtime Framework Music API jMusic API Wrapper main JVM COMPILER STRUCTURE Java Compiler

Runtime Framework Music API jMusic API Wrapper main Front End Lexer Parser main.up main SYSTEM INTEGRATION

Runtime Framework Music API jMusic API Wrapper main Front End Lexer Parser.up TOOLS

COMING UP… Intro to Upbeat Peyton Sherwood(Project Manager) Upbeat Code Adrian Weller(Language Guru) Compiler Structure & Tools Fan Lin 林凡 (System Integrator) Runtime Framework Matt D’Zmura(System Architect) Testing & What We’ve Learned Miles Ulrich(Verification and Validation)

Music API jMusic API Wrapper Run-time Framework Keyboard Drummer Data File Conductor Input Drums.mid Strings.mid Horn.mid Piano.mid Input Midi Files Cues M1: GoingNowhere M9: GoingUp M13: GoingUp Measures M1: “Drums.mid” M3: “Strings.mid” Measure 1 M2: “Drums.mid” M4: “Strings.mid” Measure 2 RUN-TIME FRAMEWORK

COMING UP… Intro to Upbeat Peyton Sherwood(Project Manager) Upbeat Code Adrian Weller(Language Guru) Compiler Structure & Tools Fan Lin 林凡 (System Integrator) Runtime Framework Matt D’Zmura(System Architect) Testing & What We’ve Learned Miles Ulrich(Verification and Validation)

TESTING SETUP UNIT TESTS Upbeat Source Files Lexer + Parser Target Java Code Validate against sample codeOne source file per feature Upbeat Source Files Compiler Toolchain Target MIDI Files Music verified by earCollection of source, input END-TO-END TESTS

WHAT WE’VE LEARNED  Agree on a metaphor early  K.I.S.S.  Project Planning  Schedule  Weekly check-ins

UPBT: STOCK TICKER