- A network music jamming app for everyone wiJam - A network music jamming app for everyone

Features Basic: Master – slave star connection Musical notes assignment Virtual instruments playback Sound sample download, store, and playback Extension(*): Sound control Location of every player Shake support(accelerometer ) Performance Adaptive Algorithm

Scenario – CanTine JAM M S S S S S S

MASTER-SLAVE relationship In this scenario: Master and slaves have different console The master distributes all the instruments , scales and notes to the players and it has full sound controls. The players do nothing other than tapping the notes for fun.

DISTRIBUTE a Key, a SCALE and instruments Philip Grand Piano Acoustic Guitar Electric Guitar Acoustic Bass Violin Key and scale … Sample Sound Crack Clap C D E G A Blu Maj Aeo Pen Lyd Groove-1 … Those interesting sounds downloaded from the internet or recorded on ones own

MASTER CONTROL Philip Bony CX CY std … Volume Bighead tangkk Panning Groove-1 Groove-2 Groove-3 Groove-4 C D E G A Blu Maj Aeo Pen Lyd

MASTER CONTROL Philip Bony CX CY Rev flg filt … Bighead tangkk Eco Rev flg filt … Bighead tangkk Groove-1 Groove-2 Groove-3 Groove-4 C D E G A Bls Maj Aeo Pen Lyd

Players only see the notes D5 E5 A4 C5 E4 G4 C4 D4

Syncing (4/4) – snap the note to the nearest grid ahead 1 2 3 4 … … In this example: 4/4 time (4 beats a bar), each beat has 2n cells

Scenario – CanTine JAM Master D5 E5 A4 C5 E4 G4 C4 D4 D5 E5 A4 C5 E4

Scenario – CanTine JAM Master D5 E5 A4 C5 E4 G4 C4 D4 D5 E5 A4 C5 E4

Sample Sound Manager(4) Layering -0.1 Master Slave User Interface Music Assigner (1) MIDI router (2) Sample Sound Manager(4) Core MIDI /GKSession AUGraph(3) Bonjour + network AudioSession User Interface Music Performer Core MIDI /GKSession AUGraph AudioSession Bonjour + network TCP/IP 1 Music assigner is for assigning instruments, scales and notes to slaves 2 MIDI router is for routing MIDI messages coming from different slaves to different virtual instruments. 3 AUGraph is actually a set of virtual instruments and audio processing units. 4 Sample sound manager is for storing preset sounds(both instrument sounds and groove tracks) and making these sounds accessible.

Master User Interface Music Assigner MIDI router CoreMIDI/ The Conductor User Interface Music Assigner MIDI router Sample Sound Manager CoreMIDI/ GKSession  AUGraph Bonjour + network AudioSession

Master User Interface Music Assigner MIDI router CoreMIDI/ Sample Sound Manager CoreMIDI/ GKSession  AUGraph Bonjour + network AudioSession The Performance

Master User Interface Music Assigner MIDI router CoreMIDI/ Sample Sound Manager CoreMIDI/ GKSession  AUGraph Bonjour + network AudioSession Virtual Instruments

Master User Interface Music Assigner MIDI router CoreMIDI/ Sample Sound Manager CoreMIDI/ GKSession  AUGraph Bonjour + network AudioSession Communication channel

Music Assigner Music Performer Assign performance prototypes Adaptive Algorithm Perform performance feedback Smart enough to adapt to different constitution of players Could be anyone who is totally out of musical feeling

How to decide which set of notes to be played? How to change keys? Music Assigner Table in C (just an example) Scale/Note 1 2 3 4 5 6 7 8 Ionian C4 E4 F4 G4 A4 B4 C5 D5 C3 E3 G3 … Dorian bE4 bB4 Phrygian Lydian Mixolydian Aeolian Locrian

How to calculate the scores? Easy Adaptive Scoreboard Level assignment Performance feedback

How to identify each slave devices in the implementation of master? Performer1 Performer2 Performer3 Performer4 Performer5 Performance MIDI Router AUGraph Input VI 1 VI 2 VI 3 VI 4 VI 5 Output

Sound Library Sound Sample Manager How to store these key-value pairs? Sound Library AUPresets + Audio files mapped to the AUPresets AUPresetName AUPresetFile download Sound Sample Manager

AUPreset File is basically a mapping file that maps sound samples to notes, so that they together form virtual musical instruments. Note 1 Note 2 Note 3 ... Sound Sample 1 Sound Sample 2 Sound Sample 3 ...

How does the master specify which player to assign to How does the master specify which player to assign to? How does each player prevent itself from receiving MIDI from other players? Master coreMIDI NetworkSession Slave coreMIDI NetworkSession Music Assignment in the form of sysEx MIDI Packets Packets Send Handler Packets Receive Handler Packets Send Handler Packets Receive Handler Music Performance in the form of normal MIDI Packets

Scenario – CanTine JAM Master D5 E5 A4 C5 E4 G4 C4 D4 D5 E5 A4 C5 E4

Scenario – CanTine JAM Master D5 E5 A4 C5 E4 G4 C4 D4 D5 E5 A4 C5 E4