Look Ma, No Keyboard user interface: Tools for music pre-production Current Developments Applying the principles to other music instruments Continuing user studies Exploring musical structures for good mappings Investigating different key boundaries The University of British Columbia Department of Electrical and Computer Engineering C2 B2 C3 B3 C6 B6 C5 B5 C4 B4 Touch Pad Interface Preliminary user studies Two interaction methods were tested: The Computer Keyboard and The Piano Controller. With an average learning time of approximately 5 minutes, subjects performed both tasks significantly better with the piano controller in Test1 (p<0.05, paired t-Test). Although Test2 (p<0.06, paired t-Test) was done faster than Test1, the performance was not significant with the piano controller. This was due to the minimal learning time, and small number of subjects (6) used for our study. Subjects found the piano interaction intuitive and an effective link between playing and editing. Human Communication Technologies Laboratory Current Sequencing Setup What is music sequencing? A music pre-production practice involving recording, arranging and playing MIDI data. Sequencing programs allow composers to edit out mistakes, play music in various tempos, change keys without replaying parts, rearrange an entire song, draw volume graphs, and more. Why cumbersome? The composer’s creative work is constantly interrupted by the time spent switching between multiple input devices and figuring out their functions. The current setup as shown above, also occupies excessive physical space. CUBASE VST by Steinberg An industry standard music sequencing package used for our study. CUBASE allows the composer to perform complex pre-production tasks. Commonly Performed Tasks Recording MIDI events on selected channels Playing back recorded channels Spatial positioning Effects and dynamic processing Editing notes etc. A volume graph for a selected channel References Farhan Mohamed, Sidney Fels Human Communication Technologies Laboratory, UBC LMNKui Setup “LMNKui System” “LMNKui System” An ergonomic method for controlling music sequencing software from a piano keyboard by adding a momentary foot switch. Touch Pad Octave (C2-B2) A touch pad is used for constrained vertical or horizontal actions, for tasks like sliding a fader and drawing a modulation graph. Mapping Example: General Edit Octave (C6-B6) An octave mapped to handle general editing tasks. Mapping Example: Key Edit Octave (C5-B5) An octave used by the composer very widely for editing notes and their values. Mapping Example: Arrange Octave (C4-B4) An octave used by the composer to arrange music by selecting and creating tracks. Mapping Example: Transport Octave (C3-B3) An octave used by the composer to play, stop, record, mute and solo a sequence or a specific MIDI channel. Mapping Example: Piano Function Mappings C3 Open/Close transport window G3 Forward C5 Open/Close key edit window G5 Move to the next note F6 Copy a sequence to the clipboard G6 Paste E4 Go to the previous part F4 Move down the track F#6Undo C2 Emulates the left button down of the mouse E2 Selecting volume C4Create a new track E5Move to the previous note Design Piano Function Mappings The design of the layout incorporates the windows and the functions most commonly used by the composer. The functions associated with each window map to a specific octave on the piano controller keyboard as shown in the Piano Function Mappings. To distinguish between keystrokes that represent notes and those that represent macros, a momentary foot switch is used as a ‘mode switch’. For continuous parameters, a single point sensitive touch pad is used in conjunction with the touch pad octave. User Testing Key Maps Mean Time with 95% Confidence Error Bars Momentary foot switch Touch Pad Interface G3Click On/Off (B3) E2 TracksParts Note Sequence