ArmKeyBoard A Mobile Keyboard Instrument Based on Chord-scale System and Tonal Hierarchy

Why making armkeyboard Introduction Why making armkeyboard

Traditional Piano Keyboard Large size (non-portable) Difficult to learn (chord-scale expressions) Linearity (rules out non-linearity) Traditional Piano Keyboard

Can we redesign the keyboard so that it is easy to carry around and easy to make good music at the same time?

Existing Mobile Piano Keyboards Simply copying piano layout Fix scale Linearity Existing Mobile Piano Keyboards

Existing Mobile Piano Keyboards Not easy to learn Existing Mobile Piano Keyboards

ArmKeyBoard Portable “Instant music, subtlety later” Embrace non-linearity

Two Types of Keyboard Layouts Linear layout and non-linear layout

Linear Piano Keyboard Layout C C# D D# E F F# G G# A A# B

Linear Piano Keyboard Layout C C# D D# E F F# G G# A A# B Linear Layout Chromatic Scale

Non-linear Piano Keyboard Layout C C# D D# E F F# G G# A A# B Non-linear Layout Chromatic Scale

Three characters for making mobile smart instruments Mobile Smart Phone Three characters for making mobile smart instruments

Three Facts on Mobile Smart Phone Most users are non-musicians (calls for flat learning curve) The screen is small (If all 88 keys are to be equally distributed on an iPhone 5 screen, each key has only a space of about 82mm2 (less than 1cm2) ) The device is programmable (can implement any possible layouts and mappings)

Musical Assumptions if we assume that within any short range of a musical process, all the perceptible notes must belong to a certain chord- scale if this short range itself belongs to a larger meaning group, then the keyboard should minimally play one chord- scale at a time. For different music forms, we need to make different musical assumptions during the design

The design of Armkeyboard’s musical note space and control mechanism

Chord, Scale and Octave ArmKeyBoard treats the small screen space as a cache, for caching the currently playing chord- scale in the current octave range Other octaves and chord-scales are waiting to be loaded when needed. Users can switch between different chord- octave-scales using a gravityX gesture

Chord-octave-scale Sequence Grid The sequence goes from left to right, and then from top to bottom The sequence can be saved as preset Every grid element stands for a single combination of chord, octave and scale Chord-octave-scale sequence grid and chord-octave-scale presets

Control Mechanisms Gravity X gesture, which is used for switching to the next or previous page of notes determined by the chord-octave-scale combination at the next or previous square within the sequence. Gravity Y gesture (on the left), which is used for controlling note velocity, leading to a smaller velocity with a larger angle to the horizontal plane; Gravity Z gesture (on the right), which is used for quitting the current keyboard to reset everything again

Key-note Mapping How to map keys to notes

Linear Mapping and Non-linear Mapping Mapping Overview ArmKeyBoard linear layout contains 15–17 notes within the active chord-octave- scale and they are mapped linearly to 15– 17 bars equally divided along the y-axis. The velocity is controlled by the X position. ArmKeyBoard Non-linear layout starts with a user specified image. The image is then algorithmically divided into contours and these contours are then algorithmically mapped to the 15–17 notes within the currently active chord-octave-scale. Linear Mapping and Non-linear Mapping

Non-linear Mapping Algorithm Contour separation Contour Ranking Tonal Hierarchy Final Mapping

Before and after contour separation

Contour Ranking The minimal musical concern is, when the keyboard is being played, the notes being generated should at least imply the currently active chord-scale most of the time. The assumption is that most user tends to tap on: 1, a contour with a larger area; 2, a contour closer to the center of the screen; 3, a contour that contains more sub- contours. Based on how often most users will tap on a certain contour, the importance of a contour can be determined. This actually corresponds to how important a note is in implying a certain chord-scale, which is discussed below.

Tonal Hierarchy Tonal Hierarchies in ArmKeyBoard. L1 is the first level of notes which are to be mapped to regions with highest importance, and L2 to be mapped to regions with second highest scores, then L3 to be mapped to the least important regions

Final Mapping

Final Mapping 2 6 1’ 3 5 3’ 7 1 5’ 7’ 4 2’ 6’ 4’

Final Mapping 4 2 6 2’ 6’ 1,3,5,7 1’,3’,5’ 7’ 1’’

Video Demo How it sounds like

ArmKeyBoard Demo http://youtu.be/ZhTleEXKeu4

How do the users rate armkeyboard Evaluation How do the users rate armkeyboard

Evaluation Framework Audience Experience Performer Experience http://gdriv.es/armkeyboardaudienceformfin The audience experience questionnaires are distributed via the authors’ personal social networks on the internet, and the receivers may further distribute them to their networks. Performer Experience http://gdriv.es/armkeyboardperformerformfin To gather results from the performer experience, we invite people we know and who are available to participate

Evaluation Results Audience Experience Performer Experience There are totally 33 responses from improvisation evaluation and 31 from solo evaluation, 5 and 4 of them regard themselves as musicians or amateur musicians respectively. Performer Experience 15 people participate in the performer experience evaluation

Conclusion What have we done

THANKS FOR WATCHING!