1. ArmKeyBoard A Mobile Keyboard Instrument Based on Chord-scale System
and Tonal Hierarchy

2. Why making armkeyboard
Introduction
Why making armkeyboard

3. Traditional Piano Keyboard
Large size (non-portable)
Difficult to learn (chord-scale expressions)
Linearity (rules out non-linearity)
Traditional Piano Keyboard

4. Can we redesign the keyboard so that it is easy to carry around and easy to make good music at the same time?

5. Existing Mobile Piano Keyboards
Simply copying piano layout
Fix scale
Linearity
Existing Mobile Piano Keyboards

6. Existing Mobile Piano Keyboards
Not easy to learn
Existing Mobile Piano Keyboards

7. ArmKeyBoard Portable “Instant music, subtlety later”
Embrace non-linearity

8. Two Types of Keyboard Layouts
Linear layout and non-linear layout

9. Linear Piano Keyboard Layout
C C# D D# E F F# G G# A A# B

10. Linear Piano Keyboard LayoutC C# D D# E F F# G G# A A# B
Linear Layout Chromatic Scale

11. Non-linear Piano Keyboard LayoutC C# D D# E F F# G G# A A# B
Non-linear Layout Chromatic Scale

12. Three characters for making mobile smart instruments
Mobile Smart Phone
Three characters for making mobile smart instruments

13. Three Facts on Mobile Smart Phone
Most users are non-musicians (calls for flat learning curve)
The screen is small (If all 88 keys are to be equally distributed on an iPhone 5 screen, each key has only a space of about 82mm2 (less than 1cm2) )
The device is programmable (can implement any possible layouts and mappings)

14. Musical Assumptions
if we assume that within any short range of a musical process, all the perceptible notes must belong to a certain chord- scale if this short range itself belongs to a larger meaning group, then the keyboard should minimally play one chord- scale at a time.
For different music forms, we need to make different musical assumptions during the design

15. The design of Armkeyboard’s musical note space and control mechanism

16. Chord, Scale and Octave
ArmKeyBoard treats the small screen space as a cache, for caching the currently playing chord- scale in the current octave range
Other octaves and chord-scales are waiting to be loaded when needed.
Users can switch between different chord- octave-scales using a gravityX gesture

17. Chord-octave-scale Sequence Grid
The sequence goes from left to right, and then from top to bottom
The sequence can be saved as preset
Every grid element stands for a single combination of chord, octave and scale
Chord-octave-scale sequence grid
and chord-octave-scale presets

18. Control Mechanisms
Gravity X gesture, which is used for switching to the next or previous page of notes determined by the chord-octave-scale combination at the next or previous square within the sequence.
Gravity Y gesture (on the left), which is used for
controlling note velocity, leading to a smaller velocity with a larger angle to the horizontal plane; Gravity Z gesture (on the right), which is used for quitting the current keyboard to reset everything again

19. Key-note Mapping
How to map keys to notes

20. Linear Mapping and Non-linear Mapping
Mapping Overview
ArmKeyBoard linear layout contains 15–17 notes within the active chord-octave- scale and they are mapped linearly to 15– 17 bars equally divided along the y-axis. The velocity is controlled by the X position.
ArmKeyBoard Non-linear layout starts with a user specified image. The image is then algorithmically divided into contours and these contours are then algorithmically mapped to the 15–17 notes within the currently active chord-octave-scale.
Linear Mapping and Non-linear Mapping

21. Non-linear Mapping Algorithm
Contour separation
Contour Ranking
Tonal Hierarchy
Final Mapping

22. Before and after contour separation

23. Contour Ranking
The minimal musical concern is, when the keyboard is being played, the notes being generated should at least imply the currently active chord-scale most of the time.
The assumption is that most user tends to tap on: 1, a contour with a larger area; 2, a contour closer to the center of the screen; 3, a contour that contains more sub- contours.
Based on how often most users will tap on a certain contour, the importance of a contour can be determined.
This actually corresponds to how important a note is in implying a certain chord-scale, which is discussed below.

24. Tonal Hierarchy
Tonal Hierarchies in ArmKeyBoard. L1 is the first level of notes which are to be mapped to regions with highest importance, and L2 to be mapped to regions with second highest scores, then L3 to be mapped to the least important regions

25. Final Mapping

26. Final Mapping 2 6 1’ 3 5 3’ 7 1 5’ 7’ 4 2’ 6’ 4’

27. Final Mapping 4 2 6 2’ 6’
1,3,5,7
1’,3’,5’ 7’
1’’

28. Video Demo
How it sounds like

29. ArmKeyBoard Demo

30. How do the users rate armkeyboard
Evaluation
How do the users rate armkeyboard

31. Evaluation Framework Audience Experience Performer Experience
The audience experience questionnaires are distributed via the authors’ personal social networks on the internet, and the receivers may further distribute them to their networks.
Performer Experience
To gather results from the performer experience, we invite people we know and who are available to participate

32. Evaluation Results Audience Experience Performer Experience
There are totally 33 responses from improvisation evaluation and 31 from solo evaluation, 5 and 4 of them regard themselves as musicians or amateur musicians respectively.
Performer Experience
15 people participate in the performer experience evaluation

33. Conclusion
What have we done

34. THANKS FOR WATCHING!