Gestures for Gestures for the Science of Collaborative Arts the Science of Collaborative Arts Guerino Mazzola U & ETH Zürich

Facts Facts Processes Processes Gestures Gestures Collaboration Collaboration to refer to to be the case to make to interact

Facts Facts Processes Processes Gestures Gestures Collaboration Collaboration

Point of departure: free jazz vs. maths Point of departure: free jazz vs. maths Splitting objective facts from subjective action

Mathematical Music Theory (MaMuTh) Books: Books: Gruppen und Kategorien in der Musik (1985) Gruppen und Kategorien in der Musik (1985) Geometrie der Töne (1990) Geometrie der Töne (1990) The Topos of Music (2002) The Topos of Music (2002) La vérité du beau dans la musique (2006) La vérité du beau dans la musique (2006)Software: presto ® (Atari 1988) presto ® (Atari 1988) Rubato ® (NEXTSTEP 1983, Mac OS X 1998, Java 2002) Rubato ® (NEXTSTEP 1983, Mac OS X 1998, Java 2002)

MaMuTh Models Tonal Modulation (Muzzulini, Straub, Noll, Radl) Tonal Modulation (Muzzulini, Straub, Noll, Radl) Fux Counterpoint (Muzzulini, Hichert, Wieser, Brunner) Fux Counterpoint (Muzzulini, Hichert, Wieser, Brunner) Riemann Function Theory (Noll) Riemann Function Theory (Noll) Riemann/GTTM Metric and Rhythms (Volk-Fleischer) Riemann/GTTM Metric and Rhythms (Volk-Fleischer) Reti Motif Theory (Straub, Buteau, Nestke) Reti Motif Theory (Straub, Buteau, Nestke) Ruwet-Nattiez Paradigmatic Composition (Blum, Milmeister) Ruwet-Nattiez Paradigmatic Composition (Blum, Milmeister) Classification and Enumeration Theory (Fripertinger, Noll) Classification and Enumeration Theory (Fripertinger, Noll) Performance Theory (Zahorka, Stange-Elbe, Müller) Performance Theory (Zahorka, Stange-Elbe, Müller) Denotator Concept Framework (Zahorka, Montiel, Göller, Milmeister) Denotator Concept Framework (Zahorka, Montiel, Göller, Milmeister)

Pitch classes in Ÿ 12 chords = truth values Pitch classes in Ÿ 12 truth values = chords I IVVIIIIIVIVII C (3)

I IV II VI V III VII Schönberg‘s „harmonic strip“ N(X (3) ) (Harmonielehre 1911) „ harmonic strip = Möbius strip“ is a logical fact!

Pattern, Precision and Profundity: it seems to me those three words characterize, to a great extent, the essence of mathematics. Additionally, I use those three words deliberately because I believe that they characterize much of music.... that there is much beauty to be beheld in mathematical structures, and that the issues of mathematical truth and beauty are profoundly intertwined. I am sure that these issues also have much to say about music, but these are matters that I do not feel competent to address. Roger Penrose (1995): Hermann Weyl James J. Sylvester

John Keats (Ode on Grecian Urn, 1819): „Beauty is truth, truth beauty,“—that is all Ye know on earth, and all ye need to know. Alexander Grothendieck ( ): Topos Theory Unification of logic and geometry „Truth values are point sets“

n/16 a b c d e

Nerve N(X MAX ) of covering MAX of X by maximal local meters: N(X MAX ) ={  |  ={M 0,...M  },   M i  Ø}  = Simplex,  = Dimension von  X M0M0M0M0 MMMM Have sober topology on N(X MAX )  >  iff   

b Nerve N(X MAX ) of covering MAX = {a, b, c, d, e} e c a d a b c d e

0 b e c a d N(X MAX ) x dominates y iff Sp(y)  Sp(x) 6

w(x) =  M Œ Sp(x), m £ length(M) length(M) p m = p =

MetroRubette ®

Facts Facts Processes Processes Gestures Gestures Collaboration Collaboration

sets of notes Transformational Theory, K-nets (Lewin et al.) Conference „Autour de la Set Theory“ IRCAM/Paris Oct (J. Rahn, R. Morris, A. Forte,...)

{c,  (c),  2 (c),...} {c,  (c),  2 (c),...} = {c, e, g} = major triad Ÿ 12 Circle Chords c g  e  c = 0  (p) = 3p+7  Classification of circle chords (cf. Gruppen und Kategorien)

Third Chains Ÿ T3T3T3T3 T4T4T4T4 T4T4T4T4 T3T3T3T3 Classification of third chains (in Presto SW, cf. The Topos of Music)

Diagram in a category C C = sets, groups, vector spaces, automata, etc.  = digraph D :   C i j l m a ij t a il q a jm s a li p a jl k a ll r a ll r D i = objects in C D i = objects in C D ij t = morphisms in C D ij t = morphisms in C DiDiDiDi D jD jD jD j DlDlDlDl D mD mD mD m D ij t D il q D jm s D li p D jl k D ll r D C Nets of dodecaphonic series Multiply iterated nets

Ÿ 12 Example 1: K-nets of pitch classes C = Ab abelian groups + affine maps Ÿ T 11.-1/Id T 11.5/Id T 4 /Id T 2 /Id 3 724

Example 3: K-nets of dodecaphonic series C = Ab Ÿ 12 s Is Rs IRs T 11.-1/Id Id/T Ÿ 11 s

Ÿ 12 T 5.-1 T 9.-1 T0T0T0T0 T4T4T4T4 Ÿ 12 T T 3.-1 T0T0T0T0 T4T4T4T4 Ÿ 12 T4T4T4T4 T4T4T4T4 T0T0T0T0 T0T0T0T0 T2T2T2T2 T2T2T2T2 T4T4T4T4 T4T4T4T4

smsmsmsm sjsjsjsj slslslsl sisisisi FiFiFiFi FjFjFjFj FlFlFlFl FmFmFmFm f ij t D f il q f jm s f li p f jl k f ll r GiGiGiGi GjGjGjGj GlGlGlGl GmGmGmGm g ij t D*D*D*D* g il q g jm s g li p g jl k g ll r s FDFDFDFD FD*FD*FD*FD* FsFsFsFs Perspectives of New Music (2006) Guerino Mazzola & Moreno Andreatta: From a Categorical Point of View: K-nets as Limit Denotators Perspectives of New Music (2006) Guerino Mazzola & Moreno Andreatta: From a Categorical Point of View: K-nets as Limit Denotators Theorem: Using topos theory, we can show that networks of networks of... are networks: Arbitrary iterations possible without conceptual changes.

global networks Theorem: Isomorphism classes of global compositions are described by local networks of affine functions. COLLOQUIUM ON MATHEMATHICAL MUSIC THEORY H. Fripertinger, L. Reich (Eds.) Grazer Math. Ber., ISSN 1016–7692 Bericht Nr. 347 (2005), Guerino Mazzola: Local and Global Limit Denotators and the Classification of Global Compositions COLLOQUIUM ON MATHEMATHICAL MUSIC THEORY H. Fripertinger, L. Reich (Eds.) Grazer Math. Ber., ISSN 1016–7692 Bericht Nr. 347 (2005), Guerino Mazzola: Local and Global Limit Denotators and the Classification of Global Compositions Corollary: The theory of global compositions is absorbed by the theory of global networks. global compositions global compositions

Gérard Milmeister (chief programmer) Julien Junod (counterpoint) Florian Thalmann (morphing) Karim Morsy (macro objects)

Facts Facts Processes Processes Gestures Gestures Collaboration Collaboration

May 18th 2002 IRCAM talk Digraph Networks Transformations Paradigm ? Journal of Mathematics and Music (2007) Guerino Mazzola & Moreno Andreatta: Diagrams, Gestures, and Formulas in Music. Journal of Mathematics and Music (2007) Guerino Mazzola & Moreno Andreatta: Diagrams, Gestures, and Formulas in Music.

l h e sonic events score analysis instrumental interface √ thaw freeze (MIDI) instrumentalizeinstrumentalize position pitch timegestures

Folie 2 Czeslaw Marek: Die Lehre des Klavierspiels

Every Noh play is a cross section of the life of one person, the shite. Shite weaves a texture of fantasy using curves. The waki is a kind of cosubject and mirror person of the shite. Waki describes reality using straight lines.

Noh = expression of a space-time The semiotic of gestures is reversed: Gestures serve mainly the expression of movements within this space-time, the (usual) semantic is of second order.

— positionpitchtime 01  1111 2222  2 +  1  t.  2222 1111  1   2 I

x z y   1 1 (t ) )   6 6 ) )   2 2 ) )   3 3 ) )   4 4 ) )   5 5 ) ) One hand  product  =  1  2  3  4  5  6 of 6 gestural curves in space-time (x,y,z;e) of piano j = 1, 2,... 5: tips of fingers, j = 6: the carpus,  6 =  root parameter t  sequence of points:  (t) = (  1 (t),...,  6 (t)) two base vectors of fingers d 2, d 5 from carpus. e = time

Gestural interpretation of Carl Czerny‘s op. 500

The universe of gestures Paul Valéry...les ouvrages de la main prodigieuse de l‘ariste égale et rivale de sa pensée - l‘une n‘est rien sans l‘autre Paul Valéry...les ouvrages de la main prodigieuse de l‘ariste égale et rivale de sa pensée - l‘une n‘est rien sans l‘autre Charles AlunniCe n‘est pas la règle qui gouverne l‘action diagrammatique, mais l‘action qui fait émerger la règle. Charles AlunniCe n‘est pas la règle qui gouverne l‘action diagrammatique, mais l‘action qui fait émerger la règle. Jean CavaillèsComprendre est attraper le geste et pouvoir continuer. Jean CavaillèsComprendre est attraper le geste et pouvoir continuer. Pierre BoulezMais on est passé depuis Adorno de l‘informel à l‘informulé, de l‘objet au geste. Pierre BoulezMais on est passé depuis Adorno de l‘informel à l‘informulé, de l‘objet au geste. Ludwig WittgensteinI think with my pen. Ludwig WittgensteinI think with my pen. Cecil TaylorThe body is in no way supposed to get involved in Western music I try to imitate on the piano the leaps in space a dancer makes. Cecil TaylorThe body is in no way supposed to get involved in Western music. I try to imitate on the piano the leaps in space a dancer makes. Robert S. HattenGiven the importance of gesture to interpretation, why do we not have a comprehensive theory of gesture in music? Robert S. HattenGiven the importance of gesture to interpretation, why do we not have a comprehensive theory of gesture in music? Francis BaconThe marks are made, and you survey the thing like you would a sort of graph. And you see within this graph the possibilities of all types of fact being planted. Francis BaconThe marks are made, and you survey the thing like you would a sort of graph. And you see within this graph the possibilities of all types of fact being planted. Bertold Brecht...non verbis sed gestibus. Bertold Brecht...non verbis sed gestibus. Theodor W. AdornoDer Ausdruck ist bereits die Rationalisierung des Gestischen, d.h. dessen Objektivierung durch Signifikation. Theodor W. AdornoDer Ausdruck ist bereits die Rationalisierung des Gestischen, d.h. dessen Objektivierung durch Signifikation.

„Abstraktmotiv“ Volker Spicker: New Piano Improvisation (Giessen 1997)

Hugues de Saint-Victor (Parisian theologian XIIth century): Gestus est motus et figuratio membrorum corporis, ad omnem agendi et habendi modum. Gesture is the movement and figuration of the body's limbs, according to whatever modality of action and attitude

Have category of gestures gesture = morphism g:   of digraphs X  position pitch time X g For a topological space X, we have the digraph of continuous curves  : I  X X  „  -gesture in X“

TipSpace positionpitchtime   What about real shapes?

circle Answer: Hypergestures! knot „loop of loops“

space space time ET-dance Gesture

X = torus  g h Hypergesture impossible!

Fact: For a a digraph  and topological space X, the set of  -gestures in X bears a canonical topology (essentially the compact-open topology).  X  „gestures of gestures of gestures of....“ of....“

homotopiccurves: homotopy classes of curves X Gestoids: Transition from gestures to algebraic structures connecting gestures composition of homotopy classes of curves is associative! X

The homotopy classes of curves of a gesture g, together with their composed curves, give rise to a category, the groupoid Gr g of the gesture. Ist objects are the vertices x,y,... of g, while the morphisms Gr g (x,y) are the homotopy classes c of composed curves from x to y. The gestoid G g is the ¬ -linear category  ø¬ Gr g. The morphism G g (x,y) are the formal ¬ -linear combinations  n a n c n of morphisms c n in Gr g (x,y). Too abstract?

e i2 p t — i—i—i—i—1 i X = S 1 G g  ¬ p 1 (S 1 ) fundamental group p 1 (S 1 )   Ÿ e i2 p nt ~ n ~ Fourier representation f(t) =  n a n e i2 p nt ~ Fourier representation f(t) =  n a n e i2 p nt  n a n e i2 p nt Gestures= Musical String Theory Musical String TheoryGestures= g:

Facts Facts Processes Processes Gestures Gestures Collaboration Collaboration

Jazz: Collaboration as a performing art R.A.R. MacDonald & G.B. Wilson: Construction of jazz: how jazz musicians present their collabortive musical practice. Musicae Scientiae X, Nr.1, Spring 2006Jazz: Collaboration as a performing art R.A.R. MacDonald & G.B. Wilson: Construction of jazz: how jazz musicians present their collabortive musical practice. Musicae Scientiae X, Nr.1, Spring 2006 Flow (Mihaly Csikszentmihalyi) is being described as „being completely involved in an activity for its own sake. The ego falls away. Time flies. Every action, movement, and thought follows inevitably from the previous one, like playing jazz.“ Jazz improvisation as flow: „...the music will work itself“ „...goes into autopilot“ „...stream of consciousness“ B..J. Kenney M. Gellrich: Improvisation. In: R. Parncutt and G.E. McPherson (eds.) The science and psychology of musical performance. Creative Strategies for teaching and learning. New York: OUP 2002Flow (Mihaly Csikszentmihalyi) is being described as „being completely involved in an activity for its own sake. The ego falls away. Time flies. Every action, movement, and thought follows inevitably from the previous one, like playing jazz.“ Jazz improvisation as flow: „...the music will work itself“ „...goes into autopilot“ „...stream of consciousness“ B..J. Kenney M. Gellrich: Improvisation. In: R. Parncutt and G.E. McPherson (eds.) The science and psychology of musical performance. Creative Strategies for teaching and learning. New York: OUP 2002 chick-sent-me-high-ee

Francis Bacon (relating to the diagram debate): In the way I work I don’t in fact know very often what the paint will do, and it does many things which are very much better than I could make it do. Is that an accident?Francis Bacon (relating to the diagram debate): In the way I work I don’t in fact know very often what the paint will do, and it does many things which are very much better than I could make it do. Is that an accident?... What has never yet been analysed is why this particular way of painting is more poignant than illustration. I suppose because it has a life completely of its own. (...) So the artist may be able to open up or rather, should I say, unlock the valves of feeling. D. Sylvester: Interview with Francis Bacon: The Brutality of Fact. Thames and Hudson, New York 1975

Thesis: Gestures help articulate/communicate articulate/communicate organize/structure organize/structure intensify/deepen the flow in collaborative arts intensify/deepen the flow in collaborative arts

Thank you! Download this talk from