Post RDC.2 research update [#8]
Thesis title—"On the Sound of Becoming: music in the course of mediated art, entertainment, and communications"
Argument : Emergence
the "common denominator"
Emergence: science & art perspectives
Emergent behavior is generated through "bottom-up" processes rather than by "top-down," hierarchical control. (Johnson, 2001)
Emergence is a fundamental aspect of contemporary digital art works, and can arise from a variety of sources, "ordering itself from a multiplicity of chaotic interactions." (Ascott, 1993)
Emergence: media & art
On the Internet, he cites Slashdot (http://slashdot.org), which exhibits an emergence in its organizational schema.
Wiki, and to a certain extent, web log (blog) systems as ideas coalesce across communities.
Contemporary computer game design.
"The Sims" by Will Wright
In a-life art, emergence "...arises (as) a result of a different order, an image or form that strikes us as novel, significant, beautiful, or surprising." (Whitelaw, 2004)
"Eden" by Jon McCormack
Emergence: Spore (by Will Wright)
Algorithms procedurally generate characters and environments in response to player decisions. (Park 2006, Wikipedia) Wright describes game development in biological terms.
"Spore" creature editor by Will Wright
Argument : Sound & Music
In media and artistic works, audio elements provide an additional channel of communication to reach an audience, user, or participant.
Sound & Music: cinema
Field has little to no "native" theory of sound, but ideas can be usefully borrowed from cinema.
Elements of sound and video are inextricably linked in a viewer's mind. Michel Chion (1994) describes this as a "simultaneous vertical relationship" between audio and narrative.
Sound & Music: sonification
Sonification "is the use of non-speech audio to convey information." (Hermann, 2004) As a field it seeks to make real-time data readable via sound.
WebMelody (Barra et. al., 2002) was designed to provide an auditory means of monitoring web servers by providing information to describe the server's workload, report severe errors, and denote normal behavior.
WebMelody model
Sound & Music: computer games
The use of audio to communicate "state" has been central to the work of composers writing for computer games since the earliest days of the genre.
"Asteroids" screen capture
In "Creatures" (by Steve Grand; music by Peter Chilvers) music builds the mood and atmosphere, and that it compensates "...for the lack of information to other senses such as smell and touch. It can also impart information about thoughts and characters that is not otherwise evident."
"Creatures" screen capture
Argument : Linearity in contemporary practice
The critique driving this research primarily addresses the incongruity of linear structures within a non-linear medium and is most clearly illustrated by the discourse of contemporary game design.
- Kurt Harland refers to "conceptual problems of interactive game scores," where listener expectations of "good" music do not necessarily match the sounds and structures best suited to these kinds of works. (2000)
- Harland observes that the vast majority of this music is composed to match the western musical traditions of a fixed form that leads listeners on an ideal musical journey.
- Interactive Audio Special Interest Group (www.iasig.org) & Sonify.org (www.sonify.org)
These groups support writing and discussion surrounding the improvement of sound practice in digital media and online environments.
Argument : Musical traditions of non-linearity
Musical linearity dominates the compositional approach used by most composers writing for interactive media, especially computer games. A rich history of music and sound art practice points to a variety of potentially fruitful alternatives.
Musical non-linearity: natural processes
The experience of the Japanese suikinkutsu is not entirely focused on the sound of the drip, but on the listening experience created in anticipation of the drip. While waiting, attention is drawn to the sounds of the garden environment; "...people forgot their ordinary time sensations little by little during the visits in the Japanese-style garden." (Imada, 1994)
Suikinkutsu construction
"Score for a Hole in the Ground" (www.scoreforaholeinthegound.org) In this work, Jem Finer designed and built a collection of metal instruments that sound when "played" by water dripping into a hole in the ground.
"Score for a Hole in the Ground" proposal sketch
"Score for a Hole in the Ground" site photograph & horn
These works rely on indeterminate, natural processes to produce a novel and unpredictable musical experience drawing on environmental conditions.
Musical non-linearity: music + a-life research
Eduardo Miranda and the Interdisciplinary Centre for Computer Music Research (ICCMR) at the University of Plymouth.
- CAMUS (Cellular Automata MUSic) uses dual, parallel cellular automata (CA) to generate musical forms by mapping x-y coordinates of a Cartesian system onto musical semi-tones. (Miranda, 1993)
- Chaosynth uses the behavior of a CA to control the production of sound granules in a Granular synthesizer. (Miranda, 2003) According to Miranda, CA dynamics are similar to the production of natural and instrumental sounds, in that sound particles start in disarray and then converge into a sonic order.
Chaosynth 'CA dynamics -to-grain' diagram
Musical non-linearity: experimental music
Earle Brown found a great deal of inspiration in the mobiles of sculptor Alexander Calder. He described them as, "...transforming works of art" (Bailey, 1992) that lent the work a different configuration but the same overall quality upon each viewing. (Nyman, 1999) Brown was interested in approaching music with an openness that allowed every performance to render a unique musical output that retained the essential character of the work.
Alexander Calder "Red Lily Pads" 1956
Terry Riley's In C illustrates John Cage's thoughts on Experimental music. He writes that the "experiment" is essentially a composition—"the outcome of which is unknown." (Cage, 1973) When performed, In C has indefinite outcomes and yet is always recognizable due to the "personality" of the composition.
Musical non-linearity: free improv
Genre took root in Europe in the early 1960s with London, England as the hub of its development. (Bailey, 1992)
David Borgo (2005) describes an improvising ensemble as an "open system" that emerges from bottom-up processes
Musical non-linearity: generative music
Musician, composer, and visual artist Brian Eno has been working with a variety of generative structures throughout his career.
Eno sees Riley's In C, or anything where the composer makes no top-down directions, as precursors to generative music. No detailed directions are not provided; instead there is "a set of conditions by which something will come into existence." (Toop, 2004)
The idea of "...making seeds rather than forests," and "...letting the forests grow themselves"
77 Million Paintings by Brian Eno
Long Now Foundation: In conversation with Will Wright, Eno explains that it is important to set constraints so that the generative system is able to produce what its creator (and hopefully others) will find to be interesting, "You have to care about your inputs and your systems a lot more since you aren't designing the whole thing (you are not specifying in detail the whole thing) you're making something that by definition is going to generate itself in a different way at different times." (Eno & Wright, 2006)
Argument : Potential of becoming
In the contemporary landscape of media and art, there are works with the embedded potential for music that shares in the emergence of the medium.
Potential of becoming: contemporary works
Eden, PANSE, Rez, Intelligent Street
Electroplankton, created by Toshio Iwai. The idea draws on Iwai's fascination with different objects across the course of his life—a microscope, a tape recorder, a synthesizer, and the Nintendo Entertainment System (NES). (Nintendo, 2006)
Electroplankton species Hanenbow
Potential of becoming: composition-instrument
As it concerns my personal work and research, I see the merger between composition and instrument as a way to move musical thinking forward. In the case of Iwai's Electroplankton, interactions turn the Nintendo DS into an instrument that can be played purposefully through the manipulation of the onscreen animations. Simultaneously, the software programming that links sounds to plankton species and their environment represents an implicit musical ordering or composition. While Electroplankton was not conceived of the specific idea, it perfectly illustrates how the combination or blurring of composition and instrument can lead to an interactive work with profound musical potential.
Thesis
Key thoughts: resistance & perturbations
The research detailed in this report leads the way to a doctoral thesis that will present the conceptual framework of composition-instrument and show how it can be used to catalyze musical opportunities within mediated art, entertainment, and communication systems. In a composition-instrument work, musical becoming is achieved through the tension between the fixedness of a composition and the potential afforded by an instrument. This tension is similar to what Aden Evens refers to as the "resistance of the instrument." (2005) In a composition-instrument work, this takes the form of a generative system that is linked to user, participant, or player input in such a way as to affect the work's musical unfolding. The link can also be viewed through the lens of Maturana and Varela's structural coupling (1992), where ontogenetic unities are represented by the generative system and the person engaged in the work. An interaction makes a perturbation, and the generative system adapts to the environmental changes it creates. The system pushes back in the form of sound, exposing its compositional autonomy in what can be perceived as an invitation for additional input. As these mutual perturbations or series of interactions progresses, the character of the system and the meaning it carries is revealed as sound, both in terms of what it is currently and what it could become.
This quality is consistent with the dynamics of the medium but not with the musical methods that are often employed. Current research points to gaps in the field where a composition-instrument approach would help show the way forward. At this point there is a need for additional research to help more clearly identify these gaps and a need for theory of a conceptual framework that can fill them.
Research Intentions
In the course of M.Phil studies, I have discovered an approach to music that complements the emergence of an interactive experience. This behavior is found in the interactive systems Roy Ascott describes as "non-trivial," where there is an "open-ended capacity to accommodate new variables." (2003) As research for the doctoral thesis goes forward, I intend to build a theory of this approach and situate that theory in the compositional practice of the field.
The theory I seek to develop is what Michael Hamman refers to as an "open episteme," (1999) a porous understanding, "open to input from a particularized situation." A composition-instrument framework leads to such an understanding because the interactive systems that facilitate this approach are themselves open, and produce unique outputs relative to the myriad interactions conducted within them. This makes the frame for understanding the system emergent—its outputs will always be specific to an interaction at a certain time, in a certain place, and under certain conditions. As the parameters surrounding the interaction shift, the system's output shifts. And most significantly, the person engaged in interaction experiences an emergent shift in their reception of that output and in their mental model of the system and its possibilities.
This theory will emphasize the confluence of emergent phenomena on several different planes: interaction, generation, improvisation, performance, and composition. I aim to identify a mode of compositional thought and action that relates music to the dynamics that arise in the course of mediated interactive experience.
Research Presentations
Audio Mostly Conference : Interactive Institute Sonic Studio (Piteå, Sweden)
"Many people perceive improvised or generative music as chaotic. How can you address that in your work?"
- This concern is tied into the idea that all music, in terms of its "goodness," is critiqued along the same lines—listening. If people like to listen to it then it's good; if they don't then it's not good.
- In my work, the experience isn't about listening it's about experiencing while in the flow of doing something else in which they are 1) actively engaged, and 2) have a direct responsibility in creating what it is they are listening to. The passivity is completely removed in lieu of a direct engagement with the environment/apparatus that creates the music.
- Istanbul presentation at Sabanci: The idea of listening modes and "using" music helps reveal some of what's at work here. People get into the mode of listening to music passively—it is their "experience creator." They play it and let it work its magic. What I propose has a completely different intent.
- With a composition-instrument work, a player/user/listener/composer is directly engaged with the music and is making the music as they go along. I have compared this to the experience of someone performing in an improvising ensemble. In this situation, new directions, meaning, and understanding emerges as a result of a player's contribution to the total musical output. As they provide a new foreground or background the overall musical texture (timbre, tonality, etc.) shifts and adopts a new character as DIRECT RESULT of their intervention. There is nothing passive about this.
demonstrating Electroplankton at Audio Mostly via video conference
Response to Piteå feedback: A new idea of listening modes ?
"listening-listening" (music for music's sake; Kivy's "pure listening")
"using-listening" (social/behavioral/cultural/commercial)
"playing-listening" (in musical performance)
"cinematic listening" (media)
"interaction-listening" (combines using, playing, and cinematic modes)
Claudia Gorbman "Unheard Melodies" (1987):
"To judge film music as one judges 'pure' music is to ignore its status as a part of the collaboration that is the film. Ultimately it is the narrative context, the interrelations between music and the rest of the film's system, that determines the effectiveness of film music." (p.12)
She cites listening in:
pure musical codes: listening to a musical discourse or musical structure itself
cultural musical codes: listening to something in a coffeehouse or appreciating music for the cultural connotations it communicates
cinematic musical codes: music that "bears specific formal relationships to coexistent elements in the film."
References
Johnson, S., Emergence: the connected lives of ants, brains, cities, and software. 2001, New York: Scribner.
Ascott, R. and E.A. Shanken, Telematic embrace: visionary theories of art, technology, and consciousness. 2003, Berkeley: University of California Press.
Whitelaw, M., Metacreation: art and artificial life. 2004, Cambridge: MIT Press.
Park, A., Will Wright Talks Spore, Leipzig, Next-gen, in www.gamespot.com/news/6155498.html. 2006.
Chion, M., C. Gorbman, and W. Murch, Audio-vision: sound on screen. 1994, New York: Columbia University Press.
Hermann, T. and H. Ritter. Sound and Meaning in Auditory Display. in Proceedings of the IEEE. 2004.
Barra, M., et al. Personal WebMelody: customized sonification of web servers. in Proceedings of the 2001 International Conference on Auditory Display. 2001. Espoo, Finland.
Harland, K., Composing for Interactive Music, in Gamasutra.com. 2000.
Imada, T., The Japanese Sound Culture. The Soundscape Newsletter, 1994(9).
Miranda, E.R., Cellular Automata Music: an interdisciplinary project. Interface, 1993. 22: p. 3-21.
________. On the Music of Emergent Behavior: what can evolutionary computation bring to the musician? Leonardo, 2003. 36(1): p. 55-59.
Bailey, D., Improvisation: its nature and practice in music. Rev ed. 1992, New York: Da Capo.
Nyman, M., Experimental music: Cage and beyond. 2nd ed. 1999, Cambridge, U.K.; New York: Cambridge University Press.
Cage, J., Silence: lectures and writings. 1973, Middletown, Conn.: Wesleyan University Press.
Borgo, D., Sync or swarm: improvising music in a complex age. 2005, Continuum: New York.
Toop, David. Haunted weather: music, silence, and memory. 2004, London: Serpent's Tail.
Eno, B. and W. Wright, "Playing with Time". 2006, The Long Now Foundation.
Nintendo of America, Electroplankton instruction Booklet. 2006, Redmond: Nintendo.
Evens, A., Sound ideas: music, machines, and experience. Theory out of bounds; v. 27. 2005, Minneapolis: University of Minnesota Press.
Maturana, H.R. and F.J. Varela, The tree of knowledge: the biological roots of human understanding. Rev. ed. 1992, Boston, New York: Shambhala.
Hammman, M., From Symbol to Semiotic: Representation, Signification, and the Composition of Music Interaction. Journal of New Music Research, 1999. 28(2): p. 90-104.
Gorbman, C., Unheard melodies: narrative film music. 1987, Bloomington: Indiana University Press.