Anxiety of Influence: Artistic Identity in Interactive New Media Robert Griffin Byron Department of Music / MEME Brown University May 2013 © Copyright 2013 by Robert Griffin Byron. iii This  dissertation  by  Robert  Griffin  Byron  is  accepted  in  its  present  form   by  the  Department  of  Music  as  satisfying  the   dissertation  requirement  for  the  degree  of  Doctor  of  Philosophy.         Date  _____________                                                                                                                                  _____________________________________                                                                                                                                                                                              Todd  Winkler,  Advisor             Recommended to the Graduate Council   Date  _____________                                                                                                                                  _____________________________________                            Roger  Dannenberg,  Reader         Date  _____________                                                                                                                                  _____________________________________                            Luke  DuBois,  Reader         Date  _____________                                                                                                                                  _____________________________________                       Butch  Rovan,  Reader   Approved by the Graduate Council   Date  _____________                                                                                            __________________________________________________                                Peter  Weber,  Dean  of  the  Graduate  School     iv Abstract Motivated by questions posed to the new media artist, concerning the difficulties with coming up with an innovative artistic style in interactive media performance, this dissertation takes as a departure point a self-created multimedia work for a dancer, 5.1 surround sound, and 3-channel HD interactive projection. The work uses a custom-built wireless gestural interface to translate kinetic energy from the performer, into real-time multichannel synthetic sound and real- time OpenGL graphics. Throughout my preliminary research I became very much aware of the importance of influence on not only creative endeavors in general, but also the influential nature of the ever-evolving technologies available. This paper attempts, firstly, to place the new work in the context of these available technologies, within a sociological context for the contemporary artist, and secondly, to analyze how personal artistic style and voice can be transposed through so many different converging elements, all the while, encompassing design, technological enhancements & spontaneous performance. The over-arching thesis was to observe the journey of self-discovery of creating the piece, to discover how it fits in with the history of interactive media and new media art today. In its entirety, the paper follows a dual purpose. Firstly, it attempts to formalize a perception of the self in this century, via the convergence of mind and body, through its relationship to software and hardware. Any deduced conclusions are then pitted against an analysis of the work itself. v In conclusion, it is useful to re-evaluate the juxtaposition of disparate media paradigms, as reconciled by the human body, to create a new way of thinking about the marriage between the moving digital image and digital sound in the context of time - as instigated by the translation of kinetic movement. Keywords: new media, intermedia, openGL, 3D animation, live performance, electro-acoustic performance, audiovision, interactive design, UX design, embedded development, Mapping techniques, digital performance, dance performance, human/computer interaction. sound design, real-time sound & image, surround sound vi Curriculum Vitae Robert was born in Melbourne, Victoria Australia and is a multimedia artist working with digital media, sound, 3D animation, painting, sculpture, cast glass, textiles, and video to create interactive installations and audiovisual art that explores the relationship between the organic and the synthetic, often through the deconstruction and abstraction of the world through physical computing technologies. Drawing inspiration from natural forms, from molecular mechanisms to the five elements of traditional philosophy, he attempts to understand through his art the hidden structures of manifest nature. Byron has a B.Mus. in composition from the Western Australian Academy of Performing Arts, a M.M. in computer music composition from Indiana University while on a Fulbright, and a M.A. in creative arts from Edith Cowan University. In 2000, he received a Peggy Glanville-Hicks Composer’s Fellowship-in-Residence. At Indiana, he won the 2005 Dean’s Prize for Electroacoustic Composition. His chamber music and orchestral works have been heard across Australasia and the United States. To date, he has received six professional commissions, firstly, 1997, the Western Australia Ballet commissioned the score for the full-length ballet Orlando, then in 1998, Future Films commissioned a soundtrack for an art film by Glen Eaves called Structures. The score won the Australian Screen Music Award in 1999. Also in 1999, the Australian Ballet commissioned the full-length ballet Mirror Mirror. Robbie currently is the Artistic Director for the non-profit arts organization based in Pittsburgh, Art Coalesce. Further information can be found at robbiebyron.com vii Acknowledgments I am grateful to my supervisor, Todd Winkler, whose helpful advice, suggestions, criticism – and patience! – were instrumental in me tackling an enormous endeavor that was of great interest, but new to me. It was a pleasure working with him. I am also indebted to Butch Rovan, for his support and unbounded knowledge in the field. I am thoroughly grateful for the assistance from Roger Dannenberg and Luke DuBois, who were so generous in finding time in their busy schedules, whose guidance and point of view were instrumental to my work. I would also like to thank profusely all my teachers, particularly Jim Moses, Paul Myoda, Jocelyne Prince, Eva Sutton, Eric Singer and Dennis Miller, who’s amazing skill and tutelage was inspiring and a great source of motivation. I am deeply indebted to Kathleen Nelson, who made sure I dotted all my I’s and crossed all my T’s through my whole time at Brown. I could have not got through this journey without her. And lastly, I wish to acknowledge the undying support from my partner Jason, who had faith in me the whole way through and gave me the strength to conquer many obstacles. viii Table of Contents 1.0   INTRODUCTION:  ANXIETY  OF  INFLUENCE:  THE  PROJECT’S  OVERALL  CONCEPT  ................  1   2.0   THE  PROBLEM  OF  IDENTITY  FOR  THE  CONTEMPORARY  ARTIST  ......................................  6   2.1   Back  to  failure  ...............................................................................................................................................................  8   2.2   Data  Cacophony  and  Glitch  as  a  Contemporary  Paradox  of  Influence  ...................................................  13   3.0   THEORETICAL  AND  HISTORICAL  CONTEXT  OF  IDENTITY  ................................................  15   3.1   Self-­‐Awareness,  Identity  and  Artistic  Voice  ....................................................................................................  15   3.2   Finding  our  own  Individual  Essence  ..................................................................................................................  18   3.3   Artistic  Value  ..............................................................................................................................................................  19   3.4   Duality  as  Aesthetic  .................................................................................................................................................  21   3.5   The  Doppelganger  ....................................................................................................................................................  27   3.6   Liveness  .......................................................................................................................................................................  28   3.7   Resistance  ...................................................................................................................................................................  32   4.0   INTERMEDIALITY:  A  CURRENT  HISTORICAL  CONTEXT  OF  DANCE  AND  TECHNOLOGY  ....  35   4.1   Motion  Tracking  ........................................................................................................................................................  36   4.2   Mounted  Sensor-­‐based  Technologies  ................................................................................................................  41   ix 4.3   Bridging  the  two  .......................................................................................................................................................  45   4.4   The  Future  of  Dance  Technology  ........................................................................................................................  47   5.0   TECHNICAL  CONCERNS  FOR  ANXIETY  OF  INFLUENCE  ....................................................  50   5.1   Preliminary  considerations  ..................................................................................................................................  50   5.2   Hardware,  Software  Configuration,  Set  Design  and  Costume  ...................................................................  51   5.3   Sculpt  2.0:  A  Wireless  Sensor  Suit  .......................................................................................................................  54   5.4   Outline  and  features  of  the  microcontroller  ...................................................................................................  57   5.5   Sculpt  2.0  documentation  ......................................................................................................................................  58   5.6   Innovations  in  the  C  code  .......................................................................................................................................  59   5.7   Initial  Data  flow  and  treatments  .........................................................................................................................  62   5.8   Mapping  Strategies  ..................................................................................................................................................  73   5.9   Initial  Setup  of  the  5.1  Surround  Configuration  and  Its  Mapping  ............................................................  75   5.10   Kinetic  movement  data  as  3D  co-­‐ordinate  transformations  ...................................................................  77   5.11   Sound  Mappings  .....................................................................................................................................................  80   5.12   Other  Miscellaneous  Mappings  .........................................................................................................................  81   5.13   The  Rendering  Pipeline  for  Acousmatic  Production  .................................................................................  83   5.14   Workflow  pipeline  for  3D  sculpting  ................................................................................................................  85   x 5.15   Research  into  choreographers  towards  deciphering  a  style  ..................................................................  88   5.16   Final  Thoughts  About  Choreography  ..............................................................................................................  92   6.0   A  DETAILED  STRUCTURAL  ANALYSIS  OF  ANXIETY  OF  INFLUENCE  ..................................  93   6.1   Preliminary  description  of  the  performance  ..................................................................................................  93   6.2   Section  2  -­‐  Completion  and  Antithesis  ...............................................................................................................  97   6.3   Bridge  .........................................................................................................................................................................  101   6.4   Section  3  –  Reduction  and  Inception  ................................................................................................................  103   6.5   Section  4  –  Metamorphosis  to  Individuality  ..................................................................................................  107   6.6   Section  5  –  Isolation,  Failure  and  Fruition  .....................................................................................................  109   6.7   Section  6  –  Future,  Graduation  &  Recreation  ................................................................................................  110   6.8   Self  Critique  ..............................................................................................................................................................  112   7.0   CONCLUSION  ..............................................................................................................  115   7.1   On  the  Road  to  Becoming  .....................................................................................................................................  115   7.2   Critique  of  Premiere  Performance  ...................................................................................................................  117   7.3   The  Future  for  Anxiety  of  Influence  and  Sculpt  ...........................................................................................  117   REFERENCES  ........................................................................................................................  119   xi Table of Figures Figure 1.   Space, body and screen................................................................................................22   Figure 2.   Here the window-screen is broken. .............................................................................23   Figure 3.   The last frozen image of the piece ...............................................................................24   Figure 4.   Downie’s example of data flow (p. 258) .....................................................................38   Figure 5.   Section three of Lucidity .............................................................................................40   Figure 6.   Schacher’s wireless sensor bracelets ...........................................................................43   Figure 7.   Standing Waves; a system exploring embodiment ......................................................44   Figure 8.   Schematic structure of a multi-tiered mapping. ..........................................................46   Figure 9.   The Granoff Center for the Creative Arts....................................................................51   Figure 10.   The back glass wall of the Studio 1 space .................................................................52   Figure 11.   Blackout-screens lowered. .........................................................................................53   Figure 12.   The tiered seating and mixing position being installed. ............................................53   Figure 13.   Sculpt 2.0 and outline of sensor positions. ................................................................55   Figure 14.   Flow diagram for the dissemination of data ..............................................................56   Figure 15.   The circuit board and its housing ..............................................................................59   Figure 16.   Task allocation breakdown ........................................................................................63   Figure 17.   The serial object collecting at 15ms via [qmetro]. ....................................................65   Figure 18.   Sensor Conditioning, smoothing, scaling and averaging. .........................................66   Figure 19.   Gating on/off sensor data...........................................................................................68   Figure 20.   The Render Context, Window setup .........................................................................69   xii Figure 21.   The transport object dispersing rhythmic subdivisions .............................................70   Figure 22.   Template abstraction for opening and closing a section ...........................................71   Figure 23.   Setting up the renderer’s window ready for performance .........................................72   Figure 24.   The [bline] object ......................................................................................................74   Figure 25.   Markov Chain patcher ...............................................................................................74   Figure 26.   Diagram used to keep up with data routing ...............................................................75   Figure 27.   Base Management and the LFE channel configuration .............................................76   Figure 28.   Sensor Panning abstraction........................................................................................77   Figure 29.   Two NURBS meshes manipulated by sensor data. ...................................................78   Figure 30.   Sensors modifying shader parameters. ......................................................................79   Figure 31.   Sensors piped through a threshold trigger. ................................................................80   Figure 32.   Spectral delays influenced by sensor streams ...........................................................81   Figure 33.   Various mappings ......................................................................................................83   Figure 34.   Patcher for audiovision playback ..............................................................................85   Figure 35.   Workflow pipeline for 3D sculpting in Mudbox .......................................................87   Figure 36.   Bass throb patch ........................................................................................................95   Figure 37.   2nd Stage of Section 1; scanslide ...............................................................................96   Figure 38.   Accumulation of various thematic materials .............................................................97   Figure 39.   Original Maya render taken from Section 2. .............................................................98   Figure 40.   Downsampling and upsampling pipeline for Section2..............................................99   Figure 41.   [jit.gl.slab] sensor scrubbing ...................................................................................100   Figure 42.   Snelson’s Needle Tower, Hirshhorn Museum, Washington D.C. ...........................102   xiii Figure 43.   The steel layered tower of influence .......................................................................103   Figure 44.   Section 3 sensor triggering ......................................................................................104   Figure 45.   Section 3, Stage 2: acceleration of FM events, towards cacophony .......................105   Figure 46.   Stage 3: The Rhythmic Groove ...............................................................................106   Figure 47.   In the score, the celesta starts the ostinato. ..............................................................106   Figure 48.   The ostinato modified, as it changes, a pitch at a time. ...........................................107   Figure 49.   The final pitch configuration ...................................................................................107   Figure 50.   The Greek Tableau concept is explored ..................................................................109   Figure 51.   Beginning of Section 5 ............................................................................................110   Figure 52.   The morphing models in Section 6..........................................................................111   1 1.0 Introduction: Anxiety of Influence: the project’s overall concept Anxiety of Influence is an interactive work for dance, triple projection, and 5.1-surround sound that utilizes a self-designed wireless sensor suit mounted on a dancer. The dancer’s movements produce real-time graphics and sound via the suit. The main concept of the work concerns the transient and unstable relationship of artists with their influences. The piece attempts to perceive through a critical eye the way influence filters into and modifies personal style and aesthetic gesture. Through interdisciplinary practice, the artist researches, develops, and refines personal approaches to provide flow in the creative process. This process of refining is also directly correlated to the perception of the self in relation to the surrounding world. And by intentionally observing this relationship, it could be possible to facilitate unexpected turns, instigating interesting creative results. It is easy to become lost in the milieu of great artistic achievements from the past dozen or so centuries. How can one go forward as a unique artist under such all-encompassing influences? And how do artists define who they are, in this century, through their art? Anxiety of Influence is about searching for an individual essence, where the human body determines the act of artistic composition, as it is impregnated with technology. Anxiety of Influence is simultaneously an internal journey and one mediated by space and screen in performance. This internal journey reveals some interesting repercussions, in light of the artist and the work he/she attempts to create. The concept of a successful realization becomes a new tool for the mediated body. The concept of a failed realization prompts further questions for the artist. What does failure and hiatus mean to the contemporary artist in this mediated state? In a way, failure is a road, or pathway: a transformative process of realizing an individual, unique essence. 2 Failure in the piece, failure through resistance in the physical body, failure in the work’s communication of a presupposed distilled message - these components could be construed as forever becoming, forever reaching for a perfect, yet undefined realization. The idea of failure is also a mechanism for feedback - a chain of perpetual influence, culminating in unfocused indecisiveness, where each improvisatory loop grows upon its own development. It is in this context of looping that the interest lies. Perhaps this reaching or striving for perfection is the essence we seek, rather than the successful distillation of something ‘perfect’? But concurrently, this reaching or striving taken to the point of breaking or failing is the only way to truly understand its capabilities or nature. This paradox is the key to a true realization, where the extension of this essence is always striving and never reaching. It is this reaching that we seem attracted to, when all is said. Hans-Georg Gadamer, in his The Play of Art, states that “Human production encounters an enormous variety of ways of trying things out, rejecting them, succeeding, or failing. “Art” begins exactly there, where we are able to do otherwise (Gadamer, 1973, p. 2). He then goes on to introduce the notion that the decisive task of creation is not the emergence of a product, but the realization that the product has a special nature of its own. “It intends something but does not do what it intends. It is not an item of equipment determined by its utility, as all such items or products of human work are. Certainly it is a product, that is, something produced by a human activity that now stands there available to use. And yet the work refuses to be used in any way.” (Gadamer, 1973, p. 3). These ideas of failure as insight were important factors to Anxiety, and will be discussed in detail throughout. 3 With the advent of body-mounted technology, a new condition of the body arises, epitomized in the dichotomy of disembodiment and embodiment. Disembodiment, or the body as abstract object, shows the mind constituting the essence of the individual, and the body a separate tool to be learned and mastered. Thus the mind and body are separated from each other, as in the notion of Cartesian Dualism1. Descartes explains that our perception and inner experience of the world is influenced by the activities of the mind, which can only be distracted and deceived by bodily experiences, as determined by the filtering and translation of misleading senses. It was an interest of mine, in Anxiety, to use the body mediated through technology as an attempt to merge this mind/body ambivalence. Perhaps the mediated body in the act of composition can resolve this: the body as a paintbrush for abstract thought channeled into the physical world via digital media. In doing so, the misguiding senses are replaced by the body- mounted technology itself, thus facilitating a mechanism for a pure connection of mind and outer physical experience. On the other hand, the idea of embodiment – or embodied cognition – proposes a more intimate connection between the mind and body, and ultimately a more fruitful way of connecting the kinetic with media. What are the repercussions of such a re-appropriation of the body, facilitating this act of creative experience? There is a strong chance that the technology will pass on its own misleading interpretation, which, in a way, replaces the original natural senses as they deceive the mind. Stelarc raises an opposing view in terms of the mind/body corporeal existence. He proposes that it is no longer meaningful to see the body as a site for the psyche or the social, but rather as a structure to be monitored and modified - the body not as a subject but as an object; and not an 1 A useful account of Cartesian Dualism can be found in the book by Francisco J. Varela, Evan T. Thompson and Eleanor Rosch, The Embodied Mind: Cognitive Science and Human Experience (1992). 4 object of desire but an object for designing.2 His objective is to abandon psychic and social subjectivity, human desires, emotions and the sensory apparatus, to use the structure of the body with technology. To return to this idea of becoming is to return to the idea of engagement in transformation. With the infiltration of technologies, the transformation is not about meaning or form, but particularly about operation. Deleuze and Guattari propose in A Thousand Plateaux, a “rhizomatic” analysis, which describes a feedback loop focusing on the way the operation functions and what it becomes, rather than what it is or what it has. Merleau-Ponty also supports this notion; feedback in the experience of the lived body is exemplified by the mind/body conglomerate that is not separate from what it perceives. Feedback also implies a gradual overlaying of change in memory. Furthermore, the framework for this memory can be based on self-modification and self-criticism, as determined by external influencing properties. In light of these concepts, Anxiety undertakes a dual purpose. Firstly, the piece attempts to formalize the self, through this mind-body relationship, and its relationship to intermedia. Which in turn, will hopefully establish an intimacy with the stage area and projection screen. And secondly, it attempts to re-evaluate the juxtaposition of disparate media paradigms, as reconciled by the human body, to create a new way of thinking about a marriage between the moving digital image and digital sound in the context of time - as instigated by the translation of kinetic movement in real-time. In chapter one, I will discuss the general problem of finding a unique artistic voice and introduce some of the theorists proposing different solutions to this problem. In chapter two, I 2 Stelarc’s website: www.stelarc.va.com.au 5 discuss some key concepts that were influential and problematic in the development of this project. Individualism, artistic value today, duality and the shadows of influence are raised. In chapter three, these ideas are explored through a theoretical and historical perspective, within the context of identity. In chapter four, a study is undertaken, discussing the various historical experiments that impacted my project. Chapter five provides a brief history of connections between dance and technology, in order to provide a perspective for the relationship between the two. This leads to an account of the piece’s technical concerns in chapter six. Technologies facilitating the production of Anxiety of Influence are summarized in detail, including the hardware and software created, strategies for mapping and the digital influence on lighting and choreography. In chapter seven, I analyze in detail my dissertation project. Finally, in the conclusion, I show how this overall analysis, hardware/software innovations and the project itself advance important contributions to new media. 6 2.0 The Problem of Identity for the Contemporary Artist It is by now a cliché that every artist must come to terms with the idea that all the great creative work has already been done. The issue is more about how originality can be conceived in the face of the overwhelming influence of previous work. Another cliché is that a stronger artist refuses to repeat or imitate influential works, and the failed artist is one who lacks originality as a result of outright imitation of the works of his precursors. A contrasting view brings to light the thoughts of Gilles Deleuze in his Difference and Repetition. He uses the term clinamen in his description of multiplicities; pointing to the observation that ancient atomism is seen as a physical idea, where the atoms are object elements of thought (Deleuze, 1994, p. 184). Though atoms affected by clinamen engage each other in a relationship of reciprocal supposition, Deleuze rejects this version of multiplicity, both because the atoms are too independent, and because the multiplicity is “spatio-temporal” rather than internal3. Deleuze’s concept translates well to the context of the artist and artistic thought; to simultaneously be in reciprocal supposition to other artists and maintain independence through isolation and internal singularity. To summarize, these spatio-temporal thought processes incite the very germination of creative thought, in the act of artistic genesis. 3 Deleuze discusses the influence of Kant upon his findings: “It is Kant who best indicates the correlation between objects endowed with only an indefinite specification, and purely spatio-temporal or oppositional, non- conceptual determinations (the paradox of symmetrical objects). However, these determinations are precisely only the figures of repetition: space and time are themselves repetitive milieux; and real opposition is not a maximum of difference but a minimum of repetition - a repetition reduced to two, echoing and returning on itself; a repetition which has found the means to define itself.” (Deleuze, Repitition and Difference, p. 13) 7 It is useful to take a look at Artaud and his influence upon Deleuze. Deleuze takes from Artaud’s play4 To Have Done with the Judgment of God (1947) the phrase “a body without organs” (Artaud, p. 571) and repurposes it to refer to the "virtual" dimension of the body. He sees the mind/body relationship as a reservoir of potentialities forever becoming as a result of the friction between the inner and outer worlds of the self. This virtual dimension of the body was a key concept explored in the choreography for Anxiety. The overshadowing and all encompassing concept of Anxiety of Influence takes its departure point from the literary critic Harold Bloom. His text from which the piece derives its name, Anxiety of Influence, tackles these issues by proposing six methods for conquering influence. Bloom refers to them as The Six Revisionary Ratios. The first of these states that the artist’s only weapon against the greats is creative misprision: only the deliberate misreading of past works will allow for a clear imaginative space, which in turn, can facilitate a unique voice. In other words, the creation of the individual poetic self requires the poet to deny the influence of previous poets by intentionally misunderstanding and misinterpreting these sources. Secondly, Bloom proposes the idea of completion and antithesis, whereby a great work is studied and taken further, in a new direction: the idea is in essence antithetical because it implies that the precursor did not go far enough. The new is written as a ‘completion’ to the first. It is influenced by, yet moving away from the original. Thirdly, Bloom suggests the reduction of one’s own style, removing all traces of repetition with a past work but attempting the same objectives as with this source, so that in the end, the two are unique and individual. Bloom’s fourth idea looks to a third opinion, both removed from the past and from the protagonist; though this third party is also the 4 Antonin Artaud. "To Have Done with the Judgment of God" in Antonin Artaud Selected Writings. Susan Sontag (ed). Berkeley, CA: University of California Press, 1976, p. 571. 8 artist himself. This third self looks at the original as a reaction to an earlier work, and as another piece of artistic tradition. By viewing the similarities between the two, the original suddenly loses some of the uniqueness originally perceived and as a result, the new in progress gains a newfound individuality. Bloom’s fifth revisionary ratio is to separate and isolate oneself from all other people and influences. As the person becomes separated, so too does his work, minimizing the similarities to potential influences. For the last method, Bloom suggests direct acknowledgement of past influences within the artist’s own work. This method can then allow for a role reversal whereby the earlier now mirrors the new. I will return to Bloom’s six methods for conquering influence in the chapter of Anxiety’s analysis. The relationship of Bloom’s six concepts informed the piece on multiple levels, thus a detailed description pertaining to these should be included when the piece itself is discussed. 2.1 Back to failure Gadager’s way of trying things out through play deems success as overrated, where risk is a viable pathway for interesting creative discoveries. And risk is determined through the perception of the self. The relationship of risk to the artist’s regard of self highlights concepts of duality; the duality of self and perception of self creates a dialog in the guise of an ideological shadow, or doppleganger5. The theme of doppleganger has a rich and varied history. Starting from the metaphysical poets, such as John Donne, to Goethe, to many other artists such as Percy Shelley, in his 5 For an interesting comprehensive account of the doppleganger, Milica Živković’s The Double As The "Unseen" Of Culture: Toward A Definition Of Doppelganger (Živković, 2000) 9 dramatic poem Prometheus Unbound (1820), Mary Shelley in Frankenstein (1818), Edgar Allan Poe's short story William Wilson (1839), Robert Louis Stevenson’s Jekyll and Hyde (1886), Fyodor Dostoyevsky's The Double (1846), José Saramago's novel The Double (2001) and Kiyoshi Kurosawa's film, Doppelganger (2003) is to name a few. Literature, more than philosophy, religion, or the social sciences, presents a detailed account of mankind's chronic duality and incompleteness, as well as his attempts, which range from the noble to the ludicrous, to achieve integration (Živković, 2000, p. 122). This shadowy figure supplies an outer context for the artist. A context that keeps some kind of freshness and uniqueness through experimentation, where everything is uncertain. The concept of the doppleganger also bears negative overtones. Instability, or tension through unrest, is present, in the guise of an ever-present shadowy alto-ego: a disfigured, fragmented reflection. It is well known that Cézanne attributed much of his artistic output as a result of his ‘anxiety,’ and it was this anxiety that Picasso observed was Cézanne’s legacy to all artists (Barolsky, 1997). This anxiety is summarized by the artist’s well-known commitment to what he termed “realizing my sensations” (Maynard, 2005, p. 229). What is particularly interesting with Maynard’s idea is its use as a device for articulating the experience of self-division, acknowledged as being colliloquies within the self: the double has become an aspect of personal and interpersonal life, a manifestation of unconscious desire and how this desire constitutes a personal reality and human identity, which brings forth a dichotomy with the rise of an increasing ideological polarization of the existential continuum into irreconcilable opposites (Maynard, 2005, p. 123). 10 Cézanne also stated “I cannot attain the intensity that unfolds itself before my senses. I do not have the magnificent richness of coloring that animates nature.” Here lies the paradox, whereby the artist is hindered by his own perceived limitations, which ultimately facilitates the fruition of the final art piece. Feix Gondalez-Torres’ unmasks further this notion of failure. His Untitled (Perfect Lovers) (1987-90) shows an identical pair of battery-operated wall clocks, placed side by side, which inevitably will fail to keep the same time. The ‘perfection’ here lies in the failure of accuracy; anything else would be romantic fiction. (Le Feuvre, p. 16). Gondalez-Torres’ begs us to consider the relationship of time and art, and artistic degradation over time. Whether it is the physical kind, or the contamination of epoch-defined criticism, art has always been considered in relation to its own period of time. As society, style, fad and fashion change through the centuries, so too does the commonly acknowledged perception of artistic merit. Thus, when considered in the context of time flux, perhaps failure is really the only way to determine a perennial artistic freedom and appreciated resulting art-form. Since it never really reaches its goal, it never really satisfies the criteria to which it searches for. And thus, defies time, context and predefined notions of merit. There has been much inquiry into memory as a mechanism for failure, which in turn, instigates creative originality. The failure of memory provides poesy: a reduction and simultaneous distortion of a reality, or an interpretation of reality. Proust was instrumental in redefining memory in the eye of the artist, which culminated famously in his newly founded idea 11 the ‘stream of consciousness’6. Walter Benjamin reflects on Proust in his essay The Image of Proust 7. He brings forth the notion of perception becoming unraveled by the power of forgetting, which, in turn, is driven by an endless methodological dissatisfaction (Benjamin, 1985, p. 210). Here is where failure, like the ‘glitch’ of this century, becomes the catalyst for creative endeavor. The process of remembering falsely degrades truth, until it takes upon its own identity. And in turn, this identity, as it wrestles spasmodically with ideas, representation and realization, takes upon a new meaning; a paradigm for artistic form and structure itself. The glitch was an important technique used in Anxiety, as will be discussed in the analysis of the piece. As mentioned above, the stylistic trend of the glitch in today’s culture has close ties to concepts of failing. A glitch is assumed to be the unexpected result of a malfunction (Moradi, 2004, p. 9). Glitch is imperfection: the result of an error that produces something new, data bending as the result of misusing digital information/code, or simply a skipping CD. Sometimes the parts of something, an image for instance, are shifted as they are incorrectly translated. Perhaps a packet of data is incomplete when it is received. A low sample-rate of a digital signal causes soundwave aliasing, or an artifact as a result of a mistranslation in video compression. Or simply opening the source code for a soundfile or image file in a text editor and changing various characters or numbers. These have become the new techniques for an aesthetic that has given rise to a whole new genre of digital performance and the creation of many new genres of digital art in general. 6 Many writers were influenced by Proust’s concept. People like Virginia Woolf and James Joyce were key influences upon my own creative awareness of structure and content within a new piece, through the fragmented flow of memory. 7 Benjamin , Illuminations: Essays and Reflections, 1969, p. 201. 12 An early example of this brings Christian Marclay to the fore. His re-appropriation of turntables and LP records, in the late 1970s/early 80’s, layed out the groundwork for the growth of DJ’ing and turntablism. He was innovative at the time, as he searched to find ways in turning the turntable and gramophone record in to an instrument capable of producing live sound collages. By scratching, breaking, bending, warping and reconstructing records he painted the path for others to explore this new paradigm8. Knizak, in his Destroyed Music, also cuts vinyl records into sections and glued them back together to create composite discs that caused the phonograph needle to pop and bang over the joints (Stuart, 2003, p. 1), while Vade creates a whole series of Max patches that ‘glitch up’ incoming video data9. There are tutorials online describing methods for how to create a soundfile by opening the text code of an image in a sound editor10. The applications of glitchery are unending. To uncover another kind of glitch, this time psychological, we return to Chapter 1 and discussions relating to cognition filtering and the problems of sensory perception. How can digital sensing mirror this and how can it facilitate the removal of influence? Does the code itself have an influence? Is this a desirable repercussion of the digital paradigm, on the road to a unique creative voice? Or can it also become a means, in the guise of a glitch, perhaps by happenstance, towards sidestepping Bloom’s six steps all together? 8 Taken from an Interview by Jason Gross: http://www.furious.com/perfect/christianmarclay.html 9 http://abstrakt.vade.info/?p=48 10 http://blog.animalswithinanimals.com/, for example. 13 2.2 Data Cacophony and Glitch as a Contemporary Paradox of Influence In L.A. they are collapsing in the streets. Down at the stock exchange, brains are melting. More is happening than anyone can cope with. Information overload – the very contemporary scourge.11 In his essay The Long Dark Phone-In of the Soul, Fuller looks at the assumption that living today our humanity, through technological advancements, has infinite capacity for qualitative improvements in an accumulated intelligence. Taking this idea, Fuller suggests that we are doomed to fail under the sheer quantitative increase of information to be processed, as delicate sensibilities are exhausted by mental stimulation (Fuller, p. 121). Here lies the paralysis; the sheer size of information passing through our senses needs to be filtered and rejected or digested. Stress is the inevitable outcome of this kind of system. It is a condition that announces the brutal inscription of a system pushing the body’s endurance to new limits. Stress expands into the world, creating a new, post-natural ecology - an information-based ecology, aimed at the ultimate fusion of capital and nervous system. The techno-neuronal system produced by this stressed ecology provides contemporaneity with a new gestural lexicon for the human body – perhaps a behavioral template the equivalent of the conflict shown in a violinist and the simultaneous friction of muscle tension and horsehair on a bow. Friction takes all sorts of forms as a result from all sorts of means. As discussed earlier, friction can also be a way to proceed as an artist, burdened by the anxieties of the preconceived limitations previously discussed. How can this template be translated into an art? Will such a venture bear fruits? Is such a venture possible? And if so, what conclusions can be found regarding the essence of an individual voice, as previously discussed? 11 The Long Dark Phone –In of the Soul, p. 121. Behind The Blip - Essays On The Culture of Software (Fuller, 2003). 14 The relationship of stress and flow within our hyper-realized culture provides a framework for modern life, as it reflects an unending series of challenges for keeping up with the kaleidoscope of saturated stimuli constantly around us. The metaphor is also appropriate to the weight contemporary artists endure by not only the shear bulk of creative work saturating the senses, but also the saturation of data bombarding the creative psyche. Perhaps by searching for the kind of template mentioned above (regardless of whether it is found), it might be possible to distill a unique identity that the artist can use to escape the tyranny of Bloom’s concerns within his Anxiety of Influence. As an artist, then, the question becomes not how to bring various ideas, media, and methods of artistic production into coherent form but rather how the architecture of the idea, space and time informs these components.12 Instead, we are forced to consider how, in bringing these different elements together into a single piece, we transform them; how the separate elements of the work frame and shape the work itself. The focus thus centers on the play among the various media, domains, and technical aspects of an artwork instead of any stable linkages between them in flat topography, of separate domains and their linear connectors. In the final analysis, it is this play between the domains that cause them to dissolve, or degrade, into “one thing” – a fusion that is much more than the sum of its parts. 12 Richard Andrews, in his critique of James Turrell (Andrews, Richard. 1992. James Turrell : sensing space. Seattle: Henry Art Gallery / University of Washington Press.) discusses the aesthetical importance of the architectural space in which projected light, and ultimately image, is observed by the spectator (Andrews, 1992). 15 3.0 Theoretical and Historical Context of Identity 3.1 Self-Awareness, Identity and Artistic Voice In returning to Bloom and his interest in establishing a unique voice, I propose a hypothesis in relation to the self and any creative byproducts – a concept that was instrumental to the inception of Anxiety. An artist must be well educated in the recent and future developments in sociological studies, technologies, trends and stylistic developments in order to consciously attempt to remove oneself from sociological constraints and general cultural frameworks. Thus forming a utopian ideal of creating an art form of constant evolution removed (if not completely) from these components of contemporary life. Preliminary discussions concerning the nature of such fundamental topics as human perception, human discourse, essence and self-awareness are necessary before any subject of enquiry utilizing these perceptive realities can be made. In other words, it is necessary to de-construct the construction of the self in order to arrive at any coherent evidence and any deduced conclusions and hypotheses. Human perception contextualizes the world through our filtering senses. A possible method of bypassing this process is to analyze it; thus being conscious of these unconscious patterns. “We never cease living in the world of perception, but we go beyond it in critical thought – almost to the point of forgetting the contribution of perception to our idea of truth.” (Merleau-Ponty 1964, p3) The concept of perception allows hypothesis of personal truth to be deduced - a truth for one individual with his/her set of perceptory filters will deduce cognition with different conclusions if compared with another’s own unique set of cognition filters. The 16 topic of truth will be explored later, since again ideas of truth are fundamental to any cultural and/or artistic enquiry. Having defined perception, the perceived world is therefore an impermanent and abstract phenomenon based on the cognitive foundations of human rationality. The flow of this rationality is framed by methods of interaction; human discourse is the mechanism for the interaction of minds, delivery of ideas and thus, the need to categorize these ideas. The methods of communication that link two or more individuals becomes the genesis of interpersonal relationships. While most artists are accepting (consciously or unconsciously) of this categorical process, others react against it in protest. Reactionary in aim, these individuals defy the parameters discussed above in an attempt to provide a non-referential art. From this situation, one of two paths will result; one will perpetuate the artist into becoming a fad and thus fade into obscurity, and second, the artist will cause the establishment of a new categorization. The irony present in this second path is interesting. For purposes of clarity, I will label this consensus Reactionary. Reactionary artists, through a conscious decision to avoid this level of influence will produce art that is void of referential treatment. Does this really exist? Does the artist succeed in their goal, since one must be conscious of the influences in order to reject them? Once a general language for the creation of an art form is developed, there are always aspects of technique that are universal to stylistic developments. As more stylistic developments arise, the older become common practice. Still more developments create diffusion in style whereby a similar vocabulary produces a diversity, which ultimately creates more specialized categorizations. The contemporary idea of fusion (preconceived notion in hybridity of style) is 17 integral to artistic endeavor today - the relationship of one style to another becomes closer and closer as common threads become more and more prominent. Therefore an artist need only be extremely sympathetic and receptive to technical developments. The principle is flawed at the beginning anyway, since human interactions (audiences) are the guiding perceptors in the final analysis and therefore each individual will witness different contexts. A purely theoretical objective, the idea has its advantages, even if its outcome is not perfect. But my aim is not so much as to exemplify a ‘reactionary art form’ but rather to elucidate this curious relationship of categorization and artistic production. This is extremely apparent within the field digital media. Dixon in his book Digital Performance discusses Walter Benjamin’s central argument concerning the ubiquitous nature of digital art, that of duplication (Dixon, p. 116). Continual sampling and re-coding of digital media works is one of the easiest and most common characteristics of new media art. Benjamin’s claim is that new media art, like other art, is trying to be authentic, with a unique existence. He proposes that in the reproduction of new media works, the presence of the original is a pre- requisite to the concept of authenticity (Dixon, p. 116) - in mechanical reproduction a depreciation of value results. This lack of authenticity as a result of mechanical dilution proves there is no distinction between originals, simulations and simulacra (Dixon, p. 117). In a way, this lack of distinction in digital art is the antithesis of the Bloom’s idea of a unique voice, by its inherent reliance on its quotation of original sources. Or is it? Bloom, as we will see, requires these original sources in order to discern his six rules in his Anxiety of Influence. I will return to this concept when Bloom’s specifics are discussed at length below. 18 3.2 Finding our own Individual Essence Through our physical existence the journey to self-realization is the resolution of inner conflicts in conjunction with the outer influences of our adopted social milieu in an attempt to refine all factors contributing to the psyche. Or alternatively, it is a journey through the realm of possibilities to produce the ‘essential self’ with the aim of creating a coherent self-reliable entity. This process is invaluable to the creative artist; the culminated state of essence logically produces a refined state of artistic style and technique. Chiari Joseph in his book Symbolisme states ‘The realization and the revelation of essence through existence is a process which involves the reduction of oneness of conflicts and opposites, which have to pass from the realm of possibilities to the reality of existence.’ (Joseph, 1970, p.1) Joseph proposes that essence informs existence and that the spirit provides the link between. Therefore essence is the epitome of this informed state of self-realization. Change in a public whole is linked to this idea of essence. A society, by its very nature collective, is a culmination of essences and this in turn can redefine the sociological whole at any stage. Symbiotically, an individual’s self-identity is informed out of a hierarchical set of correlated principles formulated by this mass - the average, if you like, of a collective and progressive theorization. This mass, of course, finds its genesis form through the institution; the physical embodiment of a post-modern culture and a symbol in itself of predetermined guidelines and regulations for human relations. With the influence of the institution, the essences of societies are formed. Thus, the historical progression of social relations is evident; from the ordering of the masses, to the formation of set institutions through to the developments of social, behavioral and economical frameworks. Self-awareness within the context of social customs, 19 cultural backgrounds and institutional operations are all major contributing factors to the creation of any work of art in any artistic discipline. These influential factors all combine in a general milieu of psyche to define the very nature of the artist and his/her artistic by-products. Thus the individual occupies a constant state of transition- the Bhuddist principle ‘of forever becoming’ (). Furthermore, Merleau-Ponty summarizes: ‘Evidence is never apodictic, nor is thought timeless, though there is some progress in objectification and thought is always valid for more than an instant…. The certainty of ideas is not the foundation of the certainty of perception but is, rather, based on it…. In this sense, all consciousness is perpetual, even the consciousness of ourselves.’’ (Merleau-Ponty 1964, p13). 3.3 Artistic Value Artistic value and meaning is universally accepted as being comprised of components, or principles, that define and trigger affect in the observer. Discussions of affect or emotional reaction are necessary in order to ascertain how artistic stimuli causes psychological and emotional effects in the listener. In Dewey’s study, The Theory of Emotion, the proposition is raised that the emotion (or affect) is aroused when a tendency to respond is arrested or inhibited. MacCurdy also supports this theory; “When instinctive reactions are stimulated that do not gain expression, either in conduct, emotional expression or fantasy, that effect is most intense.” (MacCurdy 1925, p475) In other words the prevention of expression causes a greater reaction or intensity of effect. The energy of the organism must be blocked by repression before reactive feelings are excited. Meyer summarizes this by stating that this emotional experience is non- referential. “Affect or emotion-felt is roused when an expectation – a tendency to respond – 20 activated by musical stimulus is temporarily inhibited or permanently blocked” (Meyer 1956, p31). Anxiety plays with the idea of inhibiting expectation. The relationship of the dancer to the screen is the essence of this distancing, as the bridge or window between the two is translucent, or opaque, on its way towards transparency. Inhibition is also present in the choreography, as will be discussed later: the inhibitions of the projected motifs and the mirroring of them in the dance also shows a reaching for understanding, of connection, without actually arriving. These thoughts will be explored further in the analysis of Anxiety of Influence. Within the context of art, looking at stylistic trends and well-established motifs elucidates this stimulation of affect. If common structures have been ascertained to a culture, style or a work and are presented without delay, or no tendency inhibited, the listener would not respond in an effective way. Other parameters that condition this perception also influence the response; such as how people respond automatically to the same music used in film scores, while the images alter the perception of the sound. There is definitely an emotional reaction, even though it is completely predictable and within a commonly accepted style. If the moment fails to follow its customary course, or if it involves obscurity or ambiguity, then the observer’s tendency will be inhibited or upset, thus creating affect. Hence deviations from the norm of style can be regarded as emotional or affective stimuli. Thus by attempting to break down cultural artistic reference points is to adopt a possible method of conveying meaning. In the final analysis, evaluation takes the final form of a deconstruction of one’s own personality placed within its context of determined influence, as defined by the artistic work. In other words, being conscious of the self as a product of circumambient guidelines that influence 21 in a multitude of ways. To go one step further, one must attempt to reach outside of the self (or essence of self) and trace all contributing factors to this fundamental makeup. Since it is impossible to remove oneself from cultural influences it is a requirement to synthesize all points mentioned above in an effort to reduce the influences upon artistic voice: on a journey to find the holy grail- a pure art form (whether it exists or not is irrelevant) that surpasses correlation and surpasses categorization. Possibly a totally unrealizable endeavor, but one that can influence the resulting product of artistic creation, both in its conception and it is final forms. The attempt is to produce things devoid of anything but its own context and its own journey. 3.4 Duality as Aesthetic Antonin Artaud, in his The Theatre and Its Double (Artaud, 1958) talks of the dichotomy of the inside and outside worlds. He considered dreams, thoughts and delusions as no less real than the “outside” world and are equally important to art; the relationship of the inner, particularly the stress of inner torment and labor, to the creator’s consciousness is what gives art meaning. The relationship between the actor and the enveloping space, the relationship between the actor and outward gesture with inner thought; these give the theater a dual timeline through which both mutually influence. Artaud’s interest in the relationship between the self as actor/performer and the theater space needs further clarification, as it relates to Anxiety. The dancer on stage and his intimate relationship to the projection surface but the defined performance area was vitally important to the inner workings of the piece, as we shall see. It was an objective in Anxiety that boundaries, or walls, between the performance space and performer be broken down and made transparent by gesture. Gesture that is instigating, and simultaneously, being influenced. The relationship of Artaud’s inner world of the performer and the outer world 22 of the space also feeds and catalyzes. The nuance of dance and circumambient media within the performance space plays with this idea of inner and outer worlds. Betsy Biggs writes “Space invites phenomenological listening, attention, wakefulness; it is ripe with potential, and activates freedom.” (Biggs, p. 30, 2009). This freedom is the type of emancipation that produces something specific but at the same time requires commitment, a willingness to at least temporarily define a space as one’s own. Once this relationship has been established, the previous concepts of separateness and togetherness arise. And this can facilitate the discovery of an identity of self, as defined by this space. Which, in turn, can potentially cause a merging of the two, as space and individual become one and the same. Below are two still images taken from the first performance of Anxiety of Influence. It was my will to establish a fluid window between the dancer and his stage space. All choreographic, sonic and graphical gestures grow out and through this conceptual window. Figure 1. Space, body and screen. 23 The image above shows a moment in the piece when the segmentation of the screen strongly implies a window metaphor: the dancer is concerned with passing through this dissection, where the implication of another space, or universe awaits. Figure 2. Here the window-screen is broken. In this image, the window is broken, as choreographic gesture causes the graphics to shatter. The flow of time is a vehicle facilitating the gradual disintegration, or dissolution, of this window; until the end, as the dancer becomes part of the space, as shown here: 24 Figure 3. The last frozen image of the piece This last image, held for ten seconds and coinciding with the sound reverberation fading, recalls a memory of the photographic – the chameleon-like dancer and projection surface become one – almost like the role of the Greek chorus, in their theatric tableau13. Troika Ranch’s The chemical wedding of Christian Rosenkreutz (2001) exemplifies another kind of unification between dancer, media and space. Taking inspiration from the 17th century Rosenkreutz’s transcendent transformation through the ‘chemical wedding’ of alchemy, Troika Ranch describes their objective of using sensory technologies to create a similar alchemy through the use of a custom-built motion sensing system MidiDancer 14. It was the purpose of 13 The Greek Tableau was also a key turning point in the choreography, which will be discussed later. 14 The system was influential on my own sensor suit, in its physical computing technologies, as well as its aesthetic duality and fusion of dancer and media – it was while studying this work and its technologies that I decided flex sensors would not be appropriate to the development of my project (more on this later). 25 using the system to unify the digital and corporeal elements of the work into a unified expressive whole. The system was comprised of flex sensors worn on the body of a dancer which measured the flexing of up to eight joints on the dancer’s body and then wirelessly transmitted the position of each of those joints to a computer off stage. Once interpreted by software running on the computer, the information was used to control a variety of media including digital video or audio files, theatrical lighting, robotic set pieces or any number of other devices (Farley, p. 6). The work combined pre-recorded and live sound and graphics, with the unusual inclusion of algorithmically edited video imagery and digitally controlled lighting used as the means for delineating the structural content of the piece. The interactivity was dispersed between multiple dancers, where control over sound sample playback, video scrubbing and projected text was manipulated by bending and straightening the dancer’s arms. A wireless camera mounted on the head of one dancer projected live video whenever the dancer bent her neck. In The Visible and the Invisible, Merleau-Ponty suggests a further concept of an alchemical fusion. His concept of “Chiasm” suggests that the point at which a subject and object cross is the bridge between the phenomenological subjective experience and objective existence (Merleau-Ponty. P. 106). In other words, he asks us to consider our bodies as both part of the tangible world as well as part of the perceptual realm of experience. To exemplify this, he looks at the color red: “a certain red is...less a color than a difference between things and colors, a momentary crystallization of colored being or of visibility” (Merleau-Ponty 132-133) As one perceives this red, its essence shifts, depending on how much light is present in the space around the color, the acuteness of the individual’s physical sight and all other colors surrounding. More specifically, this red, as defined by these circumambient factors, is never the 26 same experience between different individuals. “It means the color red is contextual and relatively relative as opposed to permanent and universal (Durski, 2004, p. 2)15“. The final result is a breakdown of definitive meaning in the historic duality between subject and object from the object’s perspective. Thus, red, is not an object containing inherent values and characteristics, but rather, red is defined by our participation with “it”. He then continues by breaking down the duality between physical sensation and the subject/object dichotomy, suggesting that through the interaction between the two halves, the two become inextricably bound to one another: “Since the same body sees and touches, visible and tangible belong to the same world. It is a marvel too little noticed that every movement of my eyes – even more, every displacement of my body – has its place in the same visible universe that I itemize and explore with them, as, conversely, every vision takes place somewhere in the tactile space. There is double and crossed situating of the visible in the tangible and of the tangible in the visible; the two maps are complete, and yet they do not merge into one. The two parts are total parts and yet are not superposable” (Merleau-Ponty, p. 134). Which brings to the fore another contributing factor for Anxiety, the duality in the double. To validate its importance, further discussions are required to unmask the underlying thread influencing not only the structure of the piece but also the relationship between dancer, dance, stage, screen and interactivity. 15 Phenomenology: Cook Ding’s Ming and Merleau-Ponty’s Chiasm, Matt Durski 27 3.5 The Doppelganger Merleau-Ponty asks us to consider our bodies as both part of the tangible world as well as the world through which we perceive the tangible things that lay beyond our bodies. He also discusses the concept of a ‘double’ perception. “As soon as I see, it is necessary that the vision...be doubled with a complementary vision or with another vision: myself seen from without, such as another would see me, installed in the midst of the visible...” (Merleau-Ponty, p. 134). In other words, the self as seen by the self seeing. The paradox is a parallel to Bloom, in the presence of an influential feedback loop: as the artist observes influence and acts upon it, she in turn influences. I have been fascinated with Schubert’s Die Winterreise for over fifteen years now. When I was first introduced to the piece I remember still now being compelled by the conceit proposed; tracing the wanderings and meanderings of the soul-searching narrator, always looking over his shoulder at his unseen shadowy companion. The song cycle is a long work. It is not an ephemeral glimpse at the narrator’s existential quest but one that implies a never-ending struggle, a path inward without ever finding resolve, or satisfied discovery. Susan Youens, in her book Retracing a Winter’s Journey (Youens, 1991), discusses Müller’s rich and complex text, which was used as the basis for Schubert’s cycle: “The inward experiences he portrays in this cycle, however, are not simple. His wanderer, after his beloved forsakes him, embarks on a journey into the depths of his being and there conducts a lengthy process of self-questioning. His attempts to understand his alienation from humanity are periodically interrupted by surges of emotional current. And by increasingly urgent longings for a death that is always denied him” (Youens, 1991, p. xii preface). The episodic aesthetic of the work has many resonances in much of my 28 own creative work and there is a subtle yet conscious allusion to the Schubert/Müller narrator through the dancer/protagonist in Anxiety. Many other artists have taken up the doppelganger metaphor, perhaps due to a fascination with this concept of self-identity and self-reflection. Within Schoenberg’s monodrama Erwartung, based on a libretto by Marie Pappenheim, the doppelganger becomes an effective metaphor for conflicting facets of the protagonist’s disrupted psyche, as she spirals into an internal abyss through the athematic and episodic nature of Schoenberg’s score. Berlioz’s protagonist artist and his doppelganger, in his programmatic symphony Symphonie Fantastique also takes up the doppelganger motif. Here Berlioz’s doppelganger takes the form of an idée fixe, which causes the artist to see the ever-present shadow of his unrequited love. There are many similarities between these examples of the shadow, though one specifically resonates for myself and Anxiety; the protagonist is more aware of the shadow than of anything else, including herself and including any immediate surroundings. 3.6 Liveness Probably the most compelling thing about an interactive digital performance is the structure of the piece as instigated by the performer in the specific moment. Liveness is the evanescence and temporality experienced by performer and audience alike, and the non-reproducibility of that experience. But when we look at the myriad of digital performance examples, we find a wide range in the amount of spontaneous performance and levels of “fixed” media. Philip Auslander argues that the very notion of live performance is an historical function of mediation. Theatre, dance, music and other performing arts could be perceived as ‘live’, even though components are 29 set. Not everything can be live, or chaos would abound! Take a lighting rig, for instance. Without lighting preconfigured, many of the real live elements would inevitably be lost. The amount of media that is fixed and the amount that is spontaneous raises some interesting aesthetic problems, regarding the relationship of performer and the media. “Although much of Auslander’s discourse emphasizes the reciprocity and increasingly undifferentiated ontologies of live and mediatized forms, within digital performances that attempt visual synthese of live and digital bodies, he clearly discerns that the dominant aesthetic for is the digital, into which the live is incorporated. Thus, rather than a conversation among distinct media, its production presents the assimilation of varied materials to the cultural dominant.” (Dixon, p. 123-124) Patrice Pavis, in his book Theater at the Crossroads of Culture, proposes the flux of “specificity and interference” as a dialog between theater and media that is tightly contained in the “socio-economic-technological domination” which characterizes a work of art in the present era of mechanical reproduction (Pavis, p. 99). Pavis sees the ubiquitous nature of mechanical reproduction as a sociological fabric that influences theater by contaminating it (Pavis, p. 134). This contamination paints the very nature of the relationship between live performable bodies and live performing media. Therefore the idea of the copy, as discussed by Benjamin, is a common thread through all digital performance, regardless of content; again, live and mediated forms produce the feedback loop relationship whereby each developed and derived their authority in relation to the other: “But the perceptual and epistemological shift from an oscillating back-and-forth system for referencing and validation between two poles into what amounts to a spiraling of the forms 30 into a free fall of indistinguishability is radical and problematic. In this conception, digital performance becomes not a black light theater of charm and illusion, but a black hole theater where gravity is irrelevant as both corporeal flesh and data-bodies are sucked into the same synthesizing vortex” (Dixon, p. 124). These problems concerning the contamination and eventual morphological synthesis lead us to the use of interactive digital media, as such is used today. No longer does the mechanical reproduction of media have to stunt. By having the different media changing as a result of the performing body’s nuance is to break down this pretense of liveness in performance. Mark Coniglio in his paper The Importance of Being Interactive states: “But, when the performers attempt to nuance a gesture or phrase in response to the aforementioned relationships, an unrelenting and unaware companion—the digitally recorded music with which they perform—thwarts them.” (Coniglio, p. 3). A performance where the live element is juxtaposed with fixed media highlights these concerns. And with the recent speed and power of computers nowadays, it is possible to sidestep this type of stunting completely. Or is it? These influences regarding mechanical duplication, as discussed earlier, are still present. So how can a work utilizing digital media break free from this despotism of fixed/live digital reproductions? One way, perhaps, is to give the work another overshadowing objective. By giving a new priority to choreographic gesture it may be possible to breakdown the relationships discussed above. In order to facilitate this new priority, the range of choreographic movement needs to be defined, from pure filigree dance and through to media-instigating gesture. Curtis Bahn, Tomie Hahn and Dan Trueman, in their paper Physicality and Feedback: A Focus on the Body in the Performance of Electronic Music, describes a division of musical 31 gesture, in digital musical performance, into three levels: effective gesture—that necessary to mechanically produce sound; accompanist gesture—movements associated with effective gesture engaging the whole body but not directly related to the act of sound production; and, figurative gesture—wholly symbolic gestures of the performer (Bahn, Hahn, Trueman, p. 2). This first level of physical gesture can be seen more as the controlling parameter that incites the compositional structure and is used as a means of travelling through the piece. This first level is dependent on the physical guise of the technology used for the live interactivity. In the case of Anxiety, this is the physical hardware constituting the wireless suit. Joel Ryan summarizes the need for custom-designed hardware to facilitate a unique and distinct voice in digital real-time art: “Most computer instruments in use are those provided by the commercial music industry. Their inadequacy has been obvious from the start -emphasizing rather than narrowing the separation of the musician from the sound. Too often controllers are selected to minimize the physical, selected because they are effortless. Effortlessness in fact is one of the cardinal virtues in the mythology of the computer. It is the spell of 'something for nothing' which brightly colors most people's computer expectations. Despite all experience to the contrary we continue to think of the computer as essentially a labor saving device. Though the principle of effortlessness may guide good word processor design, it may have no comparable utility in the design of a musical instrument. In designing a new instrument it might be just as interesting to make control as difficult as possible. Physical effort is a characteristic of the playing of all musical instruments. Though traditional instruments have been greatly refined over the centuries the main motivation has been to increase ranges, accuracy and subtlety of sound and not to minimize the physical. 32 Effort is so closely related to expression in the playing of traditional instruments. It is the element of energy and desire, of attraction and repulsion in the movement of music. But effort is just as important in the formal construction of music as for its expression: effort maps complex territories onto the simple grid of pitch and harmony. And it is upon such territories that much of modern musical invention is founded.” (Ryan 1991, p. 3). Custom-built hardware allows the intermedia artist to define for herself the range, accuracy, and subtlety of her instrument. It is also the moment at which it is possible to define resistance and inhibition that incites true affect, as discussed previously. If the performer is aware of the threshold between resistance and impedance in the instrument, it is possible to ‘play’ with it, alternating between failure and hierarchical resolution. How to construct the “right” instrument for a particular work is impossible to specify in the abstract. Nevertheless, there are a few considerations that apply to a wide range of applications, including off-the-shelf systems, in the effort to accurately translate human affect, through gesture, into sound and image art. 3.7 Resistance I now return to Meyer’s concept of stimulus being temporarily inhibited or permanently blocked. As discussed previously, by disrupting established patterns of behavior it is possible to stimulate affect. In the context of performance, gesture and resistance are responsible for molding and expressing the sensory stimulus of live input. How do the flow of these gestures make up a system for physicality and feedback, between the haptic and the mechanistic? How does gesture and resistance in the instrument allow conditioning and responsiveness in the artistic dialogue 33 between human and computer? Can refining a system for feedback facilitate this conditioning and responsiveness? It is widely acknowledged that traditional acoustic musical sound is a direct result of the interaction between an instrument and the performance gesture applied to it (Rovan 2006, p.45). If this model of expressivity is to be duplicated simultaneously with the construction of the digital instrument itself, physical gesture, in all its complexity, must also be modeled. Thus, a mode between digital mechanistic stimuli and human kinetic nuance must be ascertained before a meaningful expressive interactivity can arise. “Paradoxically, the instrument cooperates by resisting. In its body and its parts are weight, tension, inertia, and only by leaning into these forces of stasis does the musician feel the instrument press back against her, demanding her continued response to keep it in motion“ (Evens 2005, p. 159). The friction Evens refers to, between performer and the instrument, makes for a versatile machine for creating problems. And these problems are not dishonest ones defined by ambiguity, chance, or complexity of data, but more the type that can catalyze artistic form determined at the stage of composition. The artist must resist this friction, producing the aforementioned feedback loop. As Evans suggests, the acquisition of technique is to refine the efficiency of this collision, which culminates in a dynamic correspondence as the resistance instigates craftsmanship molded to an aesthetic and creative result. Without this resistance to the artist’s input, the instrument returns to its programmed static equilibrium, thus severing the correspondence. Of course, not all resistance is productive. It is obvious to an audience when a performed technology has reached 34 uncontrolled anarchy, for whatever reason it has; thus destroying the world of the piece and, perhaps, allowing the minds of the audience to drift to what’s for dinner… This is where programming skills required in mapping and filtering comes into play, in order to produce an adequate amount of resistance. By converting undetermined raw energy on behalf of the instrument into determined artistic expression, this process of conversion between human and computer allows for a refined and specialized artistic impartation, as the digital- instrument symbiotically becomes an extension of the artistic self. 35 4.0 Intermediality: A Current Historical Context of Dance and Technology There has been, in the last two centuries, an enormous interest in establishing some sort of movement-based interactive environment, by the adaptation of movement through dance, to manipulate sound and graphics. It has been suggested that this has arisen out of the need for further expressivity through closer connections between dance and these digital outputs. Sound artists have always had a keen interest in dance. Compared to the gestures of instrumentalists, dancers have a much higher awareness and sensitivity to the aspects of spatial and physical motion. As technology in performance practice has become cheaper, versatile and more importantly, reliable, there has been an interest for sound artists to branch away from the role of composer. As these technologies became easy tools to use for expressivity, live performance gained the capacity to incite and enhance the creative process itself. Developments in equipment used for sensing dance have closely followed this divergent pathway: from the composer composing, new possibilities are continually being uncovered for spontaneous creation. This spontaneous process promises to a work a unique freshness through limitless real-time interpretation. The role of composer has become fragmented even further, as interactive sound becomes but only one medium available for the creation of real-time works. With the advent of opengl real-time graphics processing, the relationship between sound art and dance is once again amplified beyond preconceived notions. This is the essence of a remediation of traditional media: now it is redundant to define the essential components of each separate medium. A “remediating” is required to describe the symbiotic relationship of separate components, as they are simultaneously translating, refashioning, and reforming other media, both on the level of content 36 and of form (Bolter, Grusin, 1999, p. 19). Sound can mold graphics. And graphics can mold sound. And dance can mold both. With the development of image/sound-producing applications such as the Jitter extension to Max 6, Isadora and Quartz Composer, to name a few, it is possible to evoke all sorts of data conversions. Probably the biggest influence on dance works utilizing technology has been the accessibility of software: many people, who are not programmers, having the opportunity to experiment. The result is that even casual users with the interest can make a live performance as easily as they might use theatrical lighting. This has advantages and disadvantages, as will be discussed later. Much has been written about the early beginnings of the symbiotic relationship of dance and technology, culminating in the remarkable artistic achievements of dance groups such as Troika Ranch, Trisha Brown/Robert Rauschenberg, Merce Cunningham/Cage, William Forsythe, The Wooster Group, Dumb Type, David Rokeby, Winkler and many others. Some artists use existing technologies, while others create their own software/hardware paradigms. In the realization of live dance and live media, most use configurations from one of two areas in technological innovation; motion tracking and sensor-mounted paradigms. Both attempt to capture kinetic gesture in its purist form and rework it into tangible information useful for generating computer-generated structures. 4.1 Motion Tracking Motion-capture these days has become synonymous with real-time digital/human performance and has also given rise to new conceptual frameworks in which to understand how the human body relates to the space around us. Taking into account the countless experiments in 37 camera pixel and color re-mapping, to infrared detection, to the refinement of computer intelligent ‘blob’ definitions and marker systems, there is not the space in this paper to cover the tumultuous magnitude of fascinating examples that explore tracking a dancer in a new media work. It is my intension, rather, to lightly touch upon the subject, highlighting key works that are exemplary in their artistic accomplishments, on the road to hopefully shedding some light on the problems of the paradigm for data acquisition. Marc Downie, in his dissertation Choreographing the Extended Agent: performance graphics for dance theater, working at the MIT Media Lab and also the auspicious Open Ended Group, considers the underlying complexities of using camera marker technology. His research follows the integration of digital art with the unmistakable trace of human presence. His work attempts to correlate real-time renderings of kinetic movement into the creation of signature “hand-drawn spaces”16, where the objective is to conjure up a three-dimensional world in the manner of gesture drawing, as mediated by the camera’s lens. 16 taken from Open Ended Group’s website (http://openendedgroup.com/index.php/about/) accessed November 2011. 38 Figure 4. Downie’s example of data flow (p. 258) The aim of this kind of piece seeks to establish a clear and perhaps un-engaging relationship between the camera and dancer. The kinematic skeleton matching, or doubling of the performer in the screen shown above, has the potential to fall into the traditional pit of inauthentic mirroring through reproduction and duplication. By simply presenting the data transcription, there appears to be no inner workings, or intricate play between dancer and technology. In other words, there does not seem to be some sort of engagement in the sense of remediation, on the road to an intermediation, as discussed earlier. 16 [R]evolutions (2006), created by Mark Coniglio and choreographer Dawn Stoppiello, uses motion-tracking systems to interactively generate three-dimensional graphics that respond to movements of the dancers, and both rely on a unified methodology for using those tracked 39 movements to manipulate the media. The piece occupies a varied palette of visual imagery that is present in several sections, but then is absent in others. In a few moments throughout the piece, the imagery is not interactively controlled at all. Coniglio, when discussing this work in relation to past works exploring fixed media, says “I am drawn to notice that these works are, in essence, about the materials themselves.”17 In this, Coniglio raises a pertinent objective for contemporary dance and technology works. Technical advancements and computer processing capabilities directly influence this objective. Coniglio’s work always follows an external concept or narrative, in to which technology fits and is a part of. Technology itself is never comprehensively explored and it, like all other elements, is always content-driven. Coniglio continues: “Using new technology to further expression has always been the realm of the artist, but how often have these explorations that focused on the technology itself remained important in our canon? While the early films of the Brothers Lumère are important, they do not have the resonance of Citizen Kane (1941); the same holds true for the early pianoforte works when compared to the later compositions of Chopin. (Coniglio, 2005, p. 3)” Another intriguing work using motion tracking is Lucidity.18 Choreographed by Jodi James, with interactive music composed by Todd Ingalls, and interactive animation created by Loren Olson, the piece premiered at ASU's Galvin Playhouse on 10 April 2005. James discusses 17 Materials vs. Content in Digitally Mediated Performance, Coniglio, 2005 18 http://motione.blogspot.com/ 40 challenges and discoveries in creating Lucidity and the problems encountered with the theater space the piece was performed in. Figure 5. Section three of Lucidity Since the piece was to be performed in a typical theater space, the quality of the capture had to compete with several other considerations, such as staged lighting and the lighting of the screen projection, the costumes, for example. This determined the types of motion capture cameras to use, the cameras’ placement, the marker sets used for the dancers, and what type of data to be extracted. “The placement of cameras for motion capture in a theater space presented one of the most challenging problems in the staging of these works. The first concern was the question of camera location so that they neither disrupted the aesthetic for the audience nor impacted the projection on the downstage scrim. After experimenting with several configurations, we determined that hanging the cameras from a 20x30 foot grid suspended above the stage was the best solution. The ideal height for this grid for data collection was between 10 and 13 feet, however, this would have cut the vertical space in half, blocked the theatrical lighting, and 41 bisected the front projection. The grid was raised to 22 feet, which removed its visual impact but placed the cameras at their outer effective distance boundary. Further complicating camera placement and resolution was that the cameras with the highest range emitted a distinctive red glow towards the audience. For aesthetic purposes, we decided to use infrared emitters on the cameras facing the audience since they give off no visible light. These camera emitters lower the effective distance of the camera by 1/3, thereby affecting the robustness of the captured data.” (James, 2005). Having too many markers for the camera to track, as in the case of Lucidity, where multiple dancers need to be tracked, also diminishes the camera’s ability to accurately capture and label data in real time. Also, these markers have a tendency to fall off the performer during performance, which according to James, happened frequently. These problems render the method unsound, if its influence upon the work’s structure is paramount. 4.2 Mounted Sensor-based Technologies Motion capture is most useful in highly controlled environments where tempered lighting is unchanged. When a work is to be performed in a typical performance space, there are too many uncontrollable parameters to facilitate the acquisition of clean data. This was the main reason I chose not to use motion capture in the making of Anxiety and was the main reason for developing the sensor suit. There are many commercial sensor interfaces available that are designed for artists. These devices allow users to connect various sensors without any, or very little, programming. These devices all have the same problems; often they are not wireless and require USB cabling, they are limited in the number of analog inputs they can support (usually 6 and usually at lower 42 resolutions, such as 10bit ADC’s), they are usually large and cumbersome in size, and after the sensors are connected, they impose the need for bulky and body-restricting wiring. In most cases, the artist realizes that these off-the-shelf models fall short of the artistic intensions of the work of art. The alternative is to create a new interface from scratch that suits the needs of the particular artistic context. There has been much work carried out in the last five years concerning wireless self-designed sensor systems that can be mounted on a body. Jan Schacher, at the Zurich University of the Arts Institute for Computer Music and Sound Technology has made many successful contributions to dance technology and sensor suits. In 2009, he created a pair of wireless sensor bracelets that were built from readily obtainable electronic components and consist of a three-axis accelerometer, a three-axis gyroscope, an Arduino Mini Pro, a XBee wireless transmitter and a small rechargeable Lithium polymer battery. The small size and compactness of the wireless system made it ideal for the dancer to wear several of these on the extremities of the limbs. The choice of sensors was also ideal for detecting the attitude in space at rest and the acceleration and rotation energy expended on the limb during motion. The downside of using the Arduino microcontroller was that several were needed, due to the need of more analog inputs in order to cover multiple limbs. By combining these data streams it was possible for Schacher to obtain an absolute reference heading value and the absolute attitude of the sensor, which was particularly useful for gathering a summary of the overall rate of movement and body position19. In this example, the dancer is 19 The Kalman filter equation is extremely useful in combining accelerometer and gyro data, in determining a global state of acceleration and position. Its implementation in some of the mapping in Anxiety will be discussed in the following chapter, when technical concerns are considered. 43 wearing wireless sensor bracelets that contain a LiPo battery, 5DOF sensor, Arduino Mini Pro, Xbee Module. Figure 6. Schacher’s wireless sensor bracelets Standing Waves (2010) is another interesting work that incorporates a wireless sensor suit mounted on a dancer. Carol Brown, Anne Niemetz and Philippa Gander created the project. The work establishes an interaction between the dancer and a malleable sound environment, as the dancer controls sound by exposing or hiding the light sensors integrated in her costume. 44 Figure 7. Standing Waves; a system exploring embodiment Brown writes; “In turn, the sonic feedback influences the emerging choreographic score, inducing constraints and generative cyclic patterns for movement. This feedback loop between movement and sonic state creates waves of sensation heightening the experience of the space as a perceptible field of embodied technology. The performance exists at the threshold between the figurative and the factual as it takes data and information from the lab practice of a sleep scientist and reinterprets this within the condition of a performance environment, effectively making visible the dynamic processes of physiological phenomena” 20. 20 Taken from http://www.adime.de/standingwaves/index.html, accessed November, 2011. 45 The look of the costume was particularly influential on my sensor suit, Sculpt 2. Specifically how the wires trace the fabric in an aesthetically interesting way, at the same time as allowing the dancer’s movements to be unhindered. The details of the costume itself will be discussed in the following chapter. 4.3 Bridging the two I return to Schacher and his experiments with merging both these methods for acquiring kinetic movement. In his paper Motion To Gesture To Sound: Mapping For Interactive Dance, Schacher discusses his use of two complementary sensing modes in an integrated fashion to obtain higher- level expressive gestures. “Gesture mapping builds the bridge between the domains of space, bodies and motion and the abstract structures of a real-time interactive audio composition.” (Schacher ,2010 p. 1). He attempts to reconcile this idea of establishing a bridge by utilizing marker-based motion capture systems which are fed into robust movement analysis engines. These motion analysis engines are being integrated with real time systems for image and sound analysis and synthesis to create unified platforms for the creation of multimodal mediated dance works (Schacher, 2010 p. 2). These two integrated sensing modes can be summarized in the below diagram: 46 Figure 8. Schematic structure of a multi-tiered mapping. Schacher proposes that using multiple input formats, namely, sensor data and motion tracking, allows abstracted gesture be made to express a richer meaning. On the other hand, results in the complexity increasing and the readability of a mapping from an outside point of view diminishes21. Attempts to propose a solution to this problem, which he implements in his interactive dance piece Move2; “Special attention has to be paid to contradictory information delivered by differing sensing mechanisms; an indication of the influence and bias exerted by the affordances the sensor channel on the acquired data. A robust data reduction, correlation and weighting algorithm is needed to help assuage this problem, but aesthetic and perceptual decisions must still be made by the composer to get the aesthetically meaningful results” (Schacher, p. 252). 21 Taken from the Proceedings of the 2010 Conference on New Interfaces for Musical Expression (NIME 2010), Sydney, Australia, p. 251. 47 4.4 The Future of Dance Technology Having provided a brief survey of the main technologies used today, it is useful to return to the artistic integrity Coniglio brings to the fore. Not in the technology used, but rather the aesthetical content of the work itself: "A particular emphasis is on affective, expressive, emotional information. In fact, it is the capability of interpreting expressive information that allows interaction of technology and art at the level of the language art employs to convey content and to provide the audience with an aesthetical experience" Camurri (2004). In summary, I return to this idea of remediation and intermediatity. Intermediatity is not just the presence of dancer bodies and media on stage. While the structural components of the work in question rely on this relationship, the performing body itself functions as a medium of expression through a plurality of meanings – and thereby generates intermediality in dance (Boenisch, p. 152). It is obvious that the diffusion of a codified semantic grammar and vocabulary in traditional dance/ballet is broken down in contemporary dance today: non- traditional dancing bodies without the homogenous uniformity of tradition technique present a pluralism that is often ambiguous and that the physicality of their presence “thickens rather than dissolves, the body with semantic mass” (Lepecki 1999, p. 137). Boenisch proposes that it is in this semantic plurality of physicality that intermediality can be found; by de-mediation of the body on stage via the fundamental breakdown of a misleading binary notion of media versus bodies22. One such way towards facilitating a breakdown of this duality is to use the dancer’s body as a means of sculpting abstract gesture, as it were, devoid of representational reference 22 Extrapolate on Boenisch’s breakdown of representational mediation – p153 48 points and preconceived notions of identity. This type of ‘corporeal intermediality’, to use Boenisch’s phrase, is to fracture and alienate the body and its surrounding media. In this way, communication through representation is emptied, leaving no trace of character, identity or persona; which instigates its own sculpted interior meaning. If one looks at a work such as Bodyscapes23 it is clear the appearance of the dancer’s body seems to belong to the negative space. The stark black set, black costume and mirroring sound break down, almost perfectly, any sense of character, identity or binary influence between body and media. The effect is one of witnessing a black hole, of sorts, where primal negative energy is writhing through a state of becoming. The term sculpting comes to the fore in this work, as means of evoking intermediality: by redefining semantic plurality of physicality. It is impossible to talk of the re-mediated body and any ideas of an intermediality without looking, at least in overview, at Merce Cunningham. Cunningham is famous for his breakdown of choreographic gesture by placing his highly-trained dancers in the position where any movement has the potential to be a spontaneous realization of a raw process, defying spatial convention and technical innovation at the same time. For Cunningham’s Biped (2000), his danc- ers worked in a studio surrounded by video cameras, which track the movements of reflective balls attached to the dancer’s body suits. These body suits then take upon the form of pointillistic constellations, as the body is reduced to a conglomerate of node points. Thus, this de-mediation of the body instigates a unique semantic vocabulary: the body is translated to pointillistic positions and distances, as the software calculates these differences between nodes. 23 The premiere of Bodyscapes by Valentina Moar, Gerhard Eckel, and David Pirrò occurred at the University of Music and Performing Arts Graz, produced in the context of the Embodied Generative Music project (embodiedgenerativemusic.org, 2009). 49 “Intermediality in contemporary dance performances is located at the point when the bodies of the dancers intersect with their role as a medium, as opposed to the dancing bodies inter-acting with technical media machinery. The latter is acknowledged only as part of the intermedial processes and not a necessary condition for intermediality.” (Boenisch, p. 151). It is the author’s belief that in order for a work exploring dance and technology to reach this “remediated” state, there needs to be a resolution between the technological hype associated with state of artistic developments and the search for a pure artistic voice and its journey through the piece in question. If this is achieved, then the bridge between technology and performer is evaporated and it can be said that intermediality has been realized. 50 5.0 Technical Concerns for Anxiety of Influence 5.1 Preliminary considerations Any production of interactive new media art via physical computing can be summarized as a stepwise decision procedure. Eduardo Miranda and Marcelo Wanderley, for example, propose the following (2006, p. 4): 1. Decide on gestures that will be used to control the system. 2. Define gesture capture strategies that will best translate these movements into electrical signals. This is typically done using a variety of sensors to measure hand, arm, lip, or other body movement, velocity of movement, pressure, or any other variable of interest. 3. Define sound synthesis algorithms that will create the sounds to be played; or, define the music software to be used for control of prerecorded musical processes. 4. Map the sensor outputs to the synthesis and music-control inputs. This mapping can be arbitrary, so any unusual combination would be feasible to instantiate as any coupling of gesture to sound known in acoustic instruments. 5. Decide on the feedback modalities available (apart from the sound generated by the system): visual tactile, and/or kinesthetic. Custom-built hardware allows the intermedia artist to define for herself the range, accuracy, and subtlety of her instrument. How to construct the “right” instrument for a particular work is impossible to specify in the abstract. Nevertheless, there are a few considerations that apply to a wide range of applications, including off-the-shelf systems, in the effort to accurately 51 translate human affect, through gesture, into sound and image art. We will return to ideas of mapping physical gesture, in the context of Anxiety, after an account dealing with the practical details concerning the first performance of the piece is carried out. 5.2 Hardware, Software Configuration, Set Design and Costume The performance venue was Granoff Center for the Creative Arts at Brown University. The work was produced in a very large black-box theatre with flexible configuration. Figure 9. The Granoff Center for the Creative Arts For the projection design, I used one very long projection screen (3840 pixels by 1020 pixels) using three side-by-side HDMI projectors, all of which had a maximum resolution of 1280 pixels by 1020 pixels. The complete configuration of equipment was as follows;  Three lightweight aluminum screen frames  6 portable Mackie speakers 5 plus a sub) and their accompanying stands  A mixer with enough aux channels to allow for 5.1  4 HD documentation cameras, for recording the performance  3 BenQ projectors  Tiered seating 52  A MOTU firewire Traveller audio interface For the performance, I used my fat Mac Pro Tower, with two Quad-Core processors, a solid state drive running the applications and 14 GB of RAM. The software used was Max 5 (multi- threaded), for the 5.1 surround sound and the projected materials. For the screen surface itself, I used one very long piece of white spandex fabric, which, when stretched taught over the three aluminum frames, looked like one seamless screen running the whole length of the space. Below is a still taken during some testing on projector placement. Figure 10. The back glass wall of the Studio 1 space 53 Figure 11. Blackout-screens lowered. Figure 12. The tiered seating and mixing position being installed. 54 To facilitate a resolution of 3840 pixels wide by 1080 pixels high, a digital TripleHeadToGo DP was used. 5.3 Sculpt 2.0: A Wireless Sensor Suit Much experimentation was carried out regarding the costume the dancer was to wear. The choice of materials had to be made of cotton, so it could be washable and would accommodate sweating (unlike polyester) without inhibiting the flow of the dancer’s body. It also had to house the sensors distributed about the body and the circuit board. The cabling was circled around the dancer, using strips of neoprene to hold them in place. Then the white pants and long-sleeved shirt were put over the top of the cabling. Some trial and error occurred, to get the cabling positioned so that is stayed secure and was out the way of the dancers joints. The electronics of the wireless sensor suit is called Sculpt 2.0, and was based on previous work completed for my Masters Thesis project at Brown University. Sculpt 1.0 was a custom designed gestural wireless glove system designed and built from scratch, including a Bluetooth wireless component and self-programmed PIC16F690 microcontroller. To expand the number of analog inputs available in this chip model it was necessary to add two multiplexers to the circuit- board, to allow for a total of 24 sensor inputs. Each hand has 5 flex sensors, one for each finger, 4 FSR sensors and 1 triple axis accelerometer, providing the fluidity of each finger whilst using the gathered data to translate moment to moment organic gesture into synthetic sound and synthetic image. The C code used to program the microcontroller was created in consultation with Butch Rovan. Sculpt 2.0 develops upon the original idea by adding sensors to the whole body, though this time, a new modern chip was used. Other benefits of using a PIC model from the 24 series will 55 be discussed later. The compiling, linking, burning to the chip and debugging of the C code was carried out in MPLAB on a Macbook Pro. Below is a diagram outlining the various sensors mounted around the body. Figure 13. Sculpt 2.0 and outline of sensor positions. 56 As with Sculpt 1.0, the second prototype attempts to create through the dancer’s movement interactive synthetic sound and synthetic visual projection. A complete parts bill for the suit, as well as the suit’s schematic can be seen in Appendix 1 and 2. Both sensor systems are responsible for collecting the raw kinetic movements, translating them into serial data picked up via the serial Bluetooth module and piped into the application Max 6.24 The below diagram shows the flow of data around the whole system. Figure 14. Flow diagram for the dissemination of data 24 Max 6 is a product of ww.cycling74.com 57 There are many fleeting instances, more substantial motifs and other gestures of varying lengths all throughout Anxiety that require a vast host of data-streams in order to work. Taking the aesthetics of dance and body nuance as the creator of real-time audiovisual art, let us look briefly at a couple of software engines used throughput the piece. This will not only convey compositional structure and ways to instigate a delicacy in phrasing, but also can highlight some practical concerns such as essential tools for travelling through the various sections of the piece. A technique I also employed in the mapping was to gather various streams of data, combine and average them to gain further, more global, data: for example, if there was an extreme movement in both the right leg and arm, then a specific threshold could announce this. Or another example, if the body is relatively still but both legs are moving, this can also instigate another event. This kind of data analysis can only be determined by the population of many streams from different but related places. Then data conglomerates can be used to provide statistics on elements of posture, the amount of movement throughout the whole body over how much time. 5.4 Outline and features of the microcontroller 1. New on the microcontroller market at the time of this paper was a larger model within the PIC24 series. This series had a much-improved technology over the old 16 series, which in essence meant faster, safer data acquisition/transferring, more ram, and faster internal clocking speeds. The 24F16KA301 Analog-to-Digital Converter (ADC) module allowed for 10-bit resolution for its analog input channels . The internal clock also allowed clock speeds of up to 32 MHz, compared to the 8MHz 16-series. It had 3896 RAM Bytes and 1024 bytes for the Data EEPROM. The chip’s supply also maxes on 3.6 volts, thus making the board a little smaller by eliminating the need for an extra 5-Volt regulator. 58 This also meant I could use a petite 3-Volt Lithium coin battery, instead of the bulky 5- Volt battery. 5.5 Sculpt 2.0 documentation I spent considerable time in consultation with many people working out methods for mounting the sensors and programming the microcontroller. Of particular help was Eric Singer, Roger Dannenberg, Joel Marchewka and Jonathan Lui. The first stage of prototype was done on a breadboard, of which the images below show the arduous phase of trial and error towards getting the microcontroller serial input into a computer. The first prototype of the suit was a key turning point in discerning the final guise of it. Most of the wiring and parts for the suit came from re-appropriated commercial components. The gloves are sturdy bike gloves, the cabling for the arms VGA cabling, cabling for the legs Ethernet, and the hub used to house the board and bluetooth module was taken from an old Verizon internet hub. The advantages of using this hub was that some required parts were already contained in the shell, such as one Ethernet jack, a power switch and 2 status LEDs. 59 Figure 15. The circuit board and its housing 5.6 Innovations in the C code Part of my intention when designing Sculpt 2.0 was to eventually create a stable and relatively reliable system for dancers to use in a variety of artistic situations. This meant that I needed to research the way the chip was to be programmed, so it could work at an optimal rate. To this end, the following factors were added to the C code: • Each channel in the incrementing loop utilized a 10-bit ADC resolution, converted to binary efficiently via USART and only on a changed value. This saves on power and cycle speed. 60 • Analog conversion and serial transmit are re-tried on a detected failure after re-setting the peripheral to avoid missing good data. • Special Function Registers are turned off right after use, to save power even within a loop. • The code was broken up as much as possible into functions, since this allows for faster cycling in microcontrollers. • Naming conventions were determined before writing the code: functions were to use camelCase, start with verb, optionally followed by modifiers or object of verb, variables use camelCase, end with noun, optionally preceded by adjectives modifying noun, any pre-processor constants would use TAB_DELIMITED_CAPITALS, describing constant values throughout the main file and all other #includes. • Each analog channel value is sent coupled with its ‘control’ value – this way the data for each channel can be routed separately throughout the whole network. • Variables were moved into the main function and passed as required, which this avoids having "global states" as much as possible, in order to simplify debugging. • For the PIC24 series, passing char or void is faster than any other type of passing because there is a special 8-bit register that can be used. This alleviated more time-intensive memory copying. Integers are passed if they were required and pointers were not used as they would pass extra bits that weren’t required. • Initializing all local variables occurred at the start of each function, to ensure a value is set before being used. 61 • Nested loops can be computationally expensive on microcontrollers in terms of memory and time. Avoiding them when practical results in less memory usage and faster execution. • When transmitting on the TX pin, the code tries to transmit once and reattempts if necessary until successfully passed. This provides a lot more stability in the data being received in the serial port. • Timers were started and stopped as needed to reduce unnecessary current draw. • Furthermore in reference to power management, the code turns off VDD to all unnecessary peripherals and their associated registers so they don't consume power, which is fairly quick to do within the loop. This feature is not available on every chip model, but it is for the PIC24FV16KA301. • All interrupts were shut off, since flag testing was hard-coded into the functions, only to be called up as part of the data parsing. • A reset ADC function was included in case of any errors or lockup occurs. • Function calls within a loop can be expensive in terms of time on a PIC, so a microsecond delay routine was placed within the loop to eliminate this extra delay and increase accuracy. • The two bytes of ADRESH and ADRESL were combined into a single integer value, before three bytes are sent (the first is the ID, or control number, the following two are the data). 62 5.7 Initial Data flow and treatments Bridging the physical and digital worlds in a real-time work of art requires a breakdown of task assignments from sensor data to the final artistic output. The data flow is localized in three places: the physical instrument, computer processor, and the output sound and 3D imagery. Converting the raw streams into useable data resulted in the implementation of a variety of functions for smoothing and filtering the data. After these data streams are treated and sent to the computer’s serial port, they are disseminated through the various software engines that produce the reactionary sound and image. The flow of information was influential to me when designing the sensor suit, when contemplating strategies for mapping. Many elements that make up the piece were results of scaling and mapping one data set onto another. These mappings also required experimentation afterwards, for tweaking data conversions and tempering parameters specifically for a once-off moment in the piece. By tracing the flow of data and energy from the suit’s raw physical movement to the final digital output, it was also important to find ways to transition and translate data sets between media. It was an important these kind of conversions allow for kinetic movement to influence both sound and graphics simultaneously. Depending on the material, it was also important that sound could influence graphics and graphics could influence sound. The following diagram was used as a guide, when I was developing methods to translate the kinetic into real-time sound and graphics in meaningful ways. It was useful for me to conceptualize the data flow from the physical instrument to computer and back out again via a diagram such as this; it allowed me to see how the technologies would cause the piece to grow in a natural way at the same time as 63 producing aesthetical interest. The loop, or feedback, indicated by the arrows allows the performer to be influenced and react to the composition while the program collects, modifies, and disseminates the data, unhindered by the compositional processes in motion. Figure 16. Task allocation breakdown 64 The data flow is localized in three places: the physical instrument, computer processor, and the output sound and three-dimensional imagery. Gathering the raw kinetic data is undertaken in the physical computing domain. Three types of hardware are required: input capture (the sensor), the brain that translates this data (the microcontroller), and technologies for translating the data into the software used for sound and image production (serial port conversion). Along the way, various components are also required to smooth and filter the data at every stage of its journey. The second part of the data flow is carried out by Max/MSP/Jitter. Both parts of the data flow will be discussed in greater detail below. It is relevant to now take a look at founded techniques and self-designed engines in software, to which all these considerations are put into a real-life context. To begin with, it makes sense to look at the abstraction that is responsible for capturing the wirelessly transmitted analog signals coming in from the suit. To separate each sensor channel, before labeling it ready 65 to be sent, the Max object [spray] was used. Figure 17. The serial object collecting at 15ms via [qmetro]. It was then a matter of feeding the packets immediately to a smoothing engine, to not only remove any extreme spikes in the raw data but also to filter the stream through an averaging system, in order to produce a sturdy and reliable cooked output. The data was also conditioned and then scaled to fit 127 values. Experimentation was needed on each channel since every axis of the accelerometers and gyro give slightly different ranges. 66 Figure 18. Sensor Conditioning, smoothing, scaling and averaging. The above diagram can be broken down into eight stages, as indicated. Stage 1, via the [change] object was essential towards processing power management. Only a different value to the previous is sent, as repeats are discarded. Stage 2 shows an interpolation between the new value and the previous one, as the [line] object smoothes the transition over 5ms. Stage 3 scales the stream to fit 1. to 127. The first and second arguments of the [scale] object are different for 67 each sensor, since the analog sensors of today tend to vary, sometimes greatly, in the range of values they return. Stage 4 shows a method for logical comparisons. This enabled [gate] object to be opened or closed, as a result of these comparisons. Stage 5 limits the amount of data sent to 5ms by the [speedlim] object. Stage 6 again interpolates smoothly between old and new values, this time over 10ms. Stage 7 compiles the most recent fifteen new values into an array, which is sent to the [mean] object as one complete packet. The [mean] object then outputs a running average of this packet. Stage 8 attaches the cooked final value to a unique name, to be recalled on the fly, anywhere at any time throughout the piece. The next issue was to control the flow of these streams, taking care that every single strand of data was managed efficiently around the various modules and sections of the piece. Since the serial object that is sampling the data at a very fast rate (15ms, as seen in the [qmetro] above), having any of this data being sent to obsolete places could cause the patch to trip or fail. The way to avoid this situation is to methodically route the streams on and off around the various engines in the various sections of the piece. Compositional structure, then, becomes the timeline of this kind of triggered gating. For example, the below diagram shows how the sensor data for channel 1 is gated off from the first section and then opened for the second: 68 Figure 19. Gating on/off sensor data In this example, the abstraction [bangGate] works in the same way, in that when the first section is opened (1_section is sent a 1) the gate allows bangs to flow around the subpatch. In the same way as economizing the sensor streams were [qmetro] bangs handled, when distributing bangs around all sorts of processes. The dissemination of bangs comes from one place only, where the render context is also defined. This way, no single bang is ever wasted anywhere in any moment of the piece, as bangs are routed on and off throughout each section and each sound or graphics engine. 69 Figure 20. The Render Context, Window setup Through experimentation, I found it necessary to use the [qmetro] not only for the dispersion of bangs to jitter objects (which is a common technique) but also for updating the serial object taking care of the sensor data. This meant that signal processing on the CPU was assigned first priority, regardless of how heavy the graphical processing encompassed moment to moment. This is specifically vital in Section 2, where the various FM engines must not falter in their rhythmic subdivisions, where meter and pulse must not fall from anything but stable. Notice also in the above illustration the flow of bangs at the scheduler level; all openGL objects get drawn immediately after the renderer is erased and before the renderer is newly updated. 70 Figure 21. The transport object dispersing rhythmic subdivisions The transport object provides all rhythmic foundation throughout the whole piece, and the 1_beat [send] object is set to high priority at the scheduler level. To have a tight control on signal routing, the [poly~] object was used extensively. Once the close of a section was triggered, the [sectionSettings] object would be sent a 0, causing all open MSP and Jitter engines to fade out and turn off, as the below diagram displays; 71 Figure 22. Template abstraction for opening and closing a section An addition to the final patch, something I added during the rehearsals, was a triggering mechanism, using the computer keyboard. This meant that if anything untoward occurred in performance, I could always manually move through the piece by triggering events via pressing the space bar. This kind of failsafe is essential with new media performance. When all is said and done, the piece must not be jeopardized as a result of a technological hitch. Some other necessary additions to the patch were required, regarding performance considerations. Firstly it was my desire that the audience did not see any of the technical setup 72 right before the piece began. To me, too many new media performances destroy the ambience and mood of the atmosphere of the piece, by showing all the clunky mechanics of dragging the windows around and navigating the computer’s desktop with the cursor. All of this was avoided by the following setup in the patch. I also connected the wireless sensor suit as people were coming in, since AA batteries last longer than the piece needed. Figure 23. Setting up the renderer’s window ready for performance The above patch was compiled from various posts that appeared on the cycling74 Max Forum. It was used for setting the resolution of the renderer’s window, dragging it around without the cursor showing and being able to align it at full-screen resolution perfectly on the second 73 monitor (the TripleHeadToGo). This prevented the entering audience from seeing anything on the black screen until the piece actually began. I also found in rehearsals that it was easier and more efficient to have the sensor connection abstraction open as a separate patch, instead of having it embedded in the main performance patch. This meant it was easy to stop/start sections and quit out of things, if need be, without having to reconnect the wireless device. 5.8 Mapping Strategies Two important discoveries were revealed during the experimentation phase, with the sensor suit prototype. Firstly, it was essential that there be a [float] object right after any [scale] object that has minimum and maximum values set. This guarantees that no values outside of the range will be sent, which can avoid major problems, if for some reason, the sensor sends out a rogue value that manages to get through the conditioning phase. For example, when using sensor data to control the position of a 3D object in Jitter, without this restraining it is possible to send the object spinning somewhere way beyond the screen space. Secondly, I found through trial and error that when mapping sensor data to real-time changes to opengl contexts that using the [bline] object secured stability in how the data influenced the objects in question. This meant that the data affecting the object is only sent out at the resolution of the bang [qmetro] and not any sooner. Not only was this important to the stability of the computer processors but also gave a smooth and synchronized result in both Jitter and MSP moments. 74 Figure 24. The [bline] object Figure 25. Markov Chain patcher 75 To decipher all the mappings around the whole piece, the below diagram was used. Once I had this diagram, I found it much easier to follow all the paths of data, making sure things were turned off at applicable moments and sending them seamlessly to the new destination. Figure 26. Diagram used to keep up with data routing 5.9 Initial Setup of the 5.1 Surround Configuration and Its Mapping Because the sound system for the performance was set up using a MOTU Traveller and an analog mixing board, I had to control things like Bass Management and the LFE channel from within the software. When all signals throughout the patch arrived at the final DAC~ engine, it 76 was necessary to set up a crossover frequency threshold, so that all suitably low frequencies were routed to the Subwoofer. This meant that I had separate control at the mixer on the amount of sub diffused in the actual performance, tweaking moment to moment, if needed. Figure 27. Base Management and the LFE channel configuration A very important part of the piece’s original concept was the idea of the body diffusing sounds around the space real-time. This was achieved by mapping various data streams to a crossfade mechanism that was used over and over again, from moment to moment throughout the piece. 77 Figure 28. Sensor Panning abstraction 5.10 Kinetic movement data as 3D co-ordinate transformations Much of the interactivity in the visual domain was a result of various sensor data mappings to three-dimensional space and opengl parameters. Some tasks manipulated vertices on a three- dimensional mesh, or model, some tasks manipulated color information, others influenced changes in matrices and some modified camera motion. The mappings could be opened at any time and in any combination. Trial and error was the method for deciding on which sensor channel from the host of sensors distributed around the body would be used for any given mapping. Some parts of the body seemed to be better suited to certain manipulations than others. For example, the Z axis on the accelerometer mounted on the dancer’s back was useful for once 78 off big changes, since in order for its values to change, the dancer needed to bend his body almost backwards. Where as the Z axes on the wrists could be used for more regular and subtle mappings. And sometimes only one sensor was needed to achieve a desired result, while other times many sensor channels were more useful. Below is an example of five sensor channels mapped and scaled appropriately to two NURBS meshes. Sensors 5, 6 and 15 manipulated the control vertices of the two meshes, while sensor 7 altered the amplification parameter of the [jit.bfg] object and sensor 5 was used to crossfade between the two meshes Figure 29. Two NURBS meshes manipulated by sensor data. Another example, taken from Section 2, shows how three sensor channels modify the effect of a shader loaded in a [jit.gl.slab] object. The shader used in the case was taken from Vade’s v001 79 shader collection26. In this instance, a movie source is downsampled before piped through a chain of effects objects. The 2_sectionSlabGate] send object gates the two last shader effects on or off, depending on whether the two objects are needed of bypassed. Figure 30. Sensors modifying shader parameters. Below is a different example that collects extreme values from a sensor and then triggers dramatic changes to the visual rendering. 26 http://001.vade.info/ 80 Figure 31. Sensors piped through a threshold trigger. 5.11 Sound Mappings Much of the interactivity in the sound domain ranged from simple threshold triggering through to filtering effects, such as using sensor channels for live filter sweeps or EQ fluctuations. Sensors were also used often to scrub soundfile buffers. An example can be seen in the second section, where the sound sample looped is panned around the five channels via X /Y pairs on the left and right arm accelerometers, as well as on the Y axis for the back accelerometer. This looped sample in the next stage of the second section is routed through a granular engine, where all sensor streams dictate the number of grains, the panning spread and individual lengths of these grains. The contrast between the two treatments of this sample was also explored through the choreography, where gestures were predetermined so that the dance could provide an effective scalability. 81 Often many sensor channels were mapping to many different buffers, or [groove~] objects that recall the same content from just one buffer. The effect is a sort of textural field where subtle changes to the layers of sound give an interpolated granular kind of effect. The choreography during these moments tended to reinvent the gestures in a more flowing and lucid form. Below is an abstraction that was used in multiple sections to provide very discrete EQ fluctuations on specific spectral content. Each sensor is assigned one specific band of frequencies to alter. The [inlet]s in the below are fifteen sensors packed together into chunks. Note also that the choreography was also predetermined to fit, not only to provide the right type of scalability for sensor treatments but also to aesthetically follow these subtle timbral changes. Figure 32. Spectral delays influenced by sensor streams 5.12 Other Miscellaneous Mappings Below is a collection of methods for decision making on the incoming sensors. All of these modules were used in various ways throughout the whole piece. 82 83 Figure 33. Various mappings 5.13 The Rendering Pipeline for Acousmatic Production Probably one of the most time consuming components to produce in the piece was the 3D animation. Even on a fat G5 dual quad-core machine, the rendering took weeks. Part of my initial idea for the piece as a whole was the merging of both real-time spontaneous interactivity and set pre-rendered episodes. It was my wish that by combining the two, I could create a coherent structure where thematic materials could be explored both live and developed in fixed media, section to section. This would hopefully provide a close bond between sound and image through architectural compositional design. Section 1 and the Bridge were completely comprised of set multi-channel audio and pre- rendered 3D animation. It was my aim in these two sections to allow both the sound and graphical materials to be integrated into a non-interactive audiovision work. I used the programs Metasynth, Studio Artist and Maya to produce sound from graphics and graphics from sound. I also used Maya to create three-dimensional models, before importing them into Mudbox and Zbrush for high-detailed sculpting. The process of importing OBJ models into these packages, giving them high-resolution detail, before bringing them back into Maya to animate will be discussed later. Once I had rendered out key motivic three-dimensional scenes as TARGA image 84 sequences, it was possible to recombine them in various ways to create the overall structure in Adobe Premiere Pro. I then could create the soundtrack from imported sound samples created in various audio packages. I chose to use Premiere due to its ability to easily exchange projects, compositions, sequences, tracks and layers with After Effects. After Effects is fantastic for creating moving 2D graphics, applying visual effects, performing overall color correction, and performing other post-production tasks not available in any other package. I knew I would want to end the whole project in After Effects before rendering out the final audio and movie files, so it seemed natural to make use of Premiere’s integration. Knowing I wanted the audio to be surround in its final format, I exported each audio track in After Effects separately, so I could then have control over the individual sound regions when I imported them into Logic Studio, to master the sound into 5.1 surround. Once I had finished the mix, I exported the six separate mono files representing the six channels of the 5.1 standard, ready to be used back in Max. The next problem concerned the playback of these final audiovision interludes, where it was essential that the video and audio files play back precisely in synchronization. Because the movies files were HD quality, exported using the Apple Pro-Res codec at the resolution of the final render context of 5760 pixels by 1080 pixels, I had to develop a way of playing these files in Max, that would play smoothly without lagging a frame, at the same time as being in sync with the six mono soundfiles. Max has still a way to go before the issue of HD movie playback is resolved, but with some research and much pouring over the Max Forum listserv, I developed the below patch; 85 Figure 34. Patcher for audiovision playback 5.14 Workflow pipeline for 3D sculpting To produce the high resolution sculpted meshes in Zbrush, optimized for importing back into Maya ready to animate, several stages were carried out in order for the model to be work as expected on import. Firstly, it was essential, when using Maya to create base meshes, to use only quad geometry, using triangles only if necessary – sculpting packages like Mudbox and Zbrush will fail at geometry that contains more sides that four or less sides than three. As a result of many false attempts, I discovered that using the Optimize Scene option under the File menu and the Clean Up option under the Polygons menu were crucial, to make sure your model didn’t accidentally end up with weird vertices that would cause errors down the track. The Freeze Transformations option was also absolutely necessary when finishing up the model; otherwise the model will reappear in a way off place in the three-dimensional plane field. It was also important, when considering texture mapping, that the model have a useful set of logically layer 86 out UV’s27. Spending time on these (even though its takes hours) will facilitate accurate texture mapping onto the model. Once the model was exported and successfully imported into Zbrush or Mudbox, it was lots of fun to sculpt very fine detail at impossibly high resolutions. The detail in this sculpting was finally exported separately, along with a low-resolution version of the model. This process involved exporting separate high-resolution image files at 4096 square pixels, 16bit, TIFF format for all maps. The only files I used for my Maya animations were texture color, geometry displacement (for extreme undulations in the geometry), a normal map (for very fine details in the geometry) and one for specularity (texture highlights). These separate image files could then be imported back into Maya as nodes associated with the texture shader assigned to the low- resolution mesh. Further maps can be created for use in animation, through I didn’t use them in this pipeline. Below is a diagram showing the workflow I adopted, starting with primitive model and ending with a low poly-count model with high-resolution TARGA image files, ready to be animated in Maya and exported as a final image sequence. 27 I spent quite some time learning about UV sets and their importance in generating hi-res textures over the whole model. Putting time into laying out UV’s correctly removes the chance that any pixels on the texture image file will be stretched over the three-dimensional vertices. 87 Figure 35. Workflow pipeline for 3D sculpting in Mudbox Once anything is brought into Maya from Mudbox or Zbrush the normal angle needs to be set to 180 degrees, otherwise the model looks kind of nasty; this stage took me weeks to work out. I used the Mental Ray renderer in Maya, so that I could use ray tracing, to imply natural light reflections and refractions on the geometry and textures in the scene. In order for Mental Ray not to choke, it was necessary to convert all the TARGA files to .MAP files. In Maya 2011, there is a preference that can take care of this for you. After the models had been imported, with their various texture map files added to the shader assigned to the model’s texture node, I then used the Blendshape menu in Maya extensively, to create various animated morphs between different imported models. 88 5.15 Research into choreographers towards deciphering a style Through most of my career over the last ten years I have worked with a wide range of dancers and choreographers. From two ballet commissions, I was fortunate enough to learn much about creating a score for a full-length ballet, where soloists, the corps de ballet, and staging were all delicately interlinked components. I have also worked with dancers using technology. But Anxiety was a type of piece I had never had the chance to explore. In order to find a gestural repertoire that organically fits with the interactivity (given that the body-mounted sensors have their own orientation and do not pay any attention to the natural flow of a human being’s movements), I researched the language of choreutics and the history of choreographic technique, particularly when media was incorporated. As part of this journey, I took some introductory modern dance classes, with the aim of better understanding the art form. I also did extensive reading, saw many dance performances and talked to dancers/choreographers as often as possible. My goal was that once I had acquired a fair understanding of the various techniques, I would then be in the position to find my own style in adapting movement to fit in with the body-mounted sensors. This would hopefully bring about an individualistic approach to choreographing the piece, while still exploring the interconnections between improvisation, dancer, screen and interactivity. One of the key discoveries I made was making the most of the dancer’s weight, exploring different movements from different sections of his torso, sensing his spine widening and lengthening, while the head and arms suspend in weightlessness. This provided great data from the gyro and accelerometer on the dancer’s back. By making these kinds of movements, where other parts of the body became softer while the torso was very active, a new liveliness infused in 89 the whole body – a liveliness that seemed to me to be not very ‘dancerly’ but still organic. I eventually concluded that perhaps the suit itself could provide new ways for the dancer to sense himself, as if the outer skin allowed the inner artist to explore new facets of expression. In other words, the suit, by enveloping the dancer, would redefine patterns in technique and birth new shapes for the body. Part of my discoveries in this regard were related to somatics; an approach to working with the dancer’s body where movement is experienced from within rather than objectified through outward training. The objective of this technique, in my case, was to provide a self- knowledge of movement, where the mounted sensors are part of the body. This approach alleviated the sensor suit becoming a heavy burden that caging the body and allowed new possibilities for it to be used to actually inform the dance. The dancer is also seen as a body enveloped in what has been referred to as the Kinesphere28. The Kinesphere is representative of the spherical space circumambient and is represented by various configurations of one, two and three-dimensional planes. The parts of the body are referred to as zones and bodily structure and posture is formed by a separation of these zones, as determined by an imaginary geometric shape, such as a cube, tetrahedral or pentagon (Laban, p. 20). The five principle zones align perfectly with the position of sensors on my sensor suit, and I found that thinking of the suit in terms of this Kinesphere was extremely useful. Ideokinesis was also influential on the choreography for Anxiety. The technique explores ways in which the mind informs the body – physical sensation produces movement, as if the movement itself is not as important as how the mind created it. Ideokinesis is particularly 28 For a comprehensive analysis and practical guide to Kinesphere and other related aspects of choreutics, see The Language of Choreutics, by Rudolf Laban (Plays, Inc, 1974). 90 relevant for a dancer wearing a sensor suit. It provides a tool for the dancer’s mind to produce mental images of the piece, while being aware of inner points within the body. The technique is discussed in Mabel Elsworth’s book, The Thinking Body: “The nonmaterial activity of thought literally changes the material structure of the body. This was thoroughly a non-Western view of mind’s relationship to matter and the opposite end of the stick from the ballet/modern tradition of repetition of form to train muscle. Ideokinesis placed the mind’s ability to posit imagery, and the body’s possibility of responding fully to this imagery, at the core of technique” (Todd, 2008, p 286c). The Release technique is related to Ideokinesis and was also well suited towards defining a style of dance. Release takes the physicality of the dancer’s moves as a departure point, in accordance with the laws of physics, rather than any aesthetical framework. The technique also takes into account the division of space through the moving body, where height, depth and breadth become the tools for exploration. Essentially, it was clear to me that for a dancer, the relationship of gesture to live sound and image production cannot be defined in the traditional paradigm of choreography. Traditional technique places its own semiotics on top of the ones present. A dancer is vividly aware of the sensors on their body, even if these sensors are not intruding. This creates an intrinsic problem; if the dancer is always aware of the sensors, then this awareness will inevitably get in the way of dance expression. The result is a superimposed, or schizophrenic, dual meaning, regardless of what happens in performance. A double meaning that has the effect of diluting, or even dissolving, a central cohesive whole. To resolve this issue is to ‘retrain’ the dancer. Which means to abandon traditional technique and preconceived notions of dance, in order to establish a new intimacy between the dancer and the sensor suit he wears. And this intimacy is framed by the 91 conscious awareness of the sensors as an extension to the natural body. "These relationships may be established by viewing the body and space as musical instruments, free from the associations of acoustic instruments, but with similar limitations that can lend character to sound through idiomatic movements" (Winkler, 1995 p. 2). In Winkler’s words lies a new lexicon of correlations for the trained dancer wearing technology. Yet this lexicon still presents an unstable and fluctuating system for phenomenological gesture that sees the production of human agency and meaning making as a process of embodied action. So, it seems to me the question that arises out of this new hybrid of dance and dance/sensor-induced interactivity is this; how do we identify kinetic motion that is poetically induced, in the context of the performance, and conversely, how do we identify merely a spasm of raw involuntary movement that is a reaction outside the aesthetics of the performance? This new type of linguistically orientated framework, through which the sensor suit funnels, denotes specialized types of motion. Which leads to thoughts about the meaning of this new framework, for movement and dance. Deciphering a new way for choreography to explore form, structure and content is to redefine gestures as symbolic meanings: "Gestures are not just movements and can never be fully explained in purely kinetic terms. They are not just arms waving in the air but symbols that exhibit meaning in their own right"29 (Schacher & Stoecklin, p. 292). 29 Proceedings of the International Conference on New Interfaces for Musical Expression, 30 May - 1 June 2011, Oslo, Norway 92 5.16 Final Thoughts About Choreography Choreography for Anxiety was much labored over, before working with my dancer and during rehearsals. At the very outset of the piece’s germination, it was important to me that the dance become a focal point, bringing to the table all the various preconceived notions of what the genre should and would include. Anxiety was never about technological innovations. Technology was the tool and always in the background of the narrative. That was why I wanted the suit to be hidden, like a second skin to the performer. It is ultimately the performer who moves the audience and to this end, the suit can’t be in the way. Coniglio summarizes the uniqueness of human performance, in his article Dance’s Digital Divide: “Yet, none of these experiences seem satisfying as dance primarily because they do not provide the viewer with the corporeal experience that one feels when one watches a living body move artfully through space. What is missing is the energy that is the human body. When dancers perform we see beautiful bodies that sweat and breath hard as they perform complex, difficult, and even dangerous movements. Athletes and dancers show us the human body at its most exquisite by seeking the boundaries of its capability. Both take chances with their bodies as they explore those limits. What makes dance different from sport is the fact that these movements are aesthetically pleasing and may generate mood or narrative – a sublime combination of beauty and risk” (Coniglio, p. 4). 93 6.0 A Detailed Structural Analysis of Anxiety of Influence 6.1 Preliminary description of the performance Anxiety of Influence was first performed in the Granoff Center for the Arts, Studio 1 on August 17, 2011. Performing the piece was dancer Tabaré Gowon. I was fortunate enough that Tabaré and I had access to the venue for several rehearsals, which meant that he was well acquainted with the space and the projected refractions upon his body. It is useful to summarize the piece as a whole, stating where interactivity lies, between other components. The whole structure toggles between fixed media presented, and both interactivity used as a form of embellishment over set components and pure interactivity. A simple outline of sections can be described using letters to distinguish the macrostructure; A. Section 1: set audiovision fixed media presented B. (or possibly A2) Section 2: interactive, as raw materials are taken from Section1 C. Bridge: set audiovision fixed media presented D. Section 3: interactive, with new materials presented and rhythmic in nature E. Section 4: set audiovision fixed media presented F. Section 5-6: set audiovision fixed media presented The piece begins with the dancer off stage. The audience enters an austere sterile environment, where two out of the four walls are glass. To announce the beginning of the piece, the electric shades are brought down, covering the glass walls, blocking the immediate view of the evening lights of Providence. Against the north wall, a continuous white 7’ x 40’ projection surface runs the whole length, white spandex stretched taut. 94 Section 1 - Creative misprision The concept behind Section 1 is taken from Blooms. A deliberate misreading of past works shows how the immensity of the past can become a metaphor for creating an expansive, unstable environment. The raw samples of sound and samples of video were taken from two places. Firstly, I collected a host of sliced up excerpts from audio pieces that have been important to me over the years. Secondly, I captured graphical moments taken from various sections in the patch, using an Intensity Pro graphics card for HD capture. The first section starts with an apocalyptic explosion and declamatory conflict, that rips and shreds at the senses.as the set cacophony of sound and visuals provide peremptory glimpses from future sections. A string of accelerating smashes overwhelms the senses before a last climactic shred in the sound makes way for the visual motif of a tower, made up from layers of steel-textured organically rippling sheets. The cacophony of sound also makes way for a low steely undulating pedal. The tower undulates as it moves around the screen, before a three- dimensional skeleton of a cube appears, as if to dwarf the tower. The interior white mass of the skeletal cube finally consumes the tower whole, before turning into a host of particles, or grains. The tower is still present but subdued, in the right projector. The tunnel of colored geometry covers the entire screen, as it zooms out from a central focal point. The sound announces the presence of the next important motif, as neon blue translucent rings grow from the shadows. These rings are very quickly engulfed by a series of diffuse neon blobby particles; they spread out mellifluously from the central focus. Very quickly a fire mountain motif enters. This three- 95 dimensional model was created in Zbrush, while the flickering fame above was created using Maya’s Dynamics suite. The fire mountain appears without a sound equivalent, suggesting that it is at present not fully formed. The next motif, a neon blue smoke-like gesture spirals organically around the other fragments, before a high-pitched cymbal-like sound grows. A throbbing bass follows, while various visual motives are fragmented in various layers of transparency; this throbbing bass was produced by recording a patch I created, that caused a characteristic clicking repetition at the end of the [phasor] object’s cycle. Figure 36. Bass throb patch A low sawtooth that makes the subwoofer rumble, shaking the room violently. The graphics show an unstable erratically moving primal soup. Gestures are amalgamated, fragmented and combined. 96 The unstable texture dissipates to reveal a low horn-like drone. The fire rock motif reappears; this time changed into a group of five florescent green crystals, moving delicately as a group, each one holding its own flame. The group is removed, as if floating on an isolated plane and unaffected by other components. The crystals appear momentarily, before vanishing abruptly. This relates to the narrative, almost like the dust settling after the first explosive climax. A window is created, through which the original steel tower can be seen. The tower presses against the window, growing and diminishing. The sound disperses and a series of steel- like ringlets engulf the visual space, grinding in circular motion, as each ring circles the others. The neon-blue smoke ring is still present in the background, as well as the geometry taken from the 4th section, captured live via the Intensity card. The second stage suddenly breaks from the silence that ensues. A heavy rhythmic repetition connects with its visual counterpart; a layered geometric plane, where a scanslide effect passes regularly. Figure 37. 2nd Stage of Section 1; scanslide 97 Abruptly, these monotonous repetitions are consumed by glimpses from the 5th section’s gestures. The swampy terrain returns, with a color-altered version of section 4’s visual motif, amongst other layers. A new conglomerate of previous motives infiltrate, waving in and out. Eventually, everything subsides. The graphics make way for the concrete/steel rings circling, with a dense dark background The rest of the section summarizes in various ways important themes and motives from the first section as well as sections to come. Figure 38. Accumulation of various thematic materials 6.2 Section 2 - Completion and Antithesis Secondly, Bloom proposes the idea of completion and antithesis, whereby a great work is studied and taken further, in a new direction. Perhaps the work can be taken further. The new is written as a ‘completion’ to the first. It is influenced by, yet moving away from the original. The section, in its whole, explores a subdued, introvert character. This is the first time the dancer appears on stage and he explores small motives that are linked to the mellifluous nature of 98 the drone-like soundscape and the diffuse and blurred projections. After a brief silence and black screen, right after section 1’s conclusion , the sound and visuals fade up to show a series of fluctuating deep blue lines, constantly shifting, as the dancer slowly enters the stage area from the side of the tiered seating. The sensors interrupt these fluctuations, very delicately, providing a warm diffuse atmosphere. The essence of this section concerns stripping content from the previous section and reinterpreting: something previously presumed complete is reworked into a satisfying stability. This is achieved by downsampling visual motifs from Section 1 and processing them real-time, before they are upsampled again, through frame interpolation, via the transform_reset jit.gl.videoplane attribute30, to the window size (5760x1080 pixels). The original Maya render that appears in Section 1 is a development from the steel rings revolving around each other, while this guise allows the rings to adopt the iridescent and translucent blue hue seen in other motifs. Below is a still taken from this rendered out sample: Figure 39. Original Maya render taken from Section 2. 30 The transform_reset automatically scales all vertex information to window size set in the render conext. Giving the attribute a 1 = proportional orthographic glOrtho(-aspect, aspect, -1.0, 1.0, near_clip, far_clip) – taken from the Max 6 documentation. 99 This source is downsampled and then fed into two [jit.gl.slab] objects with loaded shader files, before upsampling with interpolation. Below is a still taken from this stage: Figure 40. Downsampling and upsampling pipeline for Section2. 100 The dancer walks from one side to the other, as if exploring the whole space. Slowly, austerely, threads of subtle gestures begin, starting with the fingertips - then growing to the wrist, where the first accelerometer is situated. The left shoulder is incorporated, while the dancer’s eyes follow the wrist, as if exploring how it can be used. The flow of gesture moves: a momentary glimpse of the feet, with a small skip, before intension is returned to the wrist. His eyes follow the wrist, in concentration and discovery. This kind of discovery is the essence of the first stage of this section, while the formation of choreographic gesture takes place. Figure 41. [jit.gl.slab] sensor scrubbing The dancer returns to the original left wrist, followed by the left leg in the same motion. The dancer then returns to walking the stage, as if recalibrating and exploring the space as before. This walking also provides emphasis to the new choreographic phrases that follows. The next stage is announced by breathing undulations of movement that mirror the waves of sound. The 101 whole body seeps and ripples, in individual sections of the body, before amalgamating the whole. Memory to the previous returns, of past things, and so the dancer returns to the wrist. In conclusion, the dancer holds a reduced sculpture-like pose, implying a statue, or homage to early Greek Tableau. No movement for five seconds triggers the patch to move onto the second stage of Section 1. The trigger causes the 1_sectionSlabGate [gate] to route the movie matrix away from the first outlet to the second outlet, allowing a sequence of shader processing, before sending to the [jit.gl.videoplane]. This change in content is mirrored by the choreographic emphasis on angularity in the arms and legs, as the body jerks around in contrast to the lucid exploratory nature of the previous stage. These jitters also create scrubbing in the sound, where ten buffers all containing the same sample and are scrubbed and combined to create the granular texture stated above. 6.3 Bridge The bridge was constructed out of another Maya render, featuring several metal-looking rings circling around one another, similar to those that appeared in Section 1. Influences upon the three-dimensional scene came strongly from Snelson’s concept of tensegrity31. I was hoping to achieve in the bridge passage a parallel to Snelson’s architectural ability in defying gravity; how the constituent parts in one of his sculptures appear like suspended fragments framing the sky. These windows, of such, imply geometry that suggests alternative doors, further dimensions, or simply sky paintings. In another way, these constituent parts are themselves aesthetic objects that 31 The Needle Tower is composed of flexible and rigid components, which are arranged according to Snelson’s own concept of “tensegrity.” The word is a portmanteau of tensional and integrity and refers to the completeness, or integrity, of a structure held together via tension and compression. As Snelson uses the term, tensegrity is principally the same as textile weaving applied to architecture. I have adopted the principle to circular form, instead of angular polygons. 102 frame externalities. The components are not simply absorbed into the overall structure of the piece. The “woven” tubes and wires in his Needle Tower series, for example, produce a conglomeration of geometries and windows that retain their own distinctness and agency but yet are independent from their other parts, even as they ultimately fit together into a coherent architectural form. In my bridge, the sky is an undefined black mass of potentiality. But I think the negative spaces defined in between the moving rings give off the same effect. Figure 42. Snelson’s Needle Tower, Hirshhorn Museum, Washington D.C. The aim of the bridge also takes upon a hiatus in time and suspension of dialog between the dancer and screen. Its repetitious nature suggests stagnation and indeterminacy. Nothing changes as the circles and the sound pedal point loops. The subwoofer is also an important component to the bridge. It also implies a leaving of immediate space and time, as the room rumbles from no specific point but circumambient. The second half of the bridge announces again the layer tower of influence, which first appeared in Section1. Below is a still from the Maya render, before the footage was amalgamated into the other sources that make up the bridge: 103 Figure 43. The steel layered tower of influence The choreography for the bridge is subdued. Subtle movements in a low position close to the floor allows for the screen to take precedence. And again, once the dancer stays still for over 5 seconds, the next section is triggered. 6.4 Section 3 – Reduction and Inception Section 3 begins with an extreme gesture in all axes’ of the sensors. This section is an exploration of rhythmic impetus triggered by extreme sensor values, where silence assists the narrative. For example, when both arms jolt together, their summed data is collected. If this is over a threshold, a poly~ FM engine is triggered. Silence, as a result of no movement, is the theme here. The progress of this erratic stage is determined by a timer. This is achieved by routing off the triggering mechanism via [onebang]. Subsequent bangs are sent only after this timer resets the object, via its right inlet. This timer speeds up after a chosen number of events have passed, causing event triggering to also quicken. This timer gives the section its structure; 104 spasmodic events make way for faster triggering, until finally a dense cacophony of sound, graphics and choreographic gestures ensues. Figure 44. Section 3 sensor triggering The dance is one of spontaneous and explosive jaggedness. With a repertoire of six motivic gestures, the dancer can improvise between them, making use of juxtaposed stillness between fragments. Like the silences between isolated sound glitches, the choreography uses the minute spaces between events in its own kind of tensegrity. The effect is one of frenetic isolated energy, as both ear and eye are saturated in spasmodic stimulus. The cacophony, all of a sudden, dissipates and a rhythmic grid of glitches is triggered, while the dance takes on a rhythmic looping of gestures. 105 Figure 45. Section 3, Stage 2: acceleration of FM events, towards cacophony Stage three begins by turning off the previous FM triggering engines and sets up a regular series of FM events, first as sporadic random bursts, before a metro resets the [random] object’s threshold to 500 milliseconds, causing a more regulated rhythmic groove. 106 Figure 46. Stage 3: The Rhythmic Groove The choreography progresses through a maze of concatenated movements before a pitched ostinato enters, scrolling through five presets of MIDI-pitched phrases. These phases were taken from the ostinato section in my 2nd Piano Concerto. Below is the excerpt, before the ostinato breaks up through development: Figure 47. In the score, the celesta starts the ostinato. 107 Figure 48. The ostinato modified, as it changes, a pitch at a time. Figure 49. The final pitch configuration The conclusion of the section is announced when the erase_color attribute for the renderer is set to [0 0 0 0.01]. This last value refers to the alpha channel, which results in each frame being left to leak into the new one. The effect is a gigantic accumulating smudge across the whole screen. Simultaneously, the climax taken from Section 1 returns, in one final explosion. 6.5 Section 4 – Metamorphosis to Individuality Bloom’s fourth idea looks to a third opinion, both removed from the past and from the protagonist; though this third party is also the artist himself. This third self looks at the original as a reaction to an earlier work, and as another piece of artistic tradition. By viewing the similarities between the two, the original suddenly loses some of the uniqueness originally perceived and as a result, the new in progress gains a newfound individuality. 108 The section begins immediately after the climax of Section 3 clears. A quiet steely texture in the sound enters along with spindles of repeating geometric lines. The sound is produced by a favorite technique of mine I have used in many pieces. A host of modulating sine tones undulates, as result of the subtle fluctuations of each sensor axis. These sine tone harmonies are the product of a real-time analysis of Debussy’s Prelude a L'apres-Midi D'un Faune. This piece has influenced my work ever since I can remember, in all sorts of ways. The dance in this section was influenced by the repetitive patterns occurring in the OpenGl graphics. During this section it is the screen that informs the other components: it is mirrored by the choreography, which, in turn, influences the sine tone harmony. The structure of the section is stationary. The gentle throbbing of the sparkling sound and the precise translations of vertex positions in the graphics does not change over time. The effect is one of inner meditation, or intuitive thought, where the dancer explores his relationship with the screen. Body projection was explored extensively during rehearsals. And various points on the stage floor were marked so that the dancer knew throughout the whole section where the placement of his body would prove most effective. The dancer also tries to keep close to the screen, where again the Greek Tableau effect is explored; as if the scene is made up of photographic stills from the amalgamation of overlaid projection and body shapes. 109 Figure 50. The Greek Tableau concept is explored 6.6 Section 5 – Isolation, Failure and Fruition Section 5 and 6 use fixed audio and fixed graphics, without any interactivity. Bloom’s fifth revisionary ratio proposes to separate and isolate oneself from all other people and influences. As the person becomes separated, so too does his work, minimizing the similarities to potential influences. As a result of my recent readings, I was loosely inspired by Brian Greene's book The Elegant Universe (Greene, 1999). A sudden change in sound and image, instigated again by a pause in movement, announces the beginning of the 5th section. Similar gestures to the bridge are recalled in the choreography, where the low writhing position of the torso becomes the main focus. The dance explores further the stage floor, in the impression of an unformed state. 110 Figure 51. Beginning of Section 5 Various sound gestures taken from Section 1 return and are extended. These moments combine and grow, as the whole structure builds gradually. The graphics take on a subsidiary role here. One single three-dimensional model, the fire rock taken from the Section 1 render, is fractured and scaled across the whole projection surface. A shifting movie texture is mapped to the model, as the model’s various vertices and planes shift slowly. 6.7 Section 6 – Future, Graduation & Recreation For the last method, Bloom suggests direct acknowledgement of past influences within the artist’s own work. This method allows for a role reversal whereby the new now informs the old. Again, this section has fixed sound, graphics and choreography. It also flows straight on from the end of the previous section. This time one three-dimensional model is placed on each screen, at equal distances apart. 111 Throughout the whole section, each model morphs over four beats to a new model every two bars. These fractured models present a constantly changing texture, of sorts. The choice of colors contrasts to previous sections: vivid neon blues and other saturated colors make way for strong contrasts in shades of black and white, with a tinge of blue laced through. In previous sections, saturated colors produced interesting effects on the dancer’s suit, and in contrast, this section uses less color information to create a kind of plane depth, or platform leveling within the grey spectrum: stark binary coloring replaces luscious interpolated gradients. Figure 52. The morphing models in Section 6 The repetitious soundtrack accumulates impetus, as the progression to each new level grows out of the previous one. Old materials are replaced by newer ones, that seem to grow from out of the past, giving a sense of expectancy, conflict and suspense. All elements reach towards the last minute. The dance progresses towards the center stage, until the dancer is flush against the screen. The dancer increasingly juxtaposes choreographic motives faster and faster until this final stance is reached, again in a pose implying Greek Tableau. The piece ends with this frozen 112 statue and frozen graphics, combined as one, as the reverberation from the final chord fades out. The physical body of the dancer disappears and becomes part of the photographic image, held for what seems like forever. The piece ends expectantly with this final snapshot. I indicate for the house lights to rise, which coincides with the audience’s applause. 6.8 Self Critique Aside from a couple of hitches, the performance went well. The performance was also very useful to identify any instances where the piece could be improved. Firstly, for further performances it would be useful to take advantage of the ability for Max 6 to distribute processing across all available CPUs. My G5 tower is a quad core i5 architecture, so multithreading would facilitate enormous performance improvements. In order to implement this, I would need to restructure the whole patch, so that audio modules were separate first level patchers. This would mean each core would be designated to each new first level patch that is open. This system would also require the use of a [onecopy] object to be included in each of these patchers and the main patch would need to automate the use of [pcontrol] to open and close these patchers on the fly. I would also use the [net.maxhole] object, with the Max 6 program duplicated, so that GPU and CPU processing can be distributed across two instances within the operating system. I also think the relationship of the performer to interactivity needs be closer identified. Though it is important to avoid any clear mimicry through a one-to-one relationship, there needs to be less subtle mappings so that the interactivity is sometimes very clear: this in turn I think would give the audience a more satisfying identification of the relationship between performer 113 and technology. I would also like to revisit the choreography, since by the end of the rehearsal period, the dance seemed a lot more averaged out, less sharp, organic and distinctive. The sensor suit still needs work too. It is my plan to send my schematic to a commercial company that prints PCB. This new board would be redesigned to make the use of surface mount components, which not only makes the whole circuitry less prone to failure but also means the board and its housing can be a third of the size. The issue of sweat needs addressing further as well. The fabric needs be washable, which would mean the wiring and circuitry become detachable. My plan is to use Sparkfun’s conductible fabrickit Snap Brick32 studs to connect and detach sensors and wires from the fabric. Each accelerometer, connected to three female studs would be sewn into an elastic band. And this band would be connected to the suit’s sleeve, where three male studs are sewn in. That way, the sensors can be removed ready for the washing machine. From these studs, Also, to avoid having any cabling at all, individual wires that were attached to these studs would be sewn through the fabric, all the way to the PCB enclosure (also studded. With these points in mind, I have also thought about providing two versions of the audiovision sections (Section 1 and the Bridge), where one version is half the length of the other one. This would mean the piece could be included into programs where time restraints would otherwise prevent the piece from being accepted into the program. Another way to make the piece more accessible would be to add a preset for changing the number of discrete audio channels. This could also be applied to the size of the projection surface, if only one screen of different dimensions is available. 32 This URL shows the female stud: http://www.sparkfun.com/products/10298 114 A last note concerning the suit looks at its potential for other kinds of pieces/performances. I would also like to have any kind of dancer to be able to use the suit, for a variety of new uses. I also have plans to create a new piece, very similar in structure, that uses the suit. This piece, named Black Orpheus, will explore a contrasting color, textural, gestural and thematic pallet; thus demonstrating how the suit can yield to all sorts of aesthetical pathways. 115 7.0 Conclusion 7.1 On the Road to Becoming Cognition is the manipulation of symbols after the fashion of digital computers in that cognition is a mental representation: the mind is thought to operate by manipulating symbols that represent features of the world or represent the world in a certain way (Varela, Thompson, Rosch, p. 8). Essentially, art is about the manipulation of symbols, as we see and represent the world around us. Representation is much like symbolic connotation and the traditional notion of representation as a “veil of ideas” implies that no two renditions are true or false to any universal meaning. The dialog between these two poles is perfectly described by Richard Bernstien concept of “the Cartesian anxiety”, which proposes that anxiety, in a Freudian sense, can be understood as a friction between a dual purpose where on one side, human knowledge is based upon a fixed and stable foundation, and on the other side, we cannot remove our minds from an unformed, illogical chaos and confusion (Varela, Thompson, Rosch, p. 138). These polar opposites are explored, as they relate to Bloom’s anxiety, in Descartes and Kant: the edifice for a foundation for knowledge lies in the ability to categorize symbolic representation. And the very nature of this anxiety functions like a pendulum, balancing fluctuations between the two extremes. It is this grasp for a stable and grounded knowledge of the world that is the essence of the ego-self - and is also the source of continuous frustration (Varela, Thompson, Rosch, p. 143). Yet it is this frustration, as we have seen in accordance with Meyer and Bloom, which can be the source of creativity and the mechanism to induce affect. So in short, Anxiety has been about confronting this frustration, as a means of inducing affect. Using the human body to do this was to me a way to explore answers to this question in 116 the most quintessential form. A form that culminates in the dancer’s body, as it is well versed in the idiosyncrasies of the human anatomy, while it navigates its own elegant flow through space and time. Through the dancer’s body it is possible to search for a way of navigating the pendulum above. How do subtle movements of the left foot relate to the left palm? Or right foot to right index finger? How does the center of gravitational balance relate to all four extremities? Are there ways to explain how human muscle and human creativity meet, by tracking the movement of these body parts and reducing their differences down to averages and summations? These are the new tools towards exploring the type of internal journey discussed. To be honest, the hardest task I found, throughout the journey of creating this piece, was the difficulty in finding common ground between my rigid compositional upbringing and free improvisational nuance of the performable ephemeral moment. My dilemma has been one that questioned mechanisms for how humans and computers can interact; in a non-improvisatory and more determined way, yet still allowing the freedom and spontaneity for the dancer/performer to create a unique interpretation. It was my final goal to find a mechanism allowing a marriage between sensor modifications and fixed media playback. As a result, this piece has been a challenging and often frustrating journey for me. This journey was fraught with perils and hidden pathways; pathways that not only helped me refine the sensor suit and how it was to be used, but also revealed new ways for me to create. 117 7.2 Critique of Premiere Performance In performance, something went terribly wrong with the patch (don’t you love technology!) from Section 1 going into the Bridge. Somehow Section 2 opened at the same time as the Bridge, causing the CPU and GPU allocations to fall apart. It also meant that the triggering system for Section 2 was also opened, causing erratic repercussions from sensor data being routed around incorrectly. Thus, the bridge was a lot shorter in length than intended. This will be addressed when the patch is restructured in the ways mentioned previously. One pertinent criticism I received was that the three projectors seemed not to be aligned flush with each other. I agreed this was distracting, since on one side two projectors overlapped a couple of inches, while on the other side there was a gap between projectors, of about two inches. Some folks also thought the proportions of the various sections were a little unbalanced. The general feeling was that the first audiovision, where the dancer was not present on stage, was too long. This first section actually took up almost half of the whole, as it turned out. But this might have not proved so obvious if the bridge hadn’t failed and thus had to be cut short. 7.3 The Future for Anxiety of Influence and Sculpt As mentioned above, it is my hope to create one last version of the sensor suit, this time using the following configuration, in the hope of making it sturdy long-term, so that any dancer can slip it on easily without it breaking:  A printed surface mount PCB (which will mean the board will be considerably reduced) 118  The individual wires connecting the various sensors around the body going into the board be individually sewn into the shirt and pants, rather than wrapping the thicker cables around the body before putting the clothing over the top  A PIC24 series microcontroller will be used, to gain a faster and more stable acquisition time for multiple streams of analog data.  A Xbee wireless model by used instead of Bluetooth, since it has been widely proven this radio-wave method for wireless data transmission is much more sturdy and reliable. I have already began organizing performances of the piece in Boston and Providence and possibly New York. I had a great learning experience from this first venture and look forward to the next showings. After I develop the final version of the suit, I would also like to possibly work with more than one dancer, to create other pieces using the body-mounted suit(s). With this final version of the suit, my plan is also to connect with other practitioners for future collaborations. My techniques for using the data once it has been collected into the computer is also a new way for creating real-time graphics and sound, and it is my hope that many dancers and dance companies might be interested in using the sensor suit, in an educational setting, or one the creation of new pieces for dance and technology. 119 References Abbott, E. A. and I. Stewart (2002). The annotated flatland : a romance of many dimensions. Cambridge, Mass., Perseus Pub. Albers, J. (2006). Interaction of color. New Haven [Conn.], Yale University Press. Anker, S. and D. Nelkin (2004). The molecular gaze : art in the genetic age. Cold Spring Harbor, N.Y., Cold Spring Harbor Laboratory Press. Arnheim, R. (1974). Art and visual perception : a psychology of the creative eye. Berkeley, University of California Press. Artaud, A. (1958). The theater and its double. New York,, Grove Press. Bachelard, G. and M. Jolas (1994). The poetics of space. Boston, Beacon Press. Bishop, C. (2005). Installation art : a critical history. London, UK, Tate Publishing. Bloom, H. (1997). The anxiety of influence : a theory of poetry. New York, Oxford University Press. Bolter, J. D. and D. Gromala (2003). Windows and mirrors : interaction design, digital art, and the myth of transparency. Cambridge, Mass., MIT Press. Bolter, J. D. and R. A. Grusin (1999). Remediation : understanding new media. Cambridge, Mass., MIT Press. Braitenberg, V. (1984). Vehicles, experiments in synthetic psychology. Cambridge, Mass., MIT Press. Burnett, R. (2004). How images think. Cambridge, Mass., MIT Press. Carter, R. and C. D. Frith (2010). Mapping the mind. Berkeley, University of California Press. Chapple, F. (2006). "Intermediality in Theatre and Performance." Chion, M., C. Gorbman, et al. (1994). Audio-vision : sound on screen. New York, Columbia University Press. Crary, J. (1990). Techniques of the observer : on vision and modernity in the nineteenth century. Cambridge, Mass., MIT Press. Dawes "Anolog in, Digital Out." 120 Deleuze, G. (1986). Cinema. Minneapolis, University of Minnesota. Dewey, J. (1959). Art as experience. New York,, Capricorn Books. Dixon, S. (2007). Digital performance : a history of new media in theater, dance, performance art, and installation. Cambridge, Mass., MIT Press. Dmytryk, E. (1984). On film editing : an introduction to the art of film construction. Boston ; London, Focal Press. Doane, M. A. (2002). The emergence of cinematic time : modernity, contingency, the archive. Cambridge, Mass., Harvard University Press. Dobbert, T. (2005). Matchmoving : the invisible art of camera tracking. San Francisco, Calif. ; London, SYBEX. Evens, A. (2005). Sound ideas : music, machines, and experience. Minneapolis, University of Minnesota Press. Farley, Kathryn. (2004). Digital Dance Theatre: The Marriage of Computers, Choreography and Techno/Human Reactivity. http://ivizlab.sfu.ca/arya/Papers/Dance/Digital%20Dance%20Theatre.pdf Varela, Francisco J. and Thompson, Evan T. and Rosch, Eleanor. (1992). The Embodied Mind: Cognitive Science and Human Experience. The MIT Press. Fishwick, P. A. (2006). Aesthetic computing. Cambridge, Mass., MIT Press.