SIMON AND THE ANIMATING INTERFACE By Peter Bussigel B.Mus. University of Michigan, 2004 M.A. New York University, 2006 M.A. Brown University, 2010 Dissertation Submitted in partial fulfillment of the requirements for the Degree of Doctor of Philosophy in the Department of Music at Brown University PROVIDENCE, RI MAY 2014 © Copyright 2014 by Peter Bussigel This dissertation by Peter Bussigel is accepted in its present form by the Department of Music as satisfying the dissertation requirement for the degree of Doctor of Philosophy. Date: Joseph Butch Rovan, Advisor Recommended to the Graduate Council Date: Todd Winkler, Reader Date: Michael Theodore, Reader Date: Ed Osborn, Reader Approved by the Graduate Council Date: Peter M. Weber, Dean of the Graduate School iii CURRICULUM VITAE EDUCATION 2014 Ph.D. Multimedia & Electronic Music Experiments, Brown University, Providence 2010 M.A. Multimedia & Electronic Music Experiments, Brown University, Providence 2006 M.A. Sound and Video Art, New York University, NY 2004 B.M. Music Composition, University of Michigan, Ann Arbor, MI TEACHING 2014 Systems for Play Teaching Fellow, Brown University 2013 Computers and Music Lecturer, Department of Music, Brown University 2012 Recording Studio as a Compositional Tool Teaching Assistant, Department of Music, Brown University 2011 Sound, Media & Urban Space Adjunct Professor, Department of Digital+Media, Rhode Island School of Design 2011 Fluxus and Intermedia Ensemble Director, Brown University 2011 Narrative and Immersion Teaching Assistant, Brown University (Todd Winkler, Instructor) 2010 Real-Time Systems Teaching Assistant, Brown University (Joseph Rovan, Instructor) 2010 Listening In, Looking Out Teaching Associate, Bates University, Maine 2009 Computers and Music Teaching Assistant, Brown University (Todd Winkler, Instructor) 2006 Music Theory and Counterpoint Graduate Assistant, New York University HONORS 2013 IDAF Prize (Improvisation and Digital Arts Festival), Kingston U., England 2012 Creative Arts Council Grant, Brown University, Providence, RI 2011 Seamus Conference Performance, Miami, FL 2010 Freeman Foundation Grant, Bates College, Maine 2010 Invited Teaching Associate for the academic year, Bates College, Maine 2009 Kobushiko Commissioning Grant, Sendai, Japan 2009 International Computer Music Conference Performance, McGill University, Mtl. 2006 Associate Artist Residency, Atlantic Center for the Arts 2005 Orchestral Commission from the Newburgh Symphony Orchestra 2004 Platsis Award, Syren for Orchestra, University of Michigan iv LECTURES AND PRESENTATIONS 2013 Simon: Systems for Play, Improvisation and Digital Arts Festival, Kingston, UK reconsidering the pattern in light of the system 2013 Pattern Play, Triangles Lecture Series, Providence, RI designing animating interfaces for playing with sound 2012 Spooky Action, Brown University, Providence, RI entangling music and technology 2012 Sound & Image, Bates College, Lewiston, Maine experimental techniques for sound and image composition 2010 Intermedia, Bates College, Lewiston, Maine Dick Higgins, Fluxus and hybrid froms 2009 Sound as Cinema, Brown University, Providence, RI acousmatic music and animating space 2007 Portable Sound, New York University, NY the instruments in our pocket SELECTED PERFORMANCES AND EXHIBITIONS 2013 Pattern Play | ndial performance at IDAF 2013, Kingston UK 2013 One Room | performance installation for the RISD museum, Providence RI 2012 Reconsidering the Pattern in light of the System| Thesis Performance, Providence RI 2012 Loglo | for flute, bass clarinet, cello, piano and tape, Providence, RI 2012 ~90| Screening at Sound Off, New York NY 2011 A440 | Screening at the Seamus Conference, Miami FL 2010 Simon | Masters Thesis Performance, Providence RI 2009 Looking Out, Listening In | collaboration with Hiroya Miura | Japan 2009 Exit | International Computer Music Conference Performance, Montreal 2009 Adding Machine | Performed by Ulrich Maiss, Providence 2009 A440 | Screening at the Tank, New York NY 2008 Oroboros | Screening at Pixilerations, Providence RI 2007 Tin Rain | Performance at the Frederick Loewe Theatre, New York NY 2006 Bannerman’s Castle | Newburgh Symphony Orchestra, Newburgh NY 2004 Syren for Orchestra | Michigan Symphony Orchestra, Ann Arbor MI 2004 Traveling Circles | for the Whitney String Quartet, Ann Arbor MI v PREFACE This project is divided in two parts, each with four sections. Part I navigates a clutter of references, figures and ideas that combined to produce a system for playing with sound called the ndial. It begins with Figuring In, a method used throughout the project in which ideas are mapped to geometric figures and explored spatially. The second section introduces Simon, who serves as a guide in navigating the fuzzy boundaries between people and technology. The third section explores the ndial, whose operations and processes are explained in medium detail. The Lyrebird examines four music technologies that are entangled with the ndial, focusing on the feedback between listening and composing. Part II explores the ndial through the lens of live performance. The focus is on Reconsidering the Pattern, a performance game for four ndial players entangled in a system of light and sound. Section two, nterplay, presents a system for thinking about the relationships between player, program, sound, and listener within a live context, leading to Animating Interfaces—a perspective on digital interfacing that attends to the effects as well as the artifacts of playing with digital systems—bringing programs to life. The last section, Figuring Out, expands outward to think generally about our interactions with digital devices and proposes animating interfaces as both expressive agents and playgrounds for exploring the systems with and within which we live. vi This project has been shaped by conversations and adventures with many wonderful people. They have triggered samples, shuffled patterns and patiently slogged through the many abstract figures and technical details from which these words and experiments have emerged. The ndial would not exist if it weren't for the generosity and expertise of my advisor, Butch Rovan. Todd Winkler, Michael Theodore, Jim Moses, John Cayley, Gertrud Koch, and Ed Osborn have been infinitely generous with their time and minds. Reconsidering the Pattern relied on the advice, energy, and labor of a special group of collaborators/friends. There is not enough space to properly thank Micaela Morrissette for her words, or Peter Scheidt for his sculptures. I have been so lucky to perform with Stephan Moore, Kimberly Young, Caroline Park, and Timothy Rovinelli. Emma Cunningham, Nicole Halmi, Tim Syme, Andrew Starner, Andrea Actis, Antoine Traisnel, Hans Vermy, and Hunter Hargraves have all been part of this project from the beginning and I have been fortunate to work alongside two of my favorite instrument designers, Chris Novello and Jordan Bartee. I am grateful to Chira Delsesto, Greg Picard, Shawn Tavares, and the Granoff Center for generously supporting my work these past few years. Jacob Richman, Asha Tamarisa, Brian House, Bevin Kelley, Mark Cetilia, and the rest of the MEME community have been a source of great inspiration throughout my time here at Brown. There would be no project without Coleman Nye and I could spend another full page trying to thank her. My mother and sister are unbelievably understanding and supportive and this project is dedicated to them. Thanks to my dad who taught me to play. vii DOCUMENTATION Reconsidering the Pattern http://bussigel.com/pb/projects/re/ ndial http://bussigel.com/pb/projects/ndial/ viii TABLE OF CONTENTS Map ...................................................................................................... x Figures ..................................................................................................... xi .................................................................................................................................. Figuring In .......................................................................................... 1 Simon ...................................................................................................... 12 ndial ......................................................................................................... 33 Lyrebird ................................................................................................. 50 ............................................................. n ............................................................. Reconsidering the Pattern ........................................................... 68 nterplay .................................................................................................. 98 Animating Interfaces ..................................................................... 103 Figuring Out {Systems for Play} ............................................ 119 .................................................................................................................................. Appendix ............................................................................................. 123 Notes ...................................................................................................... 129 Bibliography ......................................................................................... 133 ix Figure 0: Project Map x LIST OF FIGURES Figure 1 : Eyes, Hands, Ears.......................................................................................................................... 2   Figure 2 : A Polymodal Lens ......................................................................................................................... 2   Figure 3 : n = navigation ................................................................................................................................ 3   Figure 4 : Outside | Inside ............................................................................................................................. 4   Figure 5 : Human ~ World ............................................................................................................................ 5   Figure 6 : Composing, Listening, Technology .............................................................................................. 6   Figure 7 : Overview – ndial & Reconsidering the Pattern ............................................................................. 8   Figure 8 : Sampled Figures { I } ..................................................................................................................... 9   Figure 9: Controls ........................................................................................................................................ 10   Figure 10: Colors ......................................................................................................................................... 11   Figure 11 : A Cover Story ............................................................................................................................ 13   Figure 12 : Simon ........................................................................................................................................ 15   Figure 13 : You See Yourself ...................................................................................................................... 16   Figure 14 : Familiar Face ............................................................................................................................. 17   Figure 15 : Final Scene ................................................................................................................................ 18   Figure 16 : Poster ......................................................................................................................................... 18   Figure 17 : Transcendence ........................................................................................................................... 19   Figure 18 : Studio 54 ................................................................................................................................... 20   Figure 19 : Simon Controls ......................................................................................................................... 21   Figure 20 : The Inside ................................................................................................................................. 22   Figure 21 : Mions ........................................................................................................................................ 25   Figure 22 : Touch-Me ................................................................................................................................. 26   Figure 23 : July 28th 1976 ............................................................................................................................. 27   Figure 24 : Reanimation .............................................................................................................................. 28   Figure 25 : ndial ........................................................................................................................................... 32   Figure 26: ndial manual ............................................................................................................................... 33   Figure 27: ndial diagram .............................................................................................................................. 34   Figure 28: ndial controls .............................................................................................................................. 35   Figure 29: Shuffle ........................................................................................................................................ 36   Figure 30: Face ............................................................................................................................................ 37   Figure 31: Right ........................................................................................................................................... 40   Figure 32: Left ............................................................................................................................................. 41   Figure 33: Front ........................................................................................................................................... 42   Figure 34: EQ .............................................................................................................................................. 42   xi Figure 35: Back ............................................................................................................................................ 43   Figure 36 : Onism ........................................................................................................................................ 44   Figure 37 : Onist Perspective ....................................................................................................................... 45   Figure 38 : Key Mode .................................................................................................................................. 45   Figure 40: Worlds ........................................................................................................................................ 47   Figure 41 : Inside ......................................................................................................................................... 48   Figure 42: Components ............................................................................................................................... 49   Figure 43: UBW32 PIC32MX795 IC & LPY503AL ................................................................................ 49   Figure 44 : Technical Processes ................................................................................................................... 50   Figure 45 : The Moog 960 Sequential Controller ....................................................................................... 59   Figure 46: Map Inside ................................................................................................................................. 67   Figure 47: Initial .......................................................................................................................................... 68   Figure 48 : Title ........................................................................................................................................... 69   Figure 49 : Simon ........................................................................................................................................ 72   Figure 50 : A Familiar Face ......................................................................................................................... 72   Figure 51 : Performance ............................................................................................................................... 74   Figure 52: Translation ................................................................................................................................. 76   Figure 53 : The Magic Circle ...................................................................................................................... 79   Figure 54 : Entrance Animation .................................................................................................................. 79   Figure 55 : ● ............................................................................................................................................... 81   Figure 56 : ● ............................................................................................................................................... 82   Figure 57 : ● ............................................................................................................................................... 82   Figure 58 : ● ............................................................................................................................................... 83   Figure 59 : Reconsidering ............................................................................................................................ 84   Figure 60 : The Pattern................................................................................................................................ 85   Figure 61 : Models ....................................................................................................................................... 87   Figure 62 : Frames ....................................................................................................................................... 89   Figure 63 : Light .......................................................................................................................................... 90   Figure 64 : Low Resolution Screen .............................................................................................................. 91   Figure 65 : Entangled .................................................................................................................................. 92   Figure 66 : Tetrahedra ................................................................................................................................. 92   Figure 67 : bumpr ........................................................................................................................................ 93   Figure 69: nterplay ....................................................................................................................................... 98   Figure 70 : nterplay 2D .............................................................................................................................. 101   Figure 71 : nterplay 3D .............................................................................................................................. 101   Figure 72 : system ystems emstys metyss ................................................................................................... 102   Figure 73: Animating Interfaces ................................................................................................................ 103   Figure 74 : Animating Interface................................................................................................................. 109   xii FIGURING IN The Institute For Figuring is an organization dedicated to the poetic and aesthetic dimensions of science, mathematics and engineering... At the core of the IFF’s work is the concept of material play. We believe that ideas usually presented in abstract terms can often be embodied in physical activities that engage audiences via kindergarten-like practices. Through activities such as cutting and folding paper, we affirm that the hands and eyes can serve as guides to developing the human mind. By inviting our audience to literally play with ideas, the IFF offers a new, hands-on approach to public science education that is at once intellectually rigorous, pedagogically rich, and aesthetically aware. : The Institute For Figuring1 I propose that in addition to hands and eyes, we also consider ears as guides... n. figure, number, quantity, amount, level, total, data, information, cost, value, computation, frame, body, shape, form, outline, personage, individual, character, personality, representative, embodiment, personification, epitome, effigy, model, statue, pattern, design, motif, illustration v. to figure, appear, feature, refer to, work out, total, reckon, compute, determine, assess, crunch the numbers, tot up, think, believe, consider, expect, suspect, sense, assume, dare say, conclude, take it as read, presume, deduce, infer, extrapolate, gather, guess, stand to reason, follow, ring true 1 Figure 1 : Eyes, Hands, Ears This project uses figuring and figures to help navigate connections between a set of records and ideas. Here, figuring is understood as a mode of engagement (like perceiving or receiving) that is especially helpful when dealing with multiple variables. Figures activate our ability to reason spatially, extending our perceptual apparatus by representing complex systems as simple spatial arrangements. Figuring does not replace perceiving or recieving, but works in consort with other modes of engagement. By mapping receiving { r }, figuring { f }, and perceiving { p } to the triangle we form a polymodal lens through which these pages can be navigated. By moving between the three vertices we explore matters from different perspectives. Figure 2 : A Polymodal Lens We can think of our engagement with sound in similar terms, triangulating between hearing { r }, listening { p }, and relating { f }. Each colored vertex represents a space of pure monomodal engagement, but in practice these ideas are always interrelated—listening reframes hearing reframes relating reframes listening and so on. Figures are conceptual playgrounds for intuitively exploring complex systems by mapping variables to simple forms. 2 To abstract is to construct a plane upon which otherwise different and unrelated matters may be brought into many possible relations. It is through the abstract that the virtual is identified, produced and released. The virtual is not just the potential latent in matters, it is the potential of potential.2 : McKenzie Wark, A Hacker Manifesto We might also attempt to figure ourselves in as navigators, enactive variables moving in relation to these three modes of engagement. In this context, we are navigating a set of ideas printed on pages. Navigating words suggests a perceptive mode that is tuned for transmitting messages through sequential combinations of predefined codes. Figures, used in conjunction with words, can be helpful in exploring how these messages relate and change over time, exploring connections between matters in non-linear and perhaps, less determined ways. Navigation { n } is our movement through a figurative space and can be depicted on the page as a series of frames or states. By interpolating between frames and extrapolating beyond these frames, we are able to figure within these forms, to think about our relationship to whatever world we might be sailing through. Figure 3 : n = navigation 3 For example, here we move out away from our figure to think for a moment about how we experience things. From here we either observe a square or tetrahedral volume for thinking about our relationship with this page specifically, sound more generally, and a polymodal lens for engaging with the world. And if we were, as we are, navigating within these figures, the triangular system would begin to animate around us. Figure 4 : Outside | Inside Figuring is a way of moving between the inside and the outside by using models that we are simultaneously looking at and living within. That which animates depends on where one is in relation to the system. From the outside, it is you { n } that is animating, but inside the system, the figure animates around you. Ok, but why? We can deal with problems in which there are very few variables. Or we can deal with problems in which there are almost infinitely many variables. But in between we’re pretty helpless. The average person can not think through a problem involving more than 3 variables without a pencil in their hand. Most problems which we deal with involve far more than three variables, so we’re incapable of thinking about them. Actually, the way we think about most of our problems is simply going through the motions of thinking. 3 : Alan Watts, Transcribed from talk 4 It is difficult to think about complicated problems without using some method or technology for representing them. At the very least, figuring takes us through the motions of thinking. I have a horrible memory and am quick to call on figures, even to explore relatively simple interactions. I have come to see the process of navigating figures in time as animation—both the technical process in which static frames resolve as motion and the act of bringing to life. After words have captured ideas, and pinned them to the page, figures describe systems for reanimating them. Figuring is a form of abstraction. We are never just listening or even just listening, hearing and relating. We are also sitting and scrolling, sometimes we are drinking coffee, sleeping, walking, watching and reading. While figuring is helpful in dealing with complicated problems, it is also a filter that reduces the noise of complex systems to allow us to focus on a particular aspect of the interplay. this here is the trap of the world: on the one hand, our ability for abstraction allows for ease in traveling dimensions of thought, dimensions so vast that we can now transcend space-time in radically amazing ways; and yet, on the other hand, abstraction is a terrible curse, a curse which can lend itself to the perpetual undermining of the potentials of our wonderfully noisy collective. It’s a trap because we oscillate between these two states. There is no path but the path between two extremes bouncing back and forth. the trap is a gift, but the trap is also, still, a trap. 4 : Sampliciter Figure 5 : Human ~ World 5 Important. Figuring is bi-directional. In reducing the world to a system of relationships, we simultaneously adopt the figure as a lens through which we make the world. Just as figures can be used as systems for exploring the world, figures can be used by the world for exploring us. By mapping materials (modes, ideas, sounds...) arbitrarily to figures such that the world becomes unpredictable we must actively navigate relations between seemingly unrelated matters. It is likely that this is not the most efficient method of solving any particular problem, but as I will attempt to show, in some cases, efficiency is not the goal. Figure 6 : Composing, Listening, Technology In describing the writing of Gertrude Stein, Judy Grahn argues that one must “insterstand... to mix with [the work] in an active engagement, rather than 'figuring it out.' Figure it in.”5 In the following pages, I adopt a similar notion of “figuring in” as a way of thinking about sound as it relates to listening, composing, and technology, figuring technology { t }, listening { l }, and composing { c } as always interrelated and observing how electrons have amplified and excited these relations. This figuring finds resonance with Jacque Attali’s view that, “One produces what technology makes possible, instead of creating the technology for what one wishes to produce.”6 6 Animating interfaces require enactive listening, where the player is simultaneously both listener and composer, actively exploring sound worlds like one might navigate a sailboat. Certain variables can be controlled, and others are unpredictable. By embracing complexity and ambiguity, by figuring in rather than figuring out, we might find ourselves in wonderfully unexpected places. But rather than limiting exploration to wonderful places—to any predetermined destinations— animating interfaces articulate playgrounds or possibility spaces for exploration. Player, program, and sound materials are entangled in the logic of a system. In such a space, productivity and efficiency are unnecessary, and the diffractive lens of figuring is foregrounded. Throughout this project, geometric figures are used as systems for figuring in, playing with relationships in which we ourselves are implicated. The goal of this methodology is not to uncover universal systems applicable to the world in some general way, but to navigate the patterns of a specific system and listen to the figuring that emerges. These opening pages are an attempt to introduce a method and a device used for navigating sound worlds by randomly sampling recordings and mapping the samples to variables on simple geometric figures. The method is called nterplay and the device is the ndial—an example of an animating interface. 7 Figure 7 : Overview – ndial & Reconsidering the Pattern The second half of this project explores a specific implementation of nterplay in a performance setting. Reconsidering the Pattern is a music-game-performance in which multiple ndial players navigate different sound worlds together. A description of the piece is intercut with reflections about the process that present some of the considerations and difficulties in bringing this type of open-ended sound play into traditional performance contexts. Drawing from game design, free improvisation, systems theory, and science fiction, Reconsidering the Pattern becomes a text for exploring how one might create public spaces for ‘figuring in’. The geometric figures that run throughout this text were assembled using systems not unlike the random yet obsessively geometric logic of nterplay. Existing systems are sampled and reassembled and new patterns emerge through playful conversation between a number of matters and disciplines. The tetrahedral diagrams that have already been introduced are themselves the artifacts of collisions within a far reaching constellation of figures. 8 Figure 8 : Sampled Figures { I } Each of these figures, above, presents a possibility space for relating ideas. Scott McCloud’s “Picture Plane” describes the area between reality, meaning, and abstraction explored in comics. A diagram by Stephan Willat’s models how artist, artwork, audience and context interrelate. For Buckminster Fuller, the tetrahedron is the ‘minimum structural system of the Universe’ and he used tetrahedral figures to model many types of systems. Thomas Hirschhorn’s “Spectre of Evaluation” depicts the absurd dance between art world, artist and audience, and Pauline Oliveros’s graphic score “Wind Horse” can be constructively applied to any creative navigation. But it is another 4-variable figure that has co-piloted this process since well before it began. Simon is an early animating interface, an ancestor to the ndial and the most popular stand-alone electronic game ever produced. The story of Simon is the science fiction from which this project samples; Simon paints our figures and points to a tension that runs throughout this volume. Simply put, as people play with systems, systems play with people. There is nothing to figure out, but we can try to figure in. 9 Controls Figure 9: Controls Trigger Change Switch Dial Shuffle Pattern Navigate Animate 10 Colors Recording Player Play Sequencing Listener Attention Sampling Sound Significance Programming Program Abstraction Figure 10: Colors 11 SIMON In 1263, Simon de Montfort overthrew King Henry the 3rd of England. He held only two parliaments before being killed in the Battle of Evesham just one year later, but history has been kind to de Montfort. His second parliament was open to common citizens and for this reason, he is considered a founder of parliamentary democracy. And while his rule over England was brief, a timeless children’s game still bares his name. During de Montfort’s short reign, all of King Henry’s orders were meaningless unless Simon said they stood. Such is the myth of Simon Says, a children’s game entangled in the power struggles of early democracy, caught between control and freedom. If “Simon says touch your toes”, you touch your toes. “Simon says flip the switch” and you flip the switch. Simon says “read these words”. You lose. Simon derives from the Hebrew “shim'on” meaning, “to hear” or “to be heard”.7 The Institute for Advanced Concepts brainwashed Simon and convinced him that he was an alien. While being brainwashed, Simon proposes a general theory of creativity, “I believe that inside of everybody is a genius waiting to be released. The secret is that you gotta get yourself into a kind of chaotic enough mental state and then the good material can break through from the unconscious.”8 The film, Simon, was released in 1980 and the main character, Simon, was played by Alan Arkin. 12 9 Figure 11 : A Cover Story10 In 1950, Radio-Electronics magazine ran a series of articles about Simon. About the size of a suitcase and costing $600 dollars, Simon could perform basic operations like adding, subtracting, and simple selection. Early computer scientist, Edmund Berkeley, first envisioned Simon in his 1949 book called, Giant Brains, or Machines That Think, in which he outlines the design of a “very simple machine that will think.” Berkeley lucidly describes Simon’s Flesh and Nerves, Simon’s Mentality, Simon’s Memory, The Control of Simon, Simon’s Thinking, and Simon’s Computing and Reasoning. He also advocates for these new mechanical brains... These new machines are important. They do the work of hundreds of human beings for the wages of a dozen. They are powerful instruments for obtaining knowledge. They apply in science, business, government, and other activities. They apply in reasoning and computing, and, the harder the problem, the more useful they are. Along with the release of atomic energy, they are one of the great achievements of the present century. No one can afford to be unaware of their significance.11 13 Simon could only conceive of 4 numbers—2 bits. While the room-sized ENIAC could perform over 5,000 operations per second, Simon clocked just over 1 operation per second. However, Simon could transfer information automatically and perform operations of indefinite length, and therefore, Simon is widely considered to be the first personal computer. Information was input by paper-tape, processed by 129 electromechanical relays and a stepping switch and displayed as light, turning on some combination of 4 bulbs to show not only the results of a problem, but also the current stage of computation. At the end of his 1950 Scientific American article, “Simple Simon”, Berkeley described how these brains might affect the future... Some day we may even have small computers in our homes, drawing their energy from electric-power lines like refrigerators or radios... They may recall facts for us that we would have trouble remembering. They may calculate accounts and income taxes. Schoolboys with homework may seek their help. They may even run through and list combinations of possibilities that we need to consider in making important decisions. We may find the future full of mechanical brains working about us.12 Important to Berkeley was the possibility that machines could help people learn, writing, “Simon has the same use in instruction as a set of simple chemical experiments has: to stimulate thinking and understanding, and to produce training and skill.” Berkeley also predicted that machines would become the fascination of hobbyists, who could build their own devices for function and wonder... Simon has two futures. In first place Simon can grow. With another chassis and some wiring and engineering, the machine will be able to compute decimally, Perhaps in six months more, we may be able to have it working on real problems. In the second place, Simon may start a fad of building baby mechanical brains, similar to the hobby of building crystal radio sets that swept the country in the 1920's.13 14 Figure 12 : Simon Simon was released in 1978 and quickly became the most popular game in the United States, selling over one million units well before Christmas. It would go on to become Milton Bradley’s top selling game of all time, bringing in tens of millions in annual revenue for the next decade. Simon became an 80s icon, remained popular through the 90s, and is still in production today, 35 years later. The story of Simon is entangled with the dawn of a digital age where smaller, faster computers radically changed both work and play. It inspired other sound and memory games—Merlin, Bob- It, Follow-Me, Space Echo, Copy Cat, and Loopz are all direct descendants—but its influence moved well beyond the realms cordoned off for play. Encoded in the flashing patterns of sound and light was a protocol. As children copied Simon, Simon copied children. A program, encoded in carbon lifeforms, spread throughout culture. Simon sets the pace. You follow right along. Light the lights that Simon lights or he’ll tell you that you’re wrong. Simon’s a computer. Simon has a brain. You either do what Simon says, or else go down the drain. Simon is a master. He tells you what to do. but you can master Simon if you follow every clue.14 15 Simon has a brain. He is named, gendered, and unpredictable. To master him, you follow his rules. He shames you when you lose and congratulates you if you win. He operates on different levels and when ignored, he reminds you to turn him off. He’s simple, but inside is a program that conjures both fascination and anxiety—the double-take of artificial life. Although Simon was cast to play the part of the robot adversary, his brightly colored plastic interface lacks the detail and surface complexity to resolve as the enemy. The low resolution, abstract gameplay is too simple and unpredictable. His memories aren’t specific and so they don’t resolve as the memories of another. Simon plays in a different uncanny valley, determined not by the realistic complexity of the surface but through the durational interplay of a system that appears almost thoughtful. In Simon, we find ourselves. From Understanding Comics, by Scott McCloud... Figure 13 : You See Yourself 15 16 Figure 14 : Familiar Face In casting Simon as the enemy, Milton Bradley was working from an out-of-date manual. Skepticism and paranoia dominated the technological discourses of the mid-60s and continued through most of the 70s, a time of social and political unrest that resonated more with a return to nature than trajectories of scientific and technical conquest. But as microprocessors allowed for complex digital circuits to be enclosed within familiar interfaces, electronic products became intuitive, practical and affordable. For children, Simon was more than just a toy. He was a friend, a touchable machine at a time when keyboards were off limits. While parents tried to communicate with their Apple II in symbols and codes, their small children were pushing giant glowing buttons, ‘repeating Simon’s flashing lights and sounds’. In escaping the productivity proposed by personal computers and focusing on tactile low-resolution patterns, children came to explore less predictable dimensions of the digital world. No longer just tools—computers became collaborators and teammates, then experiences unto themselves, not “wonderfully functional, but functionally wonderful—a merry- go-round of light, color, and music.”16 17 Figure 15 : Final Scene17 In the quote above, Vivian Sobchack describes the UFO from Spielberg’s “Close Encounters of the Third Kind”, released in 1977. The film played a large role in Simon’s success. The final scene features a team of scientists communicating with a giant, brightly illuminated spaceship through a sequence of synchronized lights and sounds. The interplay is similar in both behavior and design to Simon’s flashing tones and flying-saucer shaped appearance. A close encounter of the third kind, those in which the presence of animated creatures is reported… (I say “animated” rather than “animate” to keep open the possibility of robots or something other than “flesh and blood”)’ ~ Allen Hynek, The UFO Experience The nature of the set itself and what transpired within it was, from start to finish, veiled in top secrecy. Only those required for the filming were permitted entrance after displaying proper identification badges, checked by an around-the-clock security force. ~ Close Encounters Souvenir Book Figure 16 : Poster 18 In the film, a vacuum cleaner moves across a room on its own and an oven repeatedly cycles on and off. Cars, mechanical systems and electronic gadgets are brought to life by a mysterious power. Cameras, microphones, digital tape decks and other recording technologies make frequent cameos. Electronic gadgets are present in nearly every scene, referenced with both a paranoid and idolizing obsession that suggests the presence of a different kind of being. Are we looking for aliens or systems born of our own minds and hands? Are they real or do we make them real? Figure 17 : Transcendence [Close Encounters] initiates a new iconography of beatific human wonder, editorially linking affect to effect. Heads tilted, eyes gazing upward with childish openness and unfearful expectancy--this is the human face of transcendence whose emotion is enacted by what it sees.18 Vivian Sobchack, Screening Space Simon became the handheld controller for communicating with another life form, worshipped with cult-like fervor. With their mechanisms hidden, objects of our own design became life-like and magical. Animated devices, simultaneously depicted as both friend and foe, were worshipped with cult-like intensity. Both Close Encounters and Simon became iconographic agents of a nascent digital aesthetic, shaping popular perspectives that led to a new-wave of cybernetics that defined 80s culture—the pulsing, boldly-textured patterns of sound and light. 19 Figure 18 : Studio 54 Simon debuted in 1978 at Studio 54 in New York City. Descriptions from the party reference a 4 foot replica swinging above the dance floor, flashing colored light on the guests below. There was a console at every table, playing underneath the steady pulse of Kraftwerk’s Man-Machine, released 2 months earlier. The celebration was regularly punctuated by a harsh “RAZZ” as guests failed to keep up with Simon’s digital brain. There are show biz people galore: Baryshnikov and Alvin Ailey, cabaret artiste Lorna Luft and Neil Sedaka, tennis champ Vitas Gerulaitis, Geraldo Rivera, the Playmate of the Month (Dito: "We accepted her for what she was"). Disco Sally, who is not on the guest list, has been admitted because she is a Studio fixture, its resident senior citizen; she comes every night to boogie. "I can't tell whether she's a good dancer," Dito says to himself, "or whether it's Parkinson's disease." The rest of the guests, though— they are here because of George Ditomassi.19 : Diane McWhorter, Boston Magazine, 1978 “Dito”, the spokesperson for Milton Bradley, played host. He awarded Neil Simon “Simon of the Year” and gave away an all-expenses-paid trip to St. Simon Island. There were over a thousand in attendance and Simon was the guest of honor, squaring off with actors, sports-stars and game enthusiasts. In the most legendary club at the height of Disco, Simon became a star. 20 Figure 19 : Simon Controls20 Mode I : Player vs. Simon Simon generates a random sequence that the player must copy. If correct, Simon repeats the pattern and adds another tone to the end, and again, the player copies the sequence. Each turn, the pattern grows by 1, until a button is pressed out of order and the player loses or the player successfully repeats a 31 tone sequence, and wins. Mode II : Player vs. Player Simon begins with one tone, the first player must repeat the tone and select the next tone. Each subsequent player must repeat the current sequence and then add one tone. This continues until a player makes a mistake or the maximum sequence of 31 colors is successfully navigated. In this mode there is only a loser, no winners, Simon officiates. Mode III – Player vs. Player vs. Player vs. Player A multi-player version of Mode I. Each player is responsible for one or more colors during the sequence (up to 4 players). If a button is pressed out of order that color is removed from the game and play continues with a new sequence until only one color is left. This game has one winner and up to three losers (all human). Simon generates patterns and officiates. Levels change the maximum length of the sequence in all three modes. There are four levels. In the multi- player games, it usually makes sense to be on level 4, allowing for the longest possible gameplay. The maximum sequence lengths are as follows: Level 1: 8 Level 2: 14 Level 3: 20 Level 4: 31 21 Figure 20 : The Inside “It's incredible that Milton Bradley could have gotten hold of a microprocessor and come up with something that dumb." …”you have to make the computer dumb and slow," says an electronic game inventor, "for people to be able to play with it.”21 : Boston Magazine Simon’s brain is the TMS 1000 “computer on a chip”, the first ever microcontroller, a 4-bit single-chip CPU with 32 bytes of RAM developed by Texas Instruments in 1971 and commercialized in 1974. Microchip developments made computers practical and affordable and with the release of the Commodore 64 in 1982, digital computing entered the home. Small, cheap microcontrollers like the TMS 1000 changed the face of countless industries, and, outfitted with embeddable logic chips, the game industry went electric. A 1978 article in Boston Magazine, “Electronic Shock in Toyland”, outlines some of the new capabilities of digital games. Because of its ability to endow a game with memory, the computer has transformed their notion of what a game is: by providing random variables that dice cannot; by making play cheat-proof, since the computer neither lies nor can be lied to; by adding an "intelligent" opponent, which radically redefines the concept of solitaire games. 22 : Boston Magazine 22 The availability of microcontrollers was the driving force behind the industry’s shift to handheld digital systems, and, while video-games would quickly come to dominate the entertainment market, standalone electronic games were popular as well. By the late 80’s, digital systems— hand-held interfaces (Simon, Merlin), arcade games (Space Invaders, Pong), dedicated videogame consoles (Atari 2600), and the infinitely reprogrammable personal computer (Apple II, Commodore 64)—had become the primary sites of play, advertising not only new experiences but new modes of life (social networking, virtual reality, networked gaming). The promise of the digital extended well beyond play. With small, cheap, easily programmable chips and fast connection protocols, computers could sense, model and process the physical world in close to ‘real-time’. With the bit as a common denominator, information was to be free from material limits, and, sold on this vision, people increasingly turned their attention to the glow of the virtual—a techno-utopian optimism that recalls Richard Brautigan’s 1967 poem “All Watched Over by Machines of Loving Grace.”23 I like to think (and I like to think I like to think the sooner the better!) (right now, please!) (it has to be!) of a cybernetic meadow of a cybernetic forest of a cybernetic ecology where mammals and computers filled with pines and electronics where we are free of our labors live together in mutually where deer stroll peacefully and joined back to nature, programming harmony past computers returned to our mammal like pure water as if they were flowers brothers and sisters, touching clear sky. with spinning blossoms. and all watched over by machines of loving grace. The promise of a digital future radiated from both Silicon Valley and Hollywood, spreading with ever-increasing resolution—life free from identity, difference and body. But the digital was promoted and produced by people positioned to profit from its realization and a cast of mostly white, male tech-entrepreneurs with military contracts saw financial opportunity in digitizing life and built systems to control its currency, information. 23 As the promise of freedom was manufactured into increasingly directed systems and sold as productivity machines, the virtual came to reflect the real. ‘Digital freedom’ resembled a more efficient version of an all too human logic. Often overlooked in the spectacle of animated technologies, people catalyze the entangled influences that perpetually remake what both ‘human’ and ‘technology’ mean. People are specific, varied and noisy and only in their hands and minds do machines become more—extensions of life. Alan Perlis describes the computer program in the forward of Structure and Interpretation of Computer Programs, the basis of MIT's computer science courses since 1980: Every computer program is a model, hatched in the mind, of a real or mental process. These processes, arising from human experience and thought, are huge in number, intricate in detail, and at any time only partially understood. They are modeled to our permanent satisfaction rarely by our computer programs. Thus even though our programs are carefully handcrafted discrete collections of symbols, mosaics of interlocking functions, they continually evolve: we change them as our perception of the model deepens, enlarges, generalizes until the model ultimately attains a metastable place within still another model with which we struggle.24 : Alan J. Perlis, Structure and Interpretation of Computer Programs (Forward) Our technologies are processes abstracted from the human mind. From the Greek “technic”, a method or manner of accomplishing something, technology is not only a description of technical devices but a method for formalizing and systemizing human processes. By using our technical devices we develop new perspectives that help us build new technical devices and the concept ‘human’ is remade at an exponential pace. The rapid refiguring of both human and machine is perhaps what makes people seem different from other forms of life. But what are technologies used for, ultimately? Creating better worlds? Producing new patterns? Or are they simply something to do? Something to keep people productive. 24 New technology doubles as both savior and captor, a confusion illustrated in the lines outside Apple stores and the real blood on technology’s ‘bleeding edge’. It is the dualism at play in countless science fictions—a simultaneity and a genre that all technical projects move within. People make instruments that change the way people work and in working with them, people are changed. A cup lets its inventor live further from water but, living further from water, the inventor needs a cup. This feedback system is found throughout history—humans making systems making humans. In time and with use people have come to incorporate the silicon perspective as part of the carbon consciousness. The microcontroller, capable of performing aspects of human logic, works bi- directionally. As people copy systems, systems copy people. An iPhone is a perspective, just like a cup. When people are copies, they are easy to control. Figure 21 : Mions25 The history of Simon, like history more generally, contains confusion between original and copy. History says Simon is the child of Ralph Baer, an engineer and head of the new electronics division at top game design firm Marvin Glass and Associates. In 1977, while attending the Music Operators of America conference, Baer played a four-button memory game made by Atari called “Touch-Me.” He remembers the initial meeting… 25 Touch-Me was in a waist-high cabinet with four large, dark “buttons” facing the player on its top, nearly horizontal surface; during the game, the buttons lit up in random sequences and the machine issued truly awful, raucous accompanying sounds. It was the player’s job to follow the light sequence by pressing the appropriate buttons. Figure 22 : Touch-Me Working with programmer Lenny Cope and funded by Marvin Glass, Baer rebuilt Touch-Me in a portable enclosure with buttons arranged in a square and called it “Follow-Me”. The most notable updates were to the design and the button sounds. The arcade style console was replaced by a portable flying-saucer shape and, inspired by the tones of a bugle, Baer selected the pitches E, C#, A, & E (one octave above) to correspond with the colored buttons (Blue, Yellow, Red, Green respectively). The tones outlined a major chord in second inversion, a set that ‘musically works’ in any sequence. A deeper, chiding “Razz” sound was designed to indicate defeat. …soon we were ready for a demo to potential clients: Milton Bradley were the first to see the current incarnation of “Follow Me” at the Marvin Glass studio in Chicago. As usual, it was Mel Taft who came from Milton Bradley’s Massachusetts’ head-shed to view new products. What he saw at the time was a square unit, about 8x8 inches, which played like gangbusters. The illustration in the Simon patent still shows that configuration. The cover page of that patent is shown nearby. At the time, the game had acquired a new name: “Feedback”. 26 : Ralph Baer, Videogames: In the Beginning Touch me… Follow me… Feedback… Simon Milton Bradley decided to “go” with the game shortly after that demo; they renamed the game Simon, which made perfect sense.” 27 26 For Baer, electronic toys were a passion, but also a hobby. He worked as an engineer for Sanders & Associates developing defense technologies. There, Baer developed the Brown Box, widely considered the first videogame system. It was reworked as the Magnavox Odyssey and released a year before Atari’s Pong debuted with an identical gameplay. Baer sued Nolan Bushnell and Atari, but it was Bushnell who got both the patent and the coveted title of “Inventor of the Videogame”. Strangely, Baer received the patent for Simon, his knock-off of Bushnell’s Touch-Me. The early battles for the rights to play were taken seriously and Marvin Glass was notoriously strict about protecting the firm’s ideas. The office was double-walled and prototypes were locked in a large bank vault at the end of the work day. Two years before Simon’s release, this obsession with control led to tragedy, described in an edition of Newsweek from 1978… Toy companies have been involved in fraud, espionage, piracy and theft, and that can produce a lot of paranoia. At Chicago's Marvin Glass & Associates, probably the largest toy think tank in the nation, there are no windows and no lunches outside its building. Such intense secrecy may keep Glass at the top: it develops one of every ten games in the U.S., including this year's Simon. But the pressure can also warp minds. Two years ago, a quiet electronics engineer went berserk, killing three Glass executives and crippling two others before turning the gun on himself; the note he left said co-workers were out to get him. – Langway Figure 23 : July 28th 1976 27 Figure 24 : Reanimation Simon is now 35 years old. He lives in the subliminal space of the discarded toy, tucked into closets behind board games that pre-date him. On ebay he sells for less than he did in 1978 and he has yet to establish a relevant nostalgia market. Playing Simon today, the buttons don’t feel very responsive, the size is awkward and the timing is strange. Simon has become a passing curiosity, eliciting a smile of recognition and little else. After everyone gives him a go, he returns to the table and sits for a few minutes before beeping twice, to tell you he’s still on. But Simon’s work was already done. By introducing popular culture to the unpredictable digital interface, he shaped perspectives in the digital age. In playing with Simon, a generation found themselves inside of their systems. It is not that digital technologies produced virtual worlds that can be entered and left at will, but that digital programs merge with life—no glasses necessary— we are already here. 28 But Simon was made to model control, to see technology as an other and to disguise the programmer’s agency as friendly foe—both evil and harmless. A first step in unraveling this narrative is to become programmers ourselves, taking an active role in writing the systems with and within which we play. A second step is to consider how we interface with our programs. A computer is not a passive object, but “a process or active threshold mediating between two states.”28 Here, Simon is called upon to frame a constellation of ideas, figures and experiments that collide in a new system for playing with patterns of sound and light. Simon guides our exploration of the feedback systems between people and technologies in an attempt to blur the lines between music, games and performance. Our goal is not fixed, our boundaries are animated, and our movement is a navigation. It’s about the human/machine relationship or interface—the power of technology to expand the mind. You find solutions to creative problems and those solutions lead you into new territory where new solutions have to be found.29 : Pauline Oliveros The ndial lives in the confusing space between a game and an instrument, where the expressive nature of musical play comes in contact with the unpredictability of playing a game. At its best, this project reconsiders patterns of control and influence in an attempt to think about how new technology can be used to expand the mind rather than control it. By embracing ambiguity and playing with interfaces that push back—that are life-like—we might open new spaces for navigating the human/machine relationship. Reconsidering patterns in light of systems for reconsidering patterns is the continual navigation of the animating interface. 29 I push down on a button and the electrons of my nervous system engage the electrons in the interface. A mechanical connection allows current to pass through a circuit, tripping a pattern of switches that sends a message to a computer, triggering playback of an audio sample. The button is part of a controller that provides manual access to a specific set of digital processes. The communication is bidirectional and the controller receives feedback messages from the program. A light activates under my finger, visually mapping the sample to a location on the controller’s surface. I release the button. A sequence of lights plays unpredictably between the eight glowing buttons. I slowly turn a dial in the far right corner and the sequence gradually becomes less random. The lights move clockwise in a circle and only once in a while is the loop disrupted. I press and hold three buttons at the same time and override the pattern. The sequence now alternates between the three selected steps—a triangle. I let go and the loop resumes. I listen for a little while and then sharply turn the controller about 90 degrees. The pattern is the same but the sounds have all changed. The program has selected eight new samples and mapped them to the interface. I navigate the new sequence by flipping switches, turning steps on and off, and holding different combinations of buttons. I am listening for relationships. I am not certain what the program will do next. Together we navigate a strange trajectory through the song Indestructible by Swedish pop star, Robyn. 30 The song is in perfect time, pulsing away at 125 beats per minute, but everything else is out of order. Bits of lyrics are juxtaposed that don’t resolve as words, let alone coherent phrases. Blissful synth lines are cut off by a dry snare drum and a breathy inhale is followed by a filter sweep through a pulse of white noise. The genre is unmistakable—the saturated textures read as recent synthpop. You can still dance to the beat—but the original recording is scrambled. I twist the controller to refresh the samples. The sequence that emerges is more fragmented and angular but the lyrics now make a strange sense. “Never go, be the light. Never go, be the light.” I flip the switch for the sample “go”… “Never. be the light.” Holding down a button for a few seconds, “be the, be the, be the, be the, be the light. Never, be the light.” and once again reach for the dial in the upper right, increasing the pattern’s randomness, “light. Never. Never be the. Light, be the never light. Be the be the never light.” Holding down a button in the center of the interface, I push one of the sample buttons and let go. Nothing happens. Once more I spin the controller and get a new set of samples, but this time the sequence sounds entirely different. We have changed worlds. The compressed textures of popular production are replaced by a sequence of tones plucked on an mbira. I switch off steps one by one until only two remain, a minor 6th, triggering randomly every second or so. By turning the dial in the upper left, the mbira recedes into a large cathedral. The dial in the lower left removes the dry recording. Now just a drone of fuzz. I flip a switch on the side of the box. 31 Figure 25 : ndial The interface is an ndial, a square box about a foot across and 3 inches tall with an assortment of buttons, switches, and knobs arranged in a geometric pattern on its surface. It is made of wood and metal and looks a little like a turntable from an alternate timeline—a circular sequencer, an electronic oracle, or a puzzle from Myst. The ndial is where the two circuits fuse, where player and program work together to process patterns of sound, together manipulating sequences on the animating surface of the controller. In its primary mode, the pulse is automated by the program, and although the music is out of order, it is always ‘in time’. The program also listens to the environment and can draw sounds from live sources in addition to prerecorded materials. The interface is the control surface and the program is the co-pilot. With a player and some audio files, they form a system for navigating unpredictable trajectories through familiar sound worlds. By automating the timing functions and randomizing the sample selection process, the player is free to focus on and respond to the unpredictable patterns at hand. 32 ndiaI Figure 26: ndial manual INTERFACE AND OPERATIONS 33 Figure 27: ndial diagram The purpose of this manual is to describe the functions and controls of the ndial system, an interface (ndial) and program (Onism) for navigating sound worlds. The program selects samples at random from a live or prerecorded source and maps them to the hardware interface—an 8-step sampling sequencer that goes around instead of along. From the interface, a player can match pulses, record live samples, manually manipulate patterns, add effects and reanimate materials in real time. Samples are displayed as glowing buttons on the ndial’s surface, providing visual feedback. The system combines the expressive nature of musical play with the unpredictability of playing a game, and because control of the materials is limited, preconceived strategies give way to the matters at hand. 34 CONTROLS BUTTON / LIGHT SMALL BUTTON SWITCH DIAL Figure 28: ndial controls START The fastest way to get up and running. Using default sound worlds and effects. For new players, reading the manual first is encouraged. 1. Download Onism software. 2. Plug ndial into computer via USB. Center light will turn on. 3. Open Onism and click “connect” in the software (green). 4. Shuffle the ndial. 35 SHUFFLING One of the first things that you will notice is that the ndial spins. It has an internal gyroscope that measures angular momentum and if the ndial is turned abruptly, it chooses eight new sound events from the active world and maps them to the steps of the interface, shuffling the samples. The sensitivity is such that slight movements won’t trigger a shuffling, it requires a decisive turn. It is the player asking the program to deal again. Figure 29: Shuffle There is no simple way for the player to choose specific samples for a particular step. There are plenty of hardware and software systems that have this functionality, but the ndial is not built for realizing preconceived patterns. The shuffling mechanism forces the player to let go and explore potentials that cannot be figured beforehand. In navigating the unpredictable, we are forced to actively listen; refiguring musical expression as emerging from the interplay of people, technologies and references/materials, rather than from the mind of a composer or improviser alone. 36 INTERFACE Figure 30: Face 1 TRIGGER BUTTONS 42 RANDOMNESS DIAL / SEND 2 2 ON/OFF TOGGLES 43 DRY VOLUME / PULSE DIVISION 3 SHIFT BUTTON / POWER LIGHT 44 PITCH DIAL / SEND 3 41 STEPS DIAL / SEND 1 5 PATTERN 37 1 TRIGGER BUTTONS Momentary glowing arcade-style push buttons function as both indicators (actuators) and triggers (sensors). They are everything. When a step is triggered, its button glows. If a button is held the sequencer will repeatedly trigger the sample. If more than one button is held, the program will arpeggiate between them. Trigger buttons have additional functions when used in combination with the shift button, discussed below ( 3 ). 2 ON/OFF TOGGLES Toggles turn steps on and off. The sequencer does not trigger a step that is toggled off. Toggles can create empty spaces in the pattern. 3 SHIFT BUTTON / POWER INDICATOR (THE LIGHT IS ON IF CONNECTED) Changing sound banks. Like a shift key on a computer, pressing the center button remaps the trigger buttons and dials on the face of the ndial to access a second layer of control (Galaxy Level). If the player presses one of the trigger buttons while the center button is down the sound world associated with that button enters the queue. Next shuffle, that sound world will activate. Each of the 8 steps can be associated with a different recording, meaning 8 records (Sound Worlds) can be ‘on call’ and activated directly from the interface. The order of operations takes a bit of getting used to, but switching banks is a powerful technique for changing the direction of a trip. Secondary functions. Each of the four dials have secondary functions as well, discussed in the dial descriptions below, but the way the ndial remaps the dials between layers is consistent. When the shift button is pressed, the system stores the dial’s state and switches control to its secondary functions. When the shift button is released, the dial returns to its primary function, but turning the dial has no effect until the dial is turned past its stored state. The same is true in reverse for the secondary controls. This prevents data and thus sound from jumping when moving between control layers and allows one dial to control two continuous functions. Power. The shift button glows when the device is on (connected to a computer). 38 41 STEPS DIAL / SEND1 The steps dial allows for changes to be made to the length of the sequences. A sequence can be 1 to 8 steps long and the dial is mapped in 8 equal sections. If the player truncates the sequence, to say, 5 steps, the excluded 3 steps are still active for triggering but will not automatically play in the sequence. The steps dial introduces the first of three effects sends, which work much like a send on a mixer. Using the secondary function (shift+dial) the player sends the dry signal to a preconfigured effect. Effect 1 defaults to a long reverb but custom effects can be easily swapped in via the software. 42 CHAOS DIAL / SEND2 The chaos dial randomizes the steps. As the dial is turned clockwise the probability that the next step will be random increases. The algorithm is like picking numbers out of a hat. Steps are selected randomly until the last step is picked and the hat is empty, at which point all steps are again fair game. This method produces little step repetition but steps can be repeated manually using the triggers. The second function is effect 2, a delay that is linked to the tempo, similar to a DJ echo. The architecture is the same, sending the dry signal to the specified effect. 43 DRY VOLUME DIAL / PULSE RESOLUTION The dry volume dial affects the amplitude of all triggered events, but has no influence over the effect sends, allowing the player to remove the unprocessed samples from the mix, leaving only the effected sound. The secondary function changes the pulse resolution. Divisions of 2, 4, 8, 16, & 32 allow for compression and expansion of the pattern within the pulse. 44 PITCH DIAL / SEND 3 The pitch dial stretches the reference recording (effecting both time and pitch) enabling the player to scale the frequency and time of the entire pattern. This function operates pre- send and adjustments to the pitch dial are also sent to the effects. The dial goes from 0% to 100% in half a turn, lingers for about 12 degrees at 100% (the original frequency/speed) and then continues at a steeper rate to 400%, providing more space to play on the s- low’ing side of ordinary while adding range to the higher-faster end. The second function of the pitch dial is the last effect send (send 3), defaulting to a second reverb with a different character than effect 1. 39 RIGHT Figure 31: Right 6 SAMPLE CHANGE BUTTON If the player holds down a trigger button and then hits the sample change button, the system will randomly select a new sample for that step, allowing the player to change the sample of a single step without changing the entire pattern. For example, if samples are being pulled from bank A, the player can switch to bank B (or live mode) and use the button+shift method to change one of the steps, creating a pattern that combines elements from more than 1 source. 7 TAP TEMPO The player can set or match tempos using the tap tempo button. It ‘listens’ to a series of presses and sets the pulse based on the averaged intervals. This function is helpful in synching to the beat of pre-existing players or material. 8 TEMPO TOGGLE The pulse toggle changes the tempo mode. When on, pulled towards the front of the interface, the tempo is set by an external pulse. This allows multiple ndials to sync to the same clock. When pointing back towards the tempo dial, the tempo dial takes precedence and the external source and tap-tempo are both disabled. 9 TEMPO DIAL The tempo dial adjusts the pulse speed of the sequence. Its range can be set in the software. Used along with the Pulse Resolution dial, the sequencer speed can be quite slow (around a second per step) or very fast (8 loops or 64 steps per second) Players may also set default tempos for each reference, thereby loading preset tempos when switching between banks (especially helpful with beat-based music in which the beat of the reference is retained). 40 LEFT Figure 32: Left 10 CLEAR BUTTON The clear button is like a panic button… clearing all samples. It can also be used as a method for instant silence. All samples and effects are cut! If the ndial is shuffled after being cleared it will repopulate the steps from whatever bank is selected and resume the sequence. 11 LIVE BUTTON Holding down the live button records a live input and populates whatever bank is selected with the recorded events. The system looks for transients or sharp attacks and defines a new sample at each transient. Once the record button is released the samples are saved and available from the selected bank. Banks can be locked in the software to prevent certain banks from being overwritten. 12 RECORD TOGGLE When the player flips the record toggle up, the NDIAL records the output to an audio file that is saved on the computer. This is a quick and easy way to record loops and patterns discovered while playing the NDIAL. Recorded files are saved to the desktop and labeled with the time of the recording. 13 MODE TOGGLE ndial has two modes, pulse mode (toggle down) and free mode (toggle up). Pulse mode is the standard sequencer mode, automatically moving through steps in time and quantizing events to the beat. In free mode, the ndial is a triggering system, allowing players to play samples without being tethered to the beat. Changing modes can be used as a method for play, effectively starting and stopping the pattern without disrupting the sound like the clear button. 41 FRONT Figure 33: Front 14 MASTER VOLUME DIAL The master volume dial controls the amplitude of all the sound processed through the system. The dry signal, the effect sends, and master EQ, any live input, everything. The dial responds such that it corresponds to perceived amplitude. 15 EQ TOGGLE Like the master volume dial, The EQ affects the entire sound. The EQ toggle is a three position toggle that is off in the center position. A high-pass filter is active when the toggle is up and down activates a low pass filter. The EQ can be used to sweep through the frequency content of a reference, isolate registers or change the overall quality of a sound world. The equalizing function is helpful for Figure 34: EQ 16 CUTOFF FREQUENCY DIAL The cutoff frequency of both the hi- and low-pass functions is manipulated via the cutoff frequency dial. The default range (adjustable in the software) is 50Hz on the low end and 5000KHz on the high end. The default range manipulates the most perceptually sensitive part of the sound spectrum, privileging perceptual affect over maximum range. 42 BACK Figure 35: Back 17 USB INPUT The NDIAL connects to a computer using a standard USB2 instrument cable. Plug in the USB cable before starting the software. 18 SYNC INPUT The sync input is an analog input, for connecting a photocell or distance sensor or any other type of analog sensor. The software looks for spikes in the incoming data and if the spikes exceed the threshold (which is adjustable in the software) the NDIAL will attempt to conform to the rhythm of the data spikes. This is useful for bringing outside information into the NDIAL system and synchronizing pulses across NDIALs. The sync input can also be remapped as an additional analog control, meaning the system can take cues from its environment. For instance, the amount of light can be mapped to overall tempo such that the pulse will slow down during a sunset. 19 TRIGGER OUTPUTS The trigger outs extend NDIAL control into the environment. 8 digital triggers (with built-in 220ohm resistance) and 1 ground connection allow for easy connection to external lighting systems but they can also be used to trigger analog synthesizers, motors or microphones. Using the output triggers and simple external modules, one can easily create tightly synchronized mechanical or audiovisual experiences. 43 ONISM co-pilot Figure 36 : Onism n. a thought process, specifically one that takes hours on end. 30 - Urban Dictionary n. the frustration of being stuck in just one body, that inhabits only one place at a time, which is like standing in front of the departures screen at an airport, flickering over with strange place names like other people’s passwords, each representing one more thing you’ll never get to see before you die—and all because, as the arrow on the map helpfully points out, you are here. 31 - The Dictionary of Obscure Sorrows The program is named Onism, a virtual replica of the ndial with additional options that let the player change effects, import new sound worlds and set default tempos and behaviors. Started as a quick way to randomly sequence samples in Max/MSP, Onism has grown to do many things, but Onism has never been rewritten, just added to. It projects well enough but is a huge mess inside. 44 Figure 37 : Onist Perspective A mess of interconnecting modules and methods were built to work with the ndial. Behind the geometrical symmetry is a patchwork reality that has been cobbled together by necessity, not design. Here one can search for an idiosyncrasy for hours and never find it. Just like the programmer who produced it and the player that interacts with it, Onism is noisy and perpetually incomplete. Onism is certainly not efficient, but who cares, it works, and it’s fun. Figure 38 : Key Mode All of the functions necessary for standard set-up of the ndial are available from the interface (presentation mode). There is also an ASCII mode that makes playing with Onism possible without the ndial, and, if you dig deeper, there are a few signposts to help guide you through the inner workings, but good luck. Email me with questions. 45 1 LOADING SOUND WORLDS AND SETTING TEMPOS Sound Worlds are loaded using the panel in the upper right. Set samples (S) or file (F) or live recording (R) to tell Onism what type of sound world you are loading. You can drag and drop a folder of samples or a file into the name area to import a sound world. If you are recording a world live, make sure Onism has the correct input selected. Green arrows point to the active sound world. Figure 39 : World Type Default tempos for each sound world are set to the left of the input field. When a new sound world is shuffled the sequence also sets the default tempo. To change sound worlds without changing tempos, see the Sample Change Button above. Note: the sound world type functionality in alpha mode 2 CUSTOMIZING EFFECTS There are three programmable effects that map to the secondary functions of dials 1, 2, & 3 respectively. The effects panel can be accessed from the main interface (via pop-up). To send to a custom effect, turn off the effect that you want to replace and just add a receive:e1 (for effect 1) as input to your effect and send:E1L and send:E1R (Stereo) out from your effect. 3 CONTROL MAPPINGS AND SENSITIVITY You can change the control ranges for the four dials and the sensitivity of the shuffle gesture from the [control] pop up on the main interface panel. 4 FURTHER CUSTOMIZATION External lighting controls, interface protocol, panning, and buffer control are also available from the front panel. To play around with the sequencing system you have to leave presentation mode. The internal signal flow general goes left to right. 46 WORLDS Figure 40: Worlds Each ndial accommodates up to eight sound worlds at a given time. A sound world can be made from a recording, a set of samples, or a live microphone/input. Sound world selection is an important aspect of playing the ndial. By choosing sound worlds, the player defines the universe in which they navigate. Sound worlds are material. It doesn’t matter if they are 8-bit fuzz, delicate paper crumpling, or Kate Bush songs, they are records of a past and they will push back and assert themselves. They will allow some trajectories, and refuse others. Some players head directly to points of friction. Other players look for hidden spaces between the samples. They is no right way to go, but worlds determine possibility spaces. Try different things. 50 people saying the word “no”, the pizzicato movement of Tchaikovsky’s fourth symphony, and applause recorded 9 seconds ago. Each of these worlds presents different paths, a topology to navigate. Some world’s are good for traveling far and hearing many spaces, other’s require a more thorough search. If a player knows the source intimately, navigation becomes similar to reading a few selections from a favorite book. The horrors and constraints of building worlds live often prove worthwhile and can lead to wonderful places. Risk it. All worlds pulse, but you can freeze them. You can also just listen for while. This is not a race. There is always more time. 47 INSIDE Figure 41 : Inside In the interface is a 32-bit microcontroller 32 running custom firmware through a serial communication protocol developed by Butch Rovan that monitors and encodes all sensor data from the interface into a compact high-speed format that sends information between program and player through the interface. These sensors include various buttons, switches, and dials, along with a gyroscopic sensor that listens for the shuffle. The data controls a program, Onism, that lives in a computer and handles all of the signal processing—sound generation, timing control, manipulation, effects, and pattern mapping. The interface provides a way of influencing these processes, it also outputs trigger voltages that can be used to control an external lighting system. 48 COMPONENTS Figure 42: Components The microcontroller is the UBW32 PIC32MX795 IC. The trigger buttons are ChromaLite arcade buttons with a built-in LED and resistor. The toggles are NKK Switch SPDT toggles and all of the dials are 24mm Alpha potentiometers with heavy knurled aluminum dials. The Gyro chip is the LPY503AL with Sparkfun’s breakout board. The Sync input is designed to be used with an 1/8 inch audio cable (which can easily be soldered to a photocell or some other variable resistance sensor to gather pulse information). The trigger outs are conveniently bundled into a serial jack for connection via serial cable to external lighting modules (Pins 1-8 are digital triggers and pin 9 is the ground). An 8-channel Darlington Driver drives the LEDs. Connections are heavy-duty jumper cables on the microcontroller side that are hand soldered to controls on the interface. The dimensions are 12” x 12” x 3” (high) and the box is built out of pine and sided with walnut. Figure 43: UBW32 PIC32MX795 IC & LPY503AL 49 THE LYREBIRD Figure 44 : Technical Processes People think of a point as the most primitive thing with which to initiate geometrical conceptioning. A point is a microevent of minutiae too meager, they say, to be dignified with dimensionality: Ergo, they assume a point to be only an "imaginary fix." But speaking in the experiential language of science, whatever is optically point-to-able is a substance, and every substance has insideness and outsideness__ergo, is systemic: Ergo, all point-to-ables can never be less than the minimum system: the tetrahedron. Points always amplify optically to be identifiable as systemic polyhedra.33 : Buckminster Fuller, Synergetics 50 The lyrebird mimics the sounds of its environment, remixing multiple sources into a complex mating call. It draws primarily from other birdsongs, but lyrebirds also feature precisely rendered flute melodies, camera shutters, rifle shots and chain saws. The lyrebird makes little distinction between categories of sound—natural, artificial, mechanical, organic—it just mimics patterns that seem significant and resonant. listen People also incorporate what they hear to be significant and resonant, but many of these sounds are difficult to reproduce using only our bodies and so we develop of technologies—tools, methods, and practices—that help us to work with them. The complex sounds of our environment—the character of a voice, a specific french horn section, a chain saw—require technical devices for capture, storage, and playback. These devices have extended our abilities well beyond those of the lyrebird. In addition to remembering and reassembling, we can warp, filter, modulate, fold and automate sound. Over time, the sounds that we find resonant and significant come to reflect the prevailing technical methods used to produce them—records, tape, radio, and computers are each entangled in the music we make and hear. “One produces what technology makes possible, instead of creating the technology for what one wishes to produce.”34 Because our samples are captured through recording devices, the sounds that we sample are those that resonate with the machine. In some sense, they are recorded by mechanical ears, filtered not by our built-in biases, but by the material constraints of our external devices. In listening to recordings, we are attending to a different perspective and over time, our perceptual systems change. Although we can disengage from our devices physically, some trace of the machine perspective remains. 51 Our hearing has been framed by recordings, our thoughts have evolved to reflect the perspectives of language, and our vision has adopted the technical lenses of film and photography. Although permanently embedding complex machines into our flesh is relatively new, we have long been technological beings. There's a funny thing about the fusion of technology and culture. It has been a part of human experience since that first cave painter, but we've had a hard time seeing it until now.35 : Steve Johnson, Interface Culture There is a boundary to consider in thinking with the lyrebird. While machine perspectives are different from the built in mechanisms of a bird, they are still integral to our understanding of the world. Using a microscope we extend how we see and what appears influences how we see ourselves (we are made up of cells). In recording sound and light we can experience the passing of another time and in doing so we express symptoms of time-travel (projection, fragmentation, perceptual folding). A tangle of feedback systems perpetually blur what both human and technology mean and the lens of electronic automation has made visible this confusion. human functionality expands because the parameters of the cognitive system it inhabits expand. In this model, it is not a question of leaving the body behind but rather of extending embodied awareness in highly specific, local, and material ways that would be impossible without electronic prosthesis. 36 : Katherine Hayles, How We Became Posthuman Automated systems present new considerations, allowing us to operate in parallel with our machines. Tools can be programmed to respond and interact, animated and life-like. Control can be displaced in time. A single button press can activate chains of complex functions, and we can create systems that listen, process, and reassemble the sounds of the environment automatically. There are millions of lyrebirds at our fingertips. 52 But our machines do not simply amplify, they actively alter our perceptual systems. Although an eye is not a camera and an ear is not a microphone, with a camera and a microphone we see and hear differently. What we find to be significant and resonant changes over time and with use. Even the most accessible of today’s electronic music would have been obscene only 60 years ago. By framing listening itself as a technology or method, we can think about the ways external devices have extended or altered how we process our environments. This exploration draws on Matthew Warne’s thesis that, “Conceiving listening as a technology opens it to intervention and encourages the intentional development of new strategies for listening.”37 This exploration starts from the premise, perhaps a belief, that listening and composition are essentially the same thing, or at least similar. In playing with the ndial, which I have come to think of it as an interface for active listening—listening in which the sounding materials can be actively manipulated during playback to highlight different features or patterns in the sound. Active listening provides manual access to the technical processes of digital playback and allows for real-time multimodal navigation of recorded sound worlds. The ndial records, samples, sequences, and processes sound such that a player can ‘compose aloud’ by listening for interesting patterns and new musical encounters. I am trying to work towards Jacques Attali’s notion of composition, which is used not to describe the act of the composer, but a state where “production melds with consumption” and music “becomes a starting point rather than an end product.”38 For Attali, composition is a mode that “heralds the negation of the tool-oriented use of things”, a kind of transformative interplay that is “inventing the message at the same time as the language.” 53 Inventing new codes, inventing the message at the same time as the language. Playing for one's own pleasure, which alone can create the conditions for new communication. A concept such as this seems natural in the context of music. But it reaches far beyond that; it relates to the emergence of free act, self-transcendence, pleasure in being instead of having. 39 When composing, our technologies (both conceptual and physical) are not merely functional tools, but methods that produce a space of possibility. Again, “One produces what technology makes possible, instead of creating the technology for what one wishes to produce.”40 Our instruments are not static devices but playgrounds for exploring sonic potential, for playing with and reconsidering patterns as figured through our systems. Atau Tanaka interprets Attali’s notion of composition as “alluding to future potential musical forms that are not just finished works, but instead generated at the time of listening… conceiving of a music that is constructed not as a deterministic product, but as structures of possibility, to be completed only at render time with the active participation of the listener.”41 The following pages examine how four technical methods have altered how we listen and have opened new possibility spaces for playing with sound that push towards a practice in which composition and listening are one in the same. Recording has folded sound and time, creating non-linear listening spaces. Sampling has given rise to a practice of reading and writing sound, expanding the semiotic potentials of sound. Sequencing has allowed us to listen with the machine, framing our systems as collaborators that transcend mere usefulness, and programming has shifted the focus from sound itself to sound as a medium for exploring relationships—figuring. These processes are all at play in the ndial and together they form the possibility space of nterplay—a low resolution mode of active listening that privileges experimentation over precision and collapses the expressive nature of music and the navigational nature of games into an open ended system for play. 54 Recording Technological progress has merely provided us with more efficient means for going backwards. 42 : Aldous Huxley Before playback—wire, wax, vinyl, tape, plastic, silicon, etc—sound disappeared with time, an echo, the resonance of the world itself, was the only record. But the phonograph displaced sound from the here and now of its source and stored it in a material—readable and malleable. Etched into vinyl, sound can be saved, traded and manipulated, and in time, playback came to reframe play. “Sounds just like the record.” When we playback sound, we also playback time. Not time as represented in the signs and symbols of scores and books, but the experience of time's passing. Carried by sound, time is in play—it can be cut and folded back into itself. People have adapted to process multiple timelines simultaneously, listening across time as well as space. Audio tape allows temporal transformations of both inner and outer time in which spatial, musical, and temporal experience can alternately be distanced or narrowed, individualized or collectivized, forwarded or backgrounded.43 : Thomas Porcello, Tails Out Deep Listening for me is learning to expand the perception of sounds to include the whole space/time continuum of sound – encountering the vastness and complexities as much as possible. Simultaneously one ought to be able to target a sound or sequence of sounds as a focus within the space/time continuum and to perceive the detail or trajectory of the sound or sequence of sounds. Such focus should always return to, or be within the whole of the space/time continuum (context). Such expansion means that one is connected to the whole of the environment and beyond. 44 : Pauline Oliveros, Deep Listening 55 Before playback, we listened through our environment, but mechanical recording made it possible to listen to our environment. In passing a needle over patterns cut into wax or vinyl, we replay sound as it is captured by the mechanisms of an external system, unfiltered by our built in senses and circuits. Mechanical recording produced sounds that our ears and brains had previously filtered out—the fluctuations that resist patternicity and thus can not be remembered—noise. The lyrebird reproduces a pattern that represents a perspective formed through repetitive hearing. Background noise and inconsistent features are not recorded, or not remembered. This is how people hear as well, foregrounding and memorizing recognizable patterns like speech and music. Before mechanical reproduction—with only symbolic means for recording sounds—noise could not be figured, only experienced through the ever fleeting frame of the now. Texts and scores-Europe had no other means of storing time. Both are based on a writing system whose time is symbolic… whatever ran as time on a physical or real level, blindly and unpredictably, could by no means be encoded. Therefore, all data flows, provided they really were streams of data, had to pass through the bottleneck of the signifier. Alphabetic monopoly, grammatology. 45 : Friedrich Kittler, Gramophone, Film, Typewriter The noise within time can not be recorded or remembered symbolically because it exists only in the negative space of perception and must be experienced. Because noise is that which can not be figured, our memory of noise is lost when passed “through the bottleneck of the signifier” (words, icons, symbols). Manual notation systems are predicated on symbols and designed for communication, the transfer of predetermined meaning. Symbols are contingent on familiarity with a common language. Playback is not symbolic but indexical (an imprint) and therefore it provides a way around symbolic representation, allowing people to replay the noisy experience of a passing time’s past. 56 In folding and fracturing time, a compound perspective emerges through which we hear multiple times simultaneously. Listening becomes a process of navigating a layered, non-linear soundscape—part receiving (hearing), part perceiving (listening), and part figuring (relating). In navigating the folds of time, listening becomes an active process. By the 1950s, people were experimenting with playback media as material for musical composition, repurposing the output objects of mechanical reproduction as input for a different type of music, the assemblage. Assemblages explored our developing abilities to listen on multiple levels and took advantage of a new kind of reading and writing of sound made possible by recording and playback—sampling. From Pierre Schaeffer’s diary, dated April 22nd 1948… Once my initial joy is past, I ponder. I’ve already got quite a lot of problems with my turntables because there is only one note per turntable. With a cinematographic flash- forward, Hollywood-style, I see myself surrounded by twelve dozen turntables, each with one note. It would be, as mathematicians would say, the most general musical instrument possible. Is this another blind alley, or am I in possession of a solution whose importance I can only guess at? There is no one instrument to play musique concrète. This is the major difficulty. Instead one must imagine an enormous machine, of a cybernetic type, capable of carrying out millions of combinations, and we haven’t reached that point yet. 46 Schaeffer was envisioning the digital sampler, an instrument that could reproduce all of recorded sound without material limitation. He wanted to play with recordings like a violinist might play with pitch. He heard the potential in performing his sound objects in real time and perhaps even glimpsed the possibility of composing modes of attention—moving between hearing, listening, and figuring like a trombonist glides down a minor third. But real-time digital sound was still decades away. In the mean time, musicians were experimenting with electrons, giving voice to the machine. 57 Sequencing The abstraction that separates the sound production of an electronic instrument from the force that triggers that production allows the triggering itself to be automated, displaced from its original time and place. 47 : Aden Evens, Sound Ideas In Close Encounters of the Third Kind, a 5-tone musical sequence generated by the extraterrestrials is the primary form of inter-species communication. It moves between people and machines, navigating an impressive history of sound technologies—a voice, a choir, a toy xylophone, a tape recorder, the Yamaha SY-2 and Arp 2500 synthesizers, and a Theremin-like gesture controller. In the final scene, a stadium-sized audiovisual sequencer is used to communicate with the UFO—understood to be the original source of the pattern. The UFO toys with the human operator, playfully developing his simple responses into a futuristic Fantasia, sound and light pulsing in perfect time. The sequencer’s human operator is no match for the virtuosity of this new life-form and in the middle of the improvisatory performance, control is transferred to a computer well-suited for this type of pattern play. This shift exemplifies the emergence of a process that would come to characterize both music and play throughout the 80s. The techniques and aesthetics of electronic automation, and automated sequencing specifically, spread quickly. Its potentials were expressed in the futurist predictions of Kraftwerk’s Ralf Hütter, “With better machines, you will be able to do better work, and you will be able to spend your time and energies on a higher level.” 48 58 Music boxes, player pianos, the Moog 960, and the TR-808 are all sequencing systems; so are musical scores, books, words, computers, watches, calendars, schedules, and playback machines. People are sequencers, putting things in order and creating patterns of time. The universe, with its perpetual revolutions of planets, and galaxies, is also a sequencer. Every natural and material process, every thought on every level is a sequence. Genes, animation and life itself, sequential. While most sequences run at irregular intervals, electrons keep a regular pulse—perfectly accurate to the human ear. Over time, this precision became an aesthetic element and Tangerine Dream, Kraftwerk, Laurie Speigel and others adopted the expressive potentials of automated time. For a century, the phonograph and its descendants had been able to capture and reproduce the human voice, but Kraftwerk’s sequencers were programmed to explore the voice of the machine. Figure 45 : The Moog 960 Sequential Controller At its most basic, an analog sequencer is nothing but a bank of potentiometers and a "clock" that steps through these potentiometers one at a time and then cycles back to the beginning. The output of the sequencer is fed (as a control voltage and gate pulse) to a synthesizer. By "tuning" the potentiometers, a short repetitive rhythmic motif or riff can be set up. 49 : Thom Holmes, Electronic and Experimental Music 59 A step-sequencer does not produce sound by itself, but is a control device that sends information to an instrument that produces sound—often a synthesizer. Sequencers store a series of states whose values are transmitted at regular intervals. Effectively, sequencing automatically orders events in time. The sound of the sequence depends on the system being controlled. The violin sounds only in direct response to human effort, but the electronic sequencer separated sound from physical effort through the abstract processes of electrical systems. This disconnect produced a space where instruments could be conceived as systems to play with, not on. By externalizing the pulse, one of the most prominent aspects of music, the player could step back and listen and then experiment in real time. Composition and listening began to overlap. Western music traditionally expressed a pre-formed pattern (the score, the song), but the sequencer shifted attention to the processing of patterns as they moved in time, navigating rather than describing. By 1978, the focus on process had been explored by experimental composers, but the sequencer brought process music to the mainstream. Kraftwerk’s aptly titled Man-Machine, was released within months of Simon’s Studio 54 debut and played with a familiar tension, navigating between machine paranoia and technology worship, leveraging a similarly playful and minimalist aesthetic. Kraftwerk’s influence is found throughout electronic music and sampling culture and even crossed back into experimental spheres. “…the drive for unusual forms, and surprise. This aspect of openness, the young character is what I like.” 50 Stockhausen was a fan. A different type of automated sequence drives the digital realm. Most computers are outfitted with crystal oscillators that determine the rate at which commands are executed. Simon runs at a 2.5 microsecond clock cycle, meaning that all of his instructions are executed at 2.5 microsecond intervals. The speed of digital computing allowed for sound to be encoded as bits, manipulated mathematically as data, and played back with remarkable accuracy. 60 Sampling The street finds its own use for things—uses the manufacturers never imagined. 51 : William Gibson By sampling I refer to a composite process—the ability to record, remember and playback sound for use in some other arrangement. Sampling is not just a technical method (record, memory, playback) but also a functional description (a technique for reorganizing sounds). The tendency is to view musical sampling as a product of recent digital technologies, but the concepts that underscore sampling predate digital, electrical and even mechanical recording technologies. Musical borrowing, drawing from previous forms and adapting them to new contexts, is how music in oral traditions evolved and is found throughout notated forms as well.52 Although sampling as a method is not new, the emergence and maturation of digital technologies have changed how and how much we sample. The speed and flexibility of computers have accelerated a cutup culture of rapid reanimation—assemblages, remixes, mashups, collages, and versions have all become popular forms. Digital sampling has also influenced what we sample. The indexical nature of recording and playback has replaced the formal and structural quotations of symbolic and oral traditions with a sampling practice that Mark Katz calls “performative quotation”. [Performative quotation] recreates all the details of timbre and timing that evoke and identify a unique sound event, whether two seconds of Clyde Stubblefield’s drumming or the slow, unsteady tapping rhythms produced as I type this sentence. In other words, traditional musical quotations typically cite works; samples cite performances. 53 61 Performative quotation draws on the characteristics of sound that are intangible and un- representable, what Katz calls the “sonic aura” of a sound. A sample points to a specific instance, sampling not only the musical material, but also the experiential details and unintended noises of the original. The word phonograph means ‘writer of sounds or voice’ and with the ability to write sound, a practice analogous to reading sound emerged as well. Sampling invited new modes of listening and provided the means for animating between listening modes, composing systems of signification and meaning-making that extend beyond the sound itself to draw on a larger system of relations. Samples are be chosen for musical content, but also for recording quality or a specific feel or feeling. They can be juxtaposed with other sounds to connote an alliance, make a political statement, or directly leverage pre-existing musical value systems, resembling practices and functions traditionally associated with folk music traditions. Folk music is based on the practice of drawing on existing melodic and textual elements and recombining those elements in ways that create a song that can range from a slightly modified version of an older song to a wholly new piece that contains echoes of familiar melodic or lyrical themes. At the center of this mode of cultural production is intertextuality, in which texts are (re)made from other texts to create a “new” cultural text.54 : Kembrew McLeod, Owning Culture Digital sampling presents a new type of intertextuality, reading sound in relationship to other cultural texts and musical references. This semiotic approach to listening is at odds with the non- representational “sound object” and reduced listening desired by Schaeffer and echoed by Francisco López. López writes that “when the representational/relational level is emphasized, sounds acquire a restricted meaning or a goal, and this inner world is dissipated.” 55 By ‘inner world’ López refers to the “richness of sound matter” that is accessible through reduced (Schaeffer’s term) or profound (López’s term) listening. 62 This tension between sound-as-object and sound-as-medium has been present throughout the history of electronic music. Digital sampling—referring to both the practice and the technical process of digitization—provides musicians with an environment to play with this boundary. A drone that slowly transforms into speech. A field recording that turns into a pulse. Represented as data, sound can fluidly play between listening modes and resonate different types of closure. In psychology, closure is a Gestalt principle that explains how people fill in gaps in data, perceiving disconnected parts as a unified whole. Comic artist Scott McCloud expands the idea of closure to apply more broadly to our ability to create meaning between non-continuous events. In comics, the break is called the gutter, a spatial technology through which meaning is constructed. In music, the gutter is the cut or the fold, and closure is the resolution of the conceptual distance between disparate samples. figure 2.2 – Closure 56 There are as many types of closure as there are types of relationships, but every instance of closure requires some type of participation. Sampling practices draw on relations outside of the sound matter, expanding the range of listening into semiotic dimensions. Meaning-making occurs within a larger context of cultural relations and “the musical artifact itself becomes a kind of 57 interface.” Combined with programming, the hybrid listener/composer “is empowered to playfully experiment in the elaboration of her/his relationship with acoustical and musical materials, and with compositional procedure itself.” 58 63 Programming The computer has changed the potential of music theory since, for the first time, it has given composers a tool for capturing their processes, and for articulating a theory of music based on their knowledge of compositional planning and problem solving (as distinct from their knowledge of historical musical artifacts). 59 : Otto Laske, Composition theory By the mid 90s, computing had advanced to the point where real-time manipulation and synthesis of sound was both possible and affordable, leading to an explosion of digital play/performance practices—laptop performance, networked games, digital deejaying, controllerism, and virtual reality to name just a few. The computer’s role in these practices can be broken down along the binary of tool vs. medium. The computer as a tool extended pre-existing processes by leveraging the technical agility of the digital realm. Others saw the computer as a medium and developed custom processing systems for manipulating data, experimenting with the computer’s ability to model procedures instead of describing artifacts. By programming rules or procedures, one can model a compositional process that articulates a range of potentials. I hope to show that an experimental composition aims to set in motion a system or organism that will generate unique (that is, not necessarily repeatable) outputs, but that, at the same time, seeks to limit the range of these outputs. This is a tendency toward a "class of goals" rather than a particular goal.60 : Brian Eno Eno’s thoughts are made in reference to a piece by Cornelius Cardew for acoustic voices, revealing that even for those who saw the computer as a medium, programmability found precedent in historical practices. Procedural methods had been employed by composers of experimental music for decades, and the computer extended these potentials. 64 But as we have already seen, extending our methods to technical devices does not simply amplify pre-existing perceptions but absorbs the perspectives of the machine as part of our world view. Unlike the procedural music of the past in which instructions were written and then executed, the computer allows players to manipulate conceptual models in real time, effectively steering or improvising the rules as they unfold. Here, the distinction between listening and composing breaks down entirely, as the computer allows the player to listen actively in the most literal sense. Listening can be modeled as the discernment of differences through a process of “learning/unlearning.” A composer shapes and structures the unfolding of such a process, and is as such interested in generating cognitive frames within which musical patterns playfully reference choices made within a system. Precisely how the composer sets this up is conditioned by the notion of the task environment in which s/he works. When empowered to participate in the shaping of that environment, the composer reconditions the manner in which otherwise historically framed notions of materials and processes are engendered. 61 : Michael Hamman, From Symbol to Semiotic Real time digital processing also collapses the tension between sound-as-object and sound-as- medium into a simultaneity in which one navigates between modes or levels of listening. Listening to digital performance becomes a process of animated attention as focus shifts between the decisions of the player, the logic of the program, and the character of the sound. Because the input/output feedback loop runs at the data rate of the interface, the interacting agents can no longer be fully distinguished in time or space. Player, program, and sound matters are co- produced together, each in a state of perpetual becoming that expresses a system for play. Without stable artifacts, focus shifts to the interface which transforms into a medium with aesthetic value. This shift can be found throughout culture, most visibly in the fetishistic relationships with new communication technologies. The focus is not on what is being transmitted but how the transmission is taking place. As McLuhan famously proclaimed, “the medium is the message.” 62 65 The source of the exhilaration associated with computer programming is the continual unfolding within the mind and on the computer of mechanisms expressed as programs and the explosion of perception they generate. 63 : Alan Perlis, Structure and Interpretation of Computer Programs But again, our technical processes are bi-directional and with the ability to program, we become programmable. In and of itself, this is neither good nor bad but speaks to the importance of the interface as it becomes a site not only for processing the environment, but for processing what it means to be human. As our lives become increasingly entangled with the virtual realm, building and playing with digital interfaces becomes a process of experimenting with the very feedback loop between human and technology that we have been exploring throughout this volume. [The computer] enables the composition of interfaces and representations according to which musical problems may be formulated and solved. Consequently, the nature of the relationship between human and machine is transformed such that the human is no longer merely a “user” in the sense that the carpenter is the “user” of a hammer, but rather is a participant in a “game” of hermeneutic significance. This is because, the computer is itself a tool for the construction of tools - tools with which one might generate epistemological frameworks for imagining and solving problems of compositional significance. Like Pollock laying the canvas flat on the ground in order to effect a new way of interacting with his materials, the composer is empowered to playfully experiment in the elaboration of her/his relationship with acoustical and musical materials, and with compositional procedure itself. 64 : Michael Hammon, From Symbol to Semiotic 65 What Jordan Bartee calls “ontological toys” and Chris Novello describes as “expressive systems”66 are interfaces for playing with our relationship to the world by figuring in rather than figuring out. Experimental interfaces can provide a method for exploring the systems that we live within by modeling systems that we can play with. Ultimately, this animation is not inherently spatial or even temporal, but conceptual—refiguring the world as taking place between cognitive frames and resulting in a type of animating worldview that seeks not describe another universal system/frame, but provides a lens for navigating between them—for reconsidering the pattern in light of the system for reconsidering the pattern... 66 Figure 46: Map Inside 67 RECONSIDERING THE PATTERN I consent to be space. I consent to be time. I consent to be signal. I am already future. Figure 47: Initial _______________________________________ initial here 68 Figure 48 : Title Reconsidering the Pattern is an evening length performance-game-system. Guests sit in two areas on either side of the system, facing four dimly-lit ndial stations in the middle of an otherwise dark studio. Up to four players operate the system at a time, navigating both pre-selected sound worlds and live input provided by audience input. Sound worlds can be anything—field recordings, popular songs, speeches, synthesized sounds, and live feeds. After sound worlds are chosen and the game begins, all of the interplay occurs through the ndial’s lights, buttons, dials, and switches, requiring players to focus on the matter at hand. Performances are governed by a 42 minute score of animating colored light that dictates which players are active. Players are assigned to colors and begin each turn at their home ndial, but they are free to move to any open ndial provided there is a clear path in the clockwise direction. Collaborating with each other as well as the program, players must navigate unpredictable musical patterns in search of strange trajectories through familiar realms. Music out of order. This chapter explores the boundaries, structures, components and interplay of the system and reflects on the processes and decisions that led to this form. 69 Terms We look to the sky to reconsider, 'bring together the stars again' (from com - "with" + sideris - "constellation"), to re-constellate. We think with the stars to navigate, to visualize, and to overwhelm our senses such that we are forced to figure again, like an infant gazing into the patterns without expectation. A pattern, a perceived regularity, repeatable in some sense and in its repeatability, meaningful. It is what is remembered when an experience is over—a record is a readable pattern. Pattern is from the outside in. Environment Human. Light reflects the structures of our meaning-making, writing the world in electron patterns that trip unique circuits and produce new connections and patterns. Light is and isn’t, feedback and control—both and rather than either or. System is a focus or frame created by drawing boundaries around a portion of the Universe. System considers interplays removed from the Universe’s infinite tangle of intra-actions. System is a lens for making patterns and a method for quantizing the flow of things. System is from the inside out. Human Environment 70 I stare blankly at the ceiling in a large black box studio. Cables dangle from the lighting grid and I am lying on my back in the middle of a partially-installed system of copper pipes, half-built pedestals and small wooden discs that still need to be painted. I finished the instruments on Monday, the lights on Tuesday and spent all of today debugging the software and running wires up to a lighting grid using a scissor lift. The objects are starting to materialize, but I’ve put off some big decisions. I’m not sure what is going to actually happen here in 3 days. I have a system, but not a piece. Micaela is nearly finished with the release forms. I feel the energy of a curious audience. I have a title and a group of players, attendants, and operators, but the system is without an interplay. The interfaces are sequencers and have a built-in intraplay but it is brand new and as of yet, unexplored. Our first rehearsal was a mess. We need something to push against— a system for organizing the play—rules. “When forced to work within a strict framework the imagination is taxed to its utmost – and will produce its richest ideas. Given total freedom the work is likely to sprawl.” Who said that? Play does not happen in a vacuum, it needs something to play against. I’m kicking myself for not making something smaller, more contained, less spacetime, different kinds of risks. A system for reconsidering the pattern in light of itself is pretentious. Why always so reflexive? Why not just clear and honest. Is clear honest? What pattern? Which system is this referring too? Why not just one single light in the middle of a dark room. That would be so nice. Just one pixel. Tick. Tick. 71 I was lying there staring up at the ceiling, and Simon was up there staring down at me… Figure 49 : Simon It shouldn’t have been a surprise. He has appeared in visions before—in similar situations... Figure 50 : A Familiar Face Two years earlier I wrote a program note the night before my masters thesis performance. I called the piece Simon. If this new piece’s circular title was not already fixed in emails and poster designs, it might be called Simon as well. The program note from Simon (2010)… 72 Simon (2010) - Program Note In 1978, Milton Bradley released the electronic memory game Simon during a party at Studio 54. It was hugely success and quickly became an icon in 80s American pop culture. In its most common mode, Simon's four buttons glow in sequence with corresponding tones. The player mimics the sequence, which increases in length with each successful repetition. It is assumed that Simon was inspired by the simple children's game Simon Says, an account supported by Milton Bradley and inventor Ralph Bear. However, as years went by, a different story has emerged. Simon was a pet robot, an after hours project initiated by a few of the engineers at Marvin Glass and Associates in 1976. The goal was to create a machine that would learn by mimicking patterns, making decisions based on sensors that listened to the environment. The project was surprisingly popular. Within months, Simon was joking with his engineers and picking out colors for prototype designs. We had created a simple learning algorithm that listened and repeated sounds based on different criteria. By spring, he had developed an impressive pattern vocabulary and his responses seemed to get noticeably more bold. He would beep aggressively and interrupt conversation and once seemed to actively confront a client who was visiting the office. He audibly resisted reprogramming and would sometimes go mute for days and then for no reason, repeat a few broken sequences before returning to his dormant state. It was a busy time and most of us were too busy to look into it. We figured some connections were loose or a chip was failing. But on July 26th he seemed to return to normal, responding cheerfully even and conversing with anyone who walked by. The next day, Al Keller, a junior designer, shot and killed four employees and injured two others before shooting himself in the head. Simon was found disassembled in Keller’s office, his parts arranged neatly on the desk. In the months leading up the incident, Keller was one of the few designers still working on Simon, often staying late just talking with him. A list of names was found in Keller’s desk drawer. It included the names of all of the dead. The firm, worried about the peculiarity of the situation, didn’t mention Simon in the news story and Keller was described as a paranoid toy designer gone mad. Lenny kept Simon’s parts. About 6 months later he connected the microcontroller to a small set of speakers. It beeped a short sequence of tones and then repeated “Follow me” in Morse code. It took a few minutes to figure out what he was asking us to do. We began copying his patterns of sound and light. 73 Simon in Flatland Simon (2010) is governed by a score of animated colored squares projected behind four sound stations. Each station corresponds with a different color on screen. The score activates stations in a random sequence, directing the player’s movements. At each dimly lit station, I played one sound through a microphone into a system of long feedback delays that folded the sounds back onto each other eventually producing a playful texture of buzzing electric razors, calculator synthesizers and wine glass glissandi. Half way through the 20 minute performance, a square board attached to a rotating platform spins the audiovisual texture around the hall, the motion accelerating until the screen ‘splits open’ to reveal another world of colored squares, these miniature and moving across a white background, mimicking the patterns playing freely through the system. The projection illuminates all of the stations, toys, cables and electronics that were previously hidden and the score follows the player’. Figure 51 : Performance 74 Copy That was the end of Simon. I got the best of him publicly in the old man vs machine showdown, turning his patterns inside out. No more following his flashing lights and sounds. I was reading everything wrong of course. That’s not how technology works. In trying to destroy the system, I fused with it. The primary color schemes, symmetrical geometries, and randomized sequences became the lens through which I produced the world. In using systems we become them, or we become like them. We incorporate their perspectives. After all, that’s what systems are—perspectives. By 2012, I had been formally working with his patterns for about 3 years, and unconsciously since I was about 5. Simon structured my music, colored my videos, designed my interfaces, and organized and reorganized my thoughts. It’s not that strange. People work with computers all the time, it’s just that Simon is lower resolution than most recent digital systems. He is more abstract, more constrained. His program fits onto a chip produced in 1974, 256 bits of ram, bits not bytes. These days, Simon is no longer a hands-on collaborator. He shows up after most of the hard work has been done and asks the same question, every time. Want to play? Ok. Simon. Just one last time. 75 Adding a Dimension Simon Reconsidering the Pattern Light Players Processing transparent Interface VISIBLE Fiction COMPETITION Gameplay NAVIGATION FIGURING OUT (OUT ➛ IN) Figuring FIGURING IN (IN1 ➛ IN2) REFLECTIVE Lens DIFFRACTIVE CLOSED System OPEN PROGRAM NOTE Magic Circle RELEASE FORM   Figure 52: Translation 76 The Magic Circle All play moves and has its being within a play-ground marked off beforehand either materially or ideally, deliberately or as a matter of course. Just as there is no formal difference between play and ritual, so the ‘consecrated spot’ cannot be formally distinguished from the play-ground. The arena, the card-table, the magic circle, the temple, the stage, the screen, the tennis court, the court of justice, etc., are all in form and function play-grounds, i.e. forbidden spots, isolated, hedged round, hallowed, within which special rules obtain. All are temporary worlds within the ordinary world, dedicated to the performance of an act apart. 67 : Johnathan Huizinga, Homo Ludens In Simon (2010), the playground is marked off beforehand using the protocols of a concert. Not just the physical space, the concert hall as a playground, but using a program note. The program note was not to explain the performance as much as to create a space apart from which to experience the piece. This type of framing appeals to me, playing with formal systems of communication to provide a counterpoint to the experience and to set relations into motion. Where Simon’s playground was introduced through a program note that was slightly off, Reconsidering the Pattern was framed by a communication system drawn from doctor’s office waiting rooms and corporate legalese—a different kind of system. This time I enlisted the help of a friend skilled in playing with formal structures and practiced in navigating confused logics. At some point late in summer I first discussed the project with Micaela. I described what I was thinking and she asked a few questions. I don’t remember many of the specifics, but I recall rambling about entangled states and spirals for a while. I felt as though text needed to be a part of this project, but I had very few concrete thoughts. I sent along a few sketches I had been playing with, eight sided stars and tetrahedrons, but that was all. She emailed me in late September… 77 … So what I am imagining is this attached document printed on that long legal-sized paper. Maybe kind of cheap paper, just this side of onion-skin, with one or more of your graphics as very faint blue background to the text. The onion-skinninness so that it would feel like a blueprint as well as a legal release, but also so that the ink from people's pens would blur and distort their initials… … I am thinking that before people could enter the space where your set-up is, they would have to stop in what I am calling the "atrium" and initial and sign this legal release. This is my attempt to Herzogianly pre-frame the experience, as requested. There might perhaps be one or two people there in the atrium to enforce this step. Sexy alien librarian types… … Right now the first part of the release exempts you from liability for spooky action, the second part exempts you from damages resulting from fuzzy logic, and the third part is just weirding. The fuzzy logic section is probably the weakest, because of how I had to try to think about math… The forms attached were the forms we used in the performance, with only a few minor changes. Moreover, the forms were used in almost exactly the way described in the email. In order to enter the space, people had to sign 1 out of the 4 consent forms (see appendix 3 for all four forms), so there were four possible frames through which to experience the system. Each release form acted as a lens that sets into motion different considerations. The first release form (reproduced below) deals with the interpenetrating nature of quantum materiality, living atoms and cosmic connections. Release form II requires guests to consent to being documented, filmed, and played with. Form III asks for adherence to the technical and poetic logic of twisted mathematics and unpredictable relations and the final form IV folds into itself, asking guests to reconsider their identities in light of their fading, imperfect initials. Release forms were administered by a system attendant. Once the paperwork was completed each guest received a small wooden disc that was fashioned from the various pieces of wood and plastic left over from making the interfaces and the records. The discs were used by the system for identification and allowed guests to come and go as they pleased. 78 Figure 53 : The Magic Circle Also at the welcome desk was the first ndial, a simple but more powerful multi-colored ancestor of the ndial interfaces inside of the system. This ndial controlled a sequence of graphics that played on a screen behind the attendants, comprised of fragments from the system’s title and variations on the 8 sided figures that run throughout these pages. A generative synthesizer sequence provided ambient ‘waiting room’ music that pulsed in time with the animated title. Figure 54 : Entrance Animation 79 reconsid for eri RELEASE FORM e system ng the patt f th Read and initial each disclaimer; completion required for entry. ern o in light ____________ Even as I scratch my initials into this form, the letters are already erasing, their outlines degrading. Possibly I, too, have already begun to blur. Here at the welcome station, already, I consider my loss of resolution. ____________ I provisionally agree that my particles will stream away from me, magnetized by the electric soul of the system, burrowing into the flesh of other system users, spinning out into the logical fuzz of the cosmos. I provisionally agree to tremble. Provisionally to shiver. ____________ Still, even as I disappear, I will fill up, chokingly, with new atoms. I provisionally agree to these atoms of strangers, to the atoms of the system. Very soon, they will drive me out of myself. I’ll agree to be a mirror in the dark. ____________ It isn’t altogether agreeable here at the welcome station. This falter in my progress. This quantum stuttering. No, it isn’t quite safe, not even here, already. ____________ But perhaps I haven’t been safe for some time now? ____________ And I don’t believe I’ll ever be safe again, after tonight. ____________ I don’t not believe it, either. For if nothing is safe, then nothing is certain. ____________ I agree that if nothing is certain, here in the system, then nothing is true. Then, providentially, nothing is false. Nothing is proven. ____________ I could toy cruelly with 1, if the system would circle round 0. ____________ The system would be neither if I could be nor. ____________ Without the slightest provocation, the system might step on its own head. I would agree to that! Here at the welcome station, already, I’ve stepped into the outside of the inside-out system. ____________ The system reserves the right to seat me apart from my atoms. The system disclaims responsibility for any faces I see in the noise. ____________ I agree to dissolve. To be lost and dissolute. ____________ I agree to enter the system without my star charts and without provisions, for I agree never to return from it. ____________ Not with these molecules, not through this vertex of the multiverse. ____________ Agreed: The system ≈ nourishment if and not only if I am < > ravenous. ____________ Agreed: Within the system will not be light or dark but bleeding veins of spectral color. ____________ Agreed: The system will be sound and silence; also/rather/someday it will be the thrumming of wind forcing its way down my lungs. ____________ NOT light = (1 - truth(light)). ____________ Light AND sound = minimum(truth(light), truth(sound)). ____________ Light OR sound = maximum(truth(light), truth(sound)). ____________ LIGHT = (1 + truth(light)). SOUND = 42. ____________ I am pretty sure that although truth may agree to be measured in degrees, the fiction of the system is absolute. 80 A Perspective Figure 55 : ● Wooden disc in hand, you enter a dark space and sit in a chair on the near side of the room. The shades are up and Brown’s campus can be seen through the rain. The system is idling in some kind of auto-pilot mode and four players are seated behind each ndial, facing inward. Every few seconds the sound of a slowed down voice is triggered. A small ball of yellow light about 12 feet off the ground glows in time with the sound. After a minute or so, you notice that a yellow light on one of the ndials glows with the sample as well. “I feel like we’re inside of Stockhausen’s Tumblr”68 The shades lower and the room is dark except for occasional points of light. The darkness lasts for too long and you lose track of the room’s dimensions. You wonder if it was smart to consent to “never being safe again, after tonight.” You wonder if the greenish glow is real or if your eyes are playing tricks. 81 Figure 56 : ● No, definitely green and there is the outline of a player standing at the nearest ndial. You hear a few clicks and a slightly buzzy texture begins to grow under the intermittent vocal samples. A pattern of soft synthesizer stabs emerges as the source of both the wash and the steady cycle of greenish white lights that are now circling over head. The reverb fades out and a dry synth pattern moves around the space, punctuated by the lowered voice. The player walks around to the yellow ndial and flips a switch and the voice stops. As the player completes the journey back around to their home ndial, you notice that the green light has taken on a bluish tint and another player has entered the fold. Figure 57 : ● The second player is at the blue ndial and flips a switch. A random pattern of blips and bursts flies haphazardly around the space. Over a minute the pattern descends in pitch until the blips become beats of a scrambled song, playing back in time but out of order. You think you recognize it, and while you’re trying to work it out, you notice the first player holding a mirrored disc in the air. Before you have time to be confused, someone steps out of the darkness and up to the microphone… 82 “Still, even as I disappear, I will fill up, chokingly, with new atoms. I provisionally agree to these atoms of strangers, to the atoms of the system. Very soon, they will drive me out of myself. I’ll agree to be a mirror in the dark.” “-ror… be signal… -ror… be signal… dri dri dri dri dri dri dri to-ror… be signal…” “Nooo, dinary… no, no, no, no, no, no, nooo, dinary” The two loops fall almost into sync and the spoken words pulse in time with Sade. The vocal loop is shorter and shifts by one beat each time around. Now they are out of sync again. The system is a deep yellow. The first two players are sitting down and someone new is on the yellow interface. After shuffling the controller, they press a button. A slow playful clarinet line. Figure 58 : ● The player moves first to the blue box and then to the green box, turning a dial on the front of each interface and removing the high frequencies, leaving a low thumping pulse. Returning to the yellow ndial, the player again triggers the jazzy clarinet line and proceeds to play a strange clarinet solo over the distant beat, occasionally shuffling the interface, changing the samples. The player turns a dial and the clarinet becomes an ensemble, a coherent but unpredictable counterpoint. A red light in front of you is accompanied by a loud 808 clap and you snap out of it. 83 Physical Structure Figure 59 : Reconsidering The system’s physical composition reflects the pattern on the interfaces. Everything arranged symmetrically in groups of 4 or 8 except for the audience, seated in two sections of risers on either side of the space. Depending on the release forms and how one tends to view things, the system looks like an arena, a spaceship, a game-show, sci-fi disco nostalgia, or an sculptural translation of an animated gif. All of these sources were referenced in the design, but more than anything, the structure was derived from an obsessive adherence to a pattern produced by folding two tetrahedrons on top of one another. The result is a star with 8 vertices, 28 edges, 56 triangular faces, and 70 tetrahedral cells. In multi-dimensional geometry, it is the orthogonal projection of a 7-simplex. For me, it is a road map, and became the blueprint for determining spatial structures. With little control over the input and output of the system, the pattern was to give the piece a structural coherence—a lens from which to figure. 84 System Map Figure 60 : The Pattern 85 Legend Again, there are four stations, each associated with an ndial, identical except for the button color— . Between each ndial is a pentagonal speaker from which only the dry sounds of the lone input microphone are projected. All sequenced sound is played through four speakers positioned in the four corners of the space . The colors inside the speaker cones r represent which interfaces send to each speaker .A set of eight tetrahedral lights are arranged in an octagon and hang ten feet above the ground. Each light contains four bulbs that correspond to one of the glowing buttons on an ndial . Players move clockwise between stations . Sound moves both clockwise and randomly through the speakers . The design of the system combines the circular symbol of order, the whole with an eight pointed star, the symbol of chaos . 86 Pedestals & Pyramid Figure 61 : Models The ndial pedestals and central pyramid were built by Peter Scheidt, a sculptor and furniture designer who was also helpful in the ndial design process. Each pedestal is built with two shelves inside. The top shelf holds the records (explained later) that players use to change galaxies, check world order, and call for live input. The bottom shelf holds a computer. Once set up, there is no need to interface with the computer except for through the ndial. The pedestals are designed to accommodate all of the various cords and cables for running audio and light to and from the central pyramid, the structure through which all signals ultimately passed. The pyramid at the center of the system is designed to hold a mixer, speaker amplifiers, and the LED drivers. Each ndial sends audio and lighting information through the pyramid that is redistributed to drive the appropriate speakers and lights. 87 Light as Control & Feedback If you know how to light, you can make shit look like gold. I paint, I build, I compose with light. Light is a magic wand. 69 : Robert Wilson, quoted in The Theater of Robert Wilson Experiments with colored light were a primary focus during the early stages of this project. The ndial was originally designed as an interface for performing with light and sound simultaneously and the mapping of sound as light (and time onto space) remained an important part of the project. Light is often used on the electronic music interfaces to communicate the internal state of the machine to the player. Light is also often in used in systems as a means of focusing attention and limiting interactions to a specific area—a streetlight is a good example, so is lighting design in theatrical productions. Here, light is used as both a feedback mechanism and a control mechanism. Points of colored light arranged around the space articulate the sound patterns produced by each ndial and project the steps of the interface at the scale of the space, providing audience members with a way of visualizing the interacting interface patterns and framing the players as inside the interface. The macro-structure of the performance is controlled by colored light that functions as both a score and a ‘mood generator’. It communicates the structure of the performance-game to the players and produces different colored ambiences through which to read the system’s navigation and experience the musical artifacts. 88 Figure 62 : Frames 89 The light score is determined before each performance by first rolling a six-sided die to get a number of minutes and then flipping coins for each player. For example, if the die reads 4, red and yellow flip heads, and blue and green flip tails, the section lasts for 4 minutes with red and yellow playing. For the last 20 minutes of the performance there are only time cues and player combinations are determined by the light operator. 4 – G 1 – GRBY 3 – GBY 2 – 5 – GR 5 – 6 – RB 3 – 1 – RY 1 – 5 – Y 2 – 4 – B 3 – 3 – N F – GRBY Figure 63 : Light The light sequence was controlled by Kim Young from a lighting board in the corner of the space. Kim controlled the brightness of four spotlights (Red, Green, Blue, Yellow) that set the tone of the environment and communicate which players are active. The two functions are related, when blue and red are active, the system is purple. Transitions are up to the operator. The light could change quickly or slowly evolve over the course of a minute. About halfway through the performance both the light and the volume fade out together. The only sound that remains is the sound of the four interfaces—switches and buttons click and points of light fly around the room. After 3 minutes, all of the lights fade up quickly and the players, with little knowledge of what samples are active, turn their volumes to full and begin to explore the chaotic texture. For the last 15 minutes, Kim has more control, improvising the player combinations. 90 Light as Feedback – Visualization System Each ndial’s 8 steps are broken out into 8 lights arranged around the system, so while the player is manipulating sounds on the ndial, they are also inside of the visualization they are staring into. The lights simply communicate when samples are triggered by the program and are not affected by any manipulations made to the samples through the interface. Therefore, the lights simply visualize the cuts, projecting the interface at the scale of the space. Figure 64 : Low Resolution Screen Each tetrahedral light includes one step from each ndial interface so that the distinct sequences of each ndial become entangled in the light ring that surrounds the space. The sound produced by the four ndials is similarly entangled and the lights become a way of exploring the sounding relationships between the four players. 91 Figure 65 : Entangled Each light is made from half inch copper pipe, welded together to form a tetrahedron. At each vertex is a colored LED housed in a ping pong ball that nicely diffuses the light. The lights are triggered by the interfaces themselves, and simply mirror the circuitry of the glowing buttons on the interfaces. Figure 66 : Tetrahedra 92 Players : Performers Figure 67 : bumpr bumpr (Peter Bussigel, Stephan Moore, Caroline Park, Tim Rovinelli) is an improvising ensemble that folds outer space drones into pulsating geometries, combining acoustic instruments, digital processing and noisy handmade objects to produce playfully experimental inter-stylistic chunks of spontaneously sculpted spacetime. 93 Records Players choose their own galaxy of sound worlds to navigate. Each ndial station has a reference record that serves as a map, helping other players navigate the foreign galaxies. For Re (2012), each sound world could be re-written with live sounds, but when shuffled, the ndial redraws samples from the stored sound world, over-writing the live samples. Below are sample reference records from the Thursday night performance. 1. Ryoji Ikeda track “Superhelix” 2. Close mic’d piano samples 3. Classic 808 drum machine samples 4. Octavluv, a track by Caroline Park 5. Assorted gong samples 6. The title track from the film “Titanic” 7. Hundreds of different kick drums 8. Concrete PH by Iannis Xenakis 1. A pop song by Taylor Swift 2. Metal track by Teebs 3. Horribly played Banjo 4. 80s electro-banjo synth 5. Assorted sounds of water dripping 6. Dub synth samples 7. Sample bank of car horns and crying babies 8. Glitchy noise improvisation 94 1. Song by Belle and Sebastian 2. Creaky door samples 3. A flute improvisation 4. No idea 5. Einstein on the Beach 6. “1979” by the Smashing Pumpkins 7. A collection of drum samples 8. Music by Gavin Bryers At any time while they are active, players may hold up a mirrored record, which indicates to the other players and the guests that they are accepting live input. This procedure is explained to the audience at the welcome desk. Anyone may approach the microphone and make sounds into the system. Over the course of the three nights, a range of different live sounds were sampled. A few of them are listed below. 1. Live ukulele improvisation 2. Improvised conversation 3. a dramatic reading of the release forms 4. ear splitting melodica clusters 5. A reading from a philosophy text 6. Long sustained singing 7. tapping the microphone 8. a shower radio Figure 68 : Records 95 Systems Esthetics Conceptual focus rather than material limits define the system. Thus any situation, either in or outside the context of art, may be designed and judged as a system. Inasmuch as a system may contain people, ideas, messages, atmospheric conditions, power sources, and so on, a system is, to quote the systems biologist, Ludwig von Bertalanffy, a "complex of components in interaction," comprised of material, energy, and information in various degrees of organization. In evaluating systems the artist is a perspectivist considering goals, boundaries, structure, input, output, and related activity inside and outside the system. 70 : Jack Burnham, Systems Esthetics For Burnham, a system is defined by “conceptual focus rather than material limits.” Systems are marked off by a subjective boundary concept that encapsulates a complex of interacting components. This type of boundary thinking allows one to isolate part of the Universe as a way of considering certain behaviors. Here we find a similarity between systems and play—both are concepts that take place in a space set apart from the actual or ordinary flow of things. But, both system and play leak into the ordinary. They are often designed that way. Just as interfaces are never fully transparent, boundaries are never absolute. A spectrum of permeability is helpful for thinking in systems. On one hand we have closed systems, purely theoretical, in which there is no transfer of information between inside (system) and outside (environment). The opposite is an entirely open system. Here the boundary dissolves, producing chaos, the Universe, or simply the ordinary flow of things, depending on your perspective. The frames provided by systems thinking, like the magic circle that encapsulates play, become ways of analyzing the world. Returning to our comparison of Simon and Reconsidering the Pattern, we can think about the degree to which each system exchanged information with its environment. 96 CLOSED OPEN Simon is a predominantly closed system. In order to make this claim, I draw a boundary that places the audience outside of the system, in the system’s environment. Sure, microphones pick up noise from the environment and certainly the energy produced by a live audience has an influence on the system, but there is no mechanism designed for transferring matter between the system and its environment. The sound materials are originate from physical objects and are predetermined as part of the system. The range of outcomes was relatively narrow. Reconsidering the Pattern is a more open. The primary input materials, the sound worlds, are left to the discretion of the players and the audience. While the players themselves are part of the system, the material they bring to the system is of the environment—a reading by Alan Watts, pre-recorded Arp creatures, 3 minutes of radio static, the outtakes from an interview. There was also an open microphone through which guests of the system input sound materials for reconsideration. Someone read words from a text about narwhales and another guest brought a ukulele and strummed into the microphone. Burnham’s description of systems as conceptual frames can be a useful perspective for thinking about performing with a digital interfaces. To view a performance as a system entangles the processes and the artifacts of play such that fixed value judgments like ‘good’ and ‘bad’ might give way to a more fluid and active forms of listening and engagement. Animating interfaces propose that everyone is on the spaceship. The performer is in certain ways as clueless to the outcomes as the audience. For this reason, performing with the ndial is both freeing and terrifying—a navigation that I have come to think of as nterplay. 97 NTERPLAY Figure 69: nterplay The computer's most profound aesthetic implication is that we are being forced to dismiss the classical view of art and reality which insists that {one} stand outside of reality in order to observe it, and, in art, requires the presence of the picture frame and the sculpture pedestal. The notion that art can be separated from its everyday environment is a cultural fixation [in other words, a mythic structure] as is the ideal of objectivity in science. It may be that the computer will negate the need for such an illusion by fusing both observer and observed, "inside" and "outside." It has already been observed that the everyday world is rapidly assuming identity with the condition of art.71 : Jack Burnham, Systems Esthetics Even as I scratch my initials into this form, the letters are already erasing, their outlines degrading. Possibly I, too, have begun to blur. I consider my loss of resolution. ~ Micaela Morrissette, Release Form II 98 Performing Probability In a 1947 letter to fellow physicist Max Born, Albert Einstein wrote, “I cannot seriously believe in it because the theory cannot be reconciled with the idea that physics should represent a reality in time and space, free from spooky actions at a distance.” The uncertainty proposed by quantum mechanics challenged Einstein’s attempts to precisely model the world. He wouldn’t accept that the universe could only be described as a probability. If the states of entangled particles are indefinite until observed—if the very act of observing alters the outcome—then how can anything be figured out? Even attempts to define the relationship between frame and framed are subject to the distortion of the lens. Einstein held his ground as a new generation of physicists shook their heads—a hero past his prime. But what Einstein was clinging to was not a flawed notion of the universe, but the practice of Science itself. He refused to rely on hidden variables—spooky actions are unacceptable answers. He heard patterns. Neils Bohr had a deep appreciation for both Einstein and the quandary that this new theory suggested, “When it comes to atoms, language can be used only as in poetry. The poet, too, is not nearly so concerned with describing facts as with creating images. It is wrong to think that the task of physics is to find out how Nature is. Physics concerns what we say about Nature.” 72 Physicists of the 20th century found themselves playing dangerously close to the mysterious signals of artistic expression. 99 Low Resolution Digital Play nterplay is a navigation drawn from the subatomic and translated to the medium—medium as in medium-sized and with medium accuracy. The electron’s confusion inspires a processing system of a human scale that reanimates records through an indeterminate lens—music, in time but out of order. To truly play with a program, an audience is also asked to ‘figure in’, shifting attention between agents to make connections and produce meaning. But in order to play meaningfully with a computer, its electrons must be slowed to the pulse of human experience. Or, human experience must be accelerated to the pulse of the electron. It seems we are implementing the later approach. Low Resolution Digital Play is an argument for the former. Simon is a low resolution game brought to life through an interplay in which player and program communicate with each other, listening and acting. The interface constrains the program’s processing to a rate that interfaces with human processing. In 1981, Simon’s pattern play resonated with the cadences of thought and allowed the player to see more than just a collection of logic gates enclosed in plastic. nterplay requires similar constraints, such that the dialogue between player and program can be experienced through sound and light. Low resolution because the program’s decisions are bit crushed to a pulse that can be felt, requiring both player and program to conform to the interface. nterplay requires listening and experimentation instead of masterful expression and impressive design. It is like a game—and performing with an animating interface is a public navigation with an unpredictable co-pilot. All of these tensions can be read on the surface, and the audience is along for the ride. 100 Figure 70 : nterplay 2D From the observer’s perspective, nterplay can be reduced to the interactions between player, program, and sound. If you know what the program is doing and know the sound material well, you might tend to focus on the decisions of the player. This is position { c } in the figure above— looking through the lens of that which is known {program, material} to focus on the mysterious agent {the player}. But, if you know the player and the program well, it is the material that draws your attention { position a }. While this model presents a method for thinking about how one might attend to a system, the agency of the audience is not figured in. The figure below adds the audience to the picture creating a volumetric system in which attention animates along three axis. Here, focus moves between vertices but also expands and narrows, navigating between total immersion and distant reflection. Figure 71 : nterplay 3D 101 A listener might collapse into one of the other vertices—closing their eyes and aligning themselves fully with the sound material. Or, becoming the material by imputing sound live. Another listener could perfectly align with the yellow circle and become a player. It might frustrate Einstein, the ambiguity. But, it can also be thought of as an affordance of figuring—the ability to play with and transform the systems with and within which we live. Figure 72 : system ystems emstys metyss The system gets increasingly messy the more you try to figure it out. For example, when considering the multiplicity of perspectives, systems take less regular forms. Add more players and materials and at some point what began as a system appears as a chaotic flow. This is the functionally wonderful nature of simple processes: they can help us explore complex ideas. Attempts to figure in all of the perspectives begin to echo Bohr... When it comes to perspectives, figuring can be used only as in poetry. The poet is not nearly so concerned with describing facts as with creating images. It is wrong to think that the task of figuring is to find out how things work. Figuring considers how we work with things. 102 ANIMATING INTERFACES Figure 73: Animating Interfaces Ambiguous displays invite interpretation. Mysterious signals demand investigation. Curious interfaces provoke play.73 103 The ideal digital interface is often described as transparent, an infinitely thin membrane74 that transmits messages between people and programs with a minimum of distortion and effort. But attempts to produce truly transparent interfaces fail, not necessarily because messages are distorted by the interface but because interfaces change how people think. As we incorporate the perspectives of our digital systems, our perceptions change. The goal of transparency in interface design betrays a desire for efficiency. As I press “n” on my keyboard, “n” appears on the screen. There is no reason to engage with the various technical processes mediating between my action and the letter appearing on screen and so these levels of interfacing are invisible, or transparent. The computer keyboard produces a desired outcome with a minimum of cognitive strain. When typing words or landing an airplane, this type of efficiency is undoubtedly helpful, but there are situations where efficiency is not what we want. Should music and games—or play in general—be efficient? And perhaps more to the point, how does this governing standard of transparency and efficiency influence the way people engage with the world? It is worth thinking about who stands to benefit most from efficient interfacing. Efficiency requires a known goal or a predetermined destination, limiting the range of outcomes to produce the satisfaction of arrival. Efficiency can get you there faster and more reliably, and is instantly available at your fingertips for $2.99. But on arrival, there is another destination, and another app to help you on your way. One can always be more efficient—it keeps things in order and under control. Efficiency produces not only predictable outcomes, but predictable users. It works both ways. 104 The constraints built into the interface determine the cognitive, behavioral, and epistemological constraints available to its user. Under such constraints, human activity becomes habitual, circumspective—the human becomes almost as automated as the thing s/he is using.75 : Michael Hamman, The Technical as Aesthetic Although the politics and effects of Digital Interfaces proper are intimately related to these thoughts, the scope of this project is not so broad. Here, I am interested in digital interfaces that foreground their effects—interfaces that are obstinate, that push back and outwardly refuse this myth of transparency. By embracing complexity and ambiguity, our interfaces become more than just tools. They become systems for play. Interfaces for exploring sound are not designed to land airplanes yet they are still engaged with the tenets of efficiency and transparency. My hope is to explore the potential of new musical interfaces to respond to Alexander Galloway’s call for a “counter-aesthetic to systemic efficiency”—a mode of music making that is less about productivity and predetermined destinations and more about active listening and transformative play. From the last chapter of Galloway’s Interface Effect… There is one game in town: a positivistic dominant of reductive, systemic efficiency and expediency. Offering a counter-aesthetic in the face of such systematicity is the first step toward building a poetics for it, a language of representability adequate to it.76 Animating interfaces are inefficient and unproductive. They don’t create immediately valuable products or meet specific predetermined goals. Instead, they speak to the frustrations and transformative potentials of friction, unpredictability, and play. 105 ...an interface that offers no resistance is one in which the user only performs actions that are known, that are already familiar. A well-designed symbolic interface will disappear, cease to get in the way, but this ease is purchased at the price of creativity. To generate the new requires a resistance, the interface must push back, make itself felt, get in the way, provoke or problematize the experience of the user, who, confronted with a problem, innovates in order to solve it. The outcome is unknown; the experience is experimental; the process is improvisatory. 77 : Aden Evans, Sound Ideas Critically engaged interface designers in a number of different fields advocate for this type of resistance. Composer/performer Joseph Rovan’s obstinate interface is not “merely a passive receptor of a performer’s input, but must define a center of activity all its own. To be truly playable, in other words, the instrumental system must exhibit a certain degree of what I like to call obstinacy.”78 Performer Laetitia Sonami’s personal guidelines for designing musical interfaces cut straight to the point, “Adaptable, Inefficient, and Unreliable.” Game designer Jane McGonigal supports “ambiguity, multiplicity, and open-endedness in design, so that we perceive, rather than receive, our technologies”. In her short essay The Curious Interface, she concludes that “when what surrounds us confounds us, when the computable tends toward the inscrutable, only then we will become strategic, resourceful, poetic actors.” 79 Digital theorist Aden Evans frames the problem of immediacy, akin to transparency, “The dream of immediacy forgets the effort by which desire is forged into expression, the friction essential to this generation.”80 Evans finds resistance to be integral to digital expression, calling for machines that actively pose problems. To become an expressive instrument, to allow the generation of ideas, the computer must not disappear, neither into the sensation nor the desire of the user. On the contrary, the computer must become resistant, it must become a machine for posing problems. 81 106 The problems posed by animating interfaces can be reframed as features that underscore all kinds of life, artificial and natural—they are surprising and don’t always act as expected. While efficient interfaces seek to smooth out the noisy signals of life and play, animating interfaces embrace them as a feature. In navigating the unpredictable we explore our own tendencies as well as those of the programs and the materials with which we play. We find ourselves embedded within our systems, actively reconsidering patterns and figuring in. The ndial is an animating interface. The lights on its surface animate in a technical sense, resolving as motion in time. But here, animation is primarily considered in the poetic sense, animating as the act or process of bringing to life, seeing a system or program not as simply functional or useful, but as a vital creative agent. By manipulating buttons, dials, and switches, I guide the program’s decisions which, in turn, produce automated patterns of sound and light that affect my actions. The source material—a live recording of this paragraph, a popular song, every kick drum sample on a computer—is also animating, perhaps reanimating—given new life in the interface. While acoustic instruments are brought to life as physical extensions of the voice and body, electronic instruments act in consort with their human players. George Lewis writes, “in certain contexts ‘performance with’ may be more empowering than ‘performance on.’”82 While the ndial is an interface for playing with sound, its primary purpose is not the production of music. Music is a possible artifact, but the ndial will never be the most efficient method of expressing a particular musical idea. Virtuosic editing practices are traded for the meandering process of experimentation and discovery. Efficiency is replaced by animation—being moved— sailing in a possibility space rather than speeding towards a predetermined goal. 107 In a sense, I am arguing for taking the time to explore spaces that are not designated for exploration. To drift between success and failure through strange worlds selected out of curiosity—to get another view, to find new patterns, and to explore the space of possibility where program, player, and material come to play. A musical work can no longer be accounted for purely through examination of the acoustical experience it engenders or the formal structure it may exhibit. As technological, the work constitutes both the result and the technological forms by which the result was realized. These include the particular technical tools plus the attitude of the subject under whose unfolding those tools were taken up and applied. To equate music solely with the results of its productive activity is to disembody the result from its technique—it is to fetishize the musical work, converting it from a catalyst for experience into a commodity to be traded within an economy, whether financial or ideological.83 : Michael Hamman, The Technical as Aesthetic Simply put, animating interfaces energize relationships. A relationship is not a product, but a process of continuous co-navigation. Relationships require time—they emerge in time. Animating interfaces are not simply thresholds to pass through, but playgrounds for taking the time to explore unpredictable connections. And while the artifacts of this play may very well be musical, the navigation is of equal interest. Sound becomes more than simply a product. It becomes a lens for gaining new perspectives, and a medium for figuring within. The Sublimely Affective “Effect”: The alien spaceship not as wonderfully functional, but functionally wonderful—a merry-go-round of light, color, and music.84 Vivian Sobchack’s description of the UFO from “Close Encounters of the Third Kind” playfully frames a simultaneity that is central to the animating interface—it is both an instrument for expressing and the site of that expression. It is both effective and affective, wonderfully functional and functionally wonderful—entangling sound, player, and program in a system for playing with systems. 108 The ndial serves as a model for figuring the animating interface. I have chosen four polarities through which to navigate. Each polar set allows for movement along a spectrum that relates to interfaces and interfacing. These spectra are not descriptive of animating interfaces, but simply constitute a lens for thinking about them. Together, they outline a possibility space for exploring the interface as “an aesthetic form in itself... rather than as a functional tool for making art.”85 Figure 74 : Animating Interface PLAY ATTENTION SIGNIFICATION ABSTRACTION 109 PLAY INSTRUMENT GAME Play is free movement within a more rigid structure... Transformative play is a special case of play that occurs when the free movement of play alters the more rigid structure in which it takes shape.86 : Katie Salen & Eric Zimmerman, Rules of Play Playing the ndial, one is constructively caught between the expressive nature of musical play and the unpredictable exploration that is playing a game. In musical play, structural components— genres, forms, instruments, modes, halls, patterns, settings, etc.—are traditionally heard in support of music’s expressive capacities. A message moves from the inside (within the body of a composer, performer or recording), through the instrument, to the outside (the environment, the listener), and the structure serves to guide or support this message. In other words, structure is revealed through the musical signal and emerges over time. Even considering scores, automated programs, and other pre-existing structures, play (or playback) is required for a musical structure to be experienced. Varèse held that “music is organized sound”, not that music is organizing sound. Traditionally, musical structure emerges through listening to the artifact of musical play—music. In a game, it is structure that is typically foregrounded and visible—the rules, the lines of the tennis court, the racetrack in a videogame. Play moves through these structures, expressing a trajectory within a range of possibilities. The world is defined beforehand and the player is tasked with carving a path inside of it. In games, play emerges through structure—or structure allows for the condition of play. 110 This dichotomy is surely an oversimplification, but it draws a starting line along which to animate. In considering musical play that reflects the navigational qualities of a game and game play that approaches the expressive nature of music, these structural distinctions begin to blur. And, in the case that structure is simultaneously expressed and navigated, play is neither of the player (music) or through the world (game). It moves between, expressing a system that is continually animating or transforming. Transformative play is a special case of play that occurs when the free movement of play alters the more rigid structure in which it takes shape. The play doesn't just occupy and oppose the interstices of the system, but actually transforms the space as a whole.87 : Katie Salen & Eric Zimmerman, Rules of Play It is in moving between formal protocols that interfaces transform from functional tools to aesthetic systems. Are we making music or playing a game? Or, are we perhaps in an experimental space of discovery in which the rather fixed protocols of game and music are overwhelmed by their functions as exploratory and expressive technologies. In moving along the spectrum of play, formal protocols (game, music, sport, theater) begin to converse, revealing a space of relations where the conventions of multiple mediums are put into play. What if some games, and the more general concept of “play,” not only provide outlets for entertainment but also function as means for creative expression, as instruments for conceptual thinking, or as tools to help examine or work through social issues?88 : Mary Flanagan, Critical Play Through the musical lens, animating interfaces might question traditional paradigms of performance, reconsidering virtuosity and expression not as wholly human acts, but expressive of the systems within which they emerge. Through the lens of games, animating interfaces push against the goal-oriented trajectories of competition such that less predictable outcomes might be revealed. Winning and losing, sure, but also playful navigation and experimental figuring. 111 ATTENTION FOCUSED ENACTIVE I find multiple advantages in the term attending. By moving away from an explicitly sonic term, it leaves space to acknowledge the degree to which non-sonic phenomena can and do influence listening and composing. It also prevents listening from becoming a receptive act and focuses on an active process — attending is something one engages in.89 : Matthew Warne, Listening and Reflection in an Electroacoustic Compositional Practice The harmonica, the theremin, and the modular synthesizer encourage different types of attention, but each requires some type of effort to play. Playing music has traditionally required movement that is directly related in some way to the sound produced. The computer requires a reconsideration of this relationship. When using the computer as an instrument there is a disconnect between the physical actions of the player and the production of sound. This separation is mitigated through a process of mapping. Much has been made of this rift. Some mourn the lack of physicality, saying that effortful action is necessary for effective musical expression. Others see the disconnect as an affordance that allows for new forms of expression, dismissing the desire for embodied play as nostalgic. Both camps privilege different types of attention. Proponents of physicality argue for enactive performance practices and polymodal attention, believing that “it is essential to think hard about the physicality of an instrument, how it should present itself to the performer.”90 Those excited by the potential of the computer to allow for new motionless performance practices argue for reduced listening, perceptual focus, and auditory immersion. 112 For Kim Cascone, the spectacle of physicality in electronic music produces a “distracted reception” in which “audiences consume music as a commodity and less as an artform.”91 He quotes Francis Dhomont’s writing about acousmatic music, “What the absence of visual identification makes anonymous, unifies and prompts a more attentive listening.” 92 Cascone’s critique of physical spectacle is aligned with a critique of capitalist modes of production. Interestingly, arguments for physicality take aim at the same target, attributing the disembodying effects of the laptop to the commodification of musical value, normalizing risk and conforming to predetermined standards. As music-making becomes a more intellectualized and less bodily, intuitive process, experimentation too is deemphasized. Accidents and mistakes can always be corrected, which encourages musicians to try all sorts of things in the initial phase of the project, but it also allows them to weed out of the final product the experimental, anything offensive or uncomfortable, to effect a conformity to prevailing standards.93 : Aden Evens Newton Armstrong recognizes that “on both sides of the argument there is a suggestion that something is missing. What distinguishes one side from the other is where that missing something is located: with the performer, or with the audient.” 94 The animating interface proposes that what is missing might be explored by animating between focused attention (reduced listening) and expanded attention (embodied listening). By moving between these extremes—the motionless performer and the physically engaged body—attention becomes part of the play. Sound can engage different listening modes within a single piece. A recent ndial set in a living room began with a sequence of chaotic shuffling, moving rapidly between textures articulated by dramatic twists of the interface. Over time, activity slowed and ultimately settled into a meditative pattern of random tones that stuttered around the space. I dimmed the lights, sat down, and listened. Sound can play as both object and interface that can focus and/or expand attention. Requiring sound to conform to one mode or another, is itself a type of commodification. By animating our attention, we attend to the relationships that emerge between these perspectives. 113 SIGNIFICATION PATTERN RANDOM I would rather chance a choice than choose chance.95 : Lou Harrison or Earl Brown rather chance. choice would than a chance choose I : Lou Brown There is a tendency to privilege pattern over randomness, refiguring the output of chaotic processes as the product of conscious design. Perhaps this is a built-in technology, to hear patterns and make connections. Without the information necessary for rational thought, people are able to jump to a different cognitive domain and momentarily “catch hold of the fact that experience is not a passive interior state, but a mode of active engagement with the world.” 96 Leonardo DaVinci was fond of a phenomenon that came to be termed pareidolia, perceiving significant relationships within random or vague data. Offered as “a way to stimulate and arouse the mind to various inventions”, DaVinci wrote that “if you look at any walls spotted with various stains or with a mixture of different kinds of stones... you will be able to see figures in quick movement, and strange expressions of faces, and outlandish costumes, and an infinite number of things which you can then reduce into separate and well conceived forms... it comes about as it does with the sound of bells, in whose clanging you may discover every name and word that you can imagine.”97 Pareidolia exposes the entanglement of sense and reason, calling the technologies of both body and mind to aid in figuring. This is similar to the concept of “magical thinking” in psychology, where connections are made in the absence of conscious rational thought. 114 It is natural to assume that there would be an evolutionary advantage in always using logical and rational methods of thinking to analyze a situation. However in some cases, a perfect logical rationality is sacrificed for more important advantages; as is the case with heuristics. Heuristics, or probabilistic rules of thumb, are biases used to make judgments and decisions about the world. They lack the accuracy of complete rationality, which cause them sometimes to be wrong; however despite their flaws they have two major evolutionary advantages. Heuristic biases are very fast to employ, and they typically work. In terms of evolution, being able to make extremely fast decisions which are correct most of the time is better than always being right but being very slow. : Thomas Markel98 The ndial uses two types of chance processes to play along this spectrum. First, samples are randomly pulled from a sound world and mapped to the interface. The player has no fine control over the patterns, pushing pattern-making potentials to find relations and meaning between non- continuous times. Additionally, players can vary the amount of unpredictability within the sequenced patterns to add life to the pattern. Animating between the knowable (pattern) and the unknowable (random) we are forced to figure anew. Chance in the arts provides a means for escaping the biases engrained in our personality by our culture and personal past history, that is, it is a means of attaining greater generality. 99 : George Brecht, Chance Imagery While chance processes provide an escape from certain types of cognitive bias, they also are a way of exploring our biases, figuring them in and figuring within them. Animating interfaces are surprising, taking advantage of the computer’s ability to generate unpredictable variables. Not unlike the way a card game relies on shuffling or Simon uses pseudo-random sequences, the ndial uses chance processes to listen differently and play with materials in unexpected ways. For a computer to transcend its mere usefulness and become a vital creative agent, it must pose new problems and motivate new modes of meaning making. This is what Hamman refers to as semiotic rather than symbolic engagement.100 Evans expands this idea, “to extend into the semiotic, a program must allow the user not only to manipulate objects whose meanings are predefined and familiar but also to generate new meanings, to create objects that are unfamiliar.” 101 115 ABSTRACTION VIRTUAL ACTUAL On its own, the digital is indeed confined to abstraction, sacrificing fertility for perfection, innovation for predictability. Yet the digital is not on its own, as it engages constantly with the human world of actuality.102 The violin provides a vast possibility space for expression, but it does so through limits. The chaotic interferences as the many hairs of the bow vibrate the string give the violin its specific character. The player learns to use these constraints and imperfections to make air particles vibrate in highly nuanced ways. A player does not ‘sing through the violin’, but ‘with a violin’ and the music is as much an exploration of rules and resistance as it is freedom and design. Like any instrument, a musical instrument is a means. The player makes sound by means of the instrument, which transduces force into vibration. But a musical instrument is no mere means: it does not disappear in its use. The musical instrument remains opaque, and one does not know how it will respond to a given gesture. Thus, to play is to learn (to play), and one invents in concert with one's instrument. If there is a worry about digital instruments, it is therefore not the worry that they are instrumental—for surely they are—rather, the worry is that digital instruments are merely instrumental, nothing but means to ends. Does an electronic or digital instrument offer always a new challenge, or is it comprehensible, predictable, knowable?103 : Aden Evens, Sound Ideas Programs like Word, Photoshop, and Pro-Tools are wonderful systems for design processes where the objective is predetermined—with an image in mind, one can probably reproduce it in Photoshop. General programs are optimized for (re)producing preexisting forms and their interfaces betray these tendencies. Pro-tools emulates track-based tape editing, Photoshop has a virtual “canvas”, and I am typing on a “page”. General systems use interface metaphors that allow users to work intuitively and efficiently within models that they already understand. But more often than not, they privilege figures that have already been figured. 116 In seeing the computer only as an instrument or tool, we model pre-existing systems in a virtual reality that leverages the mutability of digital data to increase productivity. But this view of the computer as merely virtual reality simply repeats the material hierarchies that persist in the physical world. Anything is possible, as long as it is already possible. To play expressively with the computer, to hear what it has to say, one requires not virtual reality, but what Evans calls “real virtuality”104, an environment that is not pretending to be like reality. Real virtuality allows players to explore processes and structures that would otherwise be impossible to realize. To engage in this way, the symbols and icons of metaphorical interfacing in which the digital is wrapped must give way to points of collision between the noisy and specific signals of the real world and the infinite potentials of data processing. I can type in Word without encountering the computer, but what if every time I blinked, it rearranged my block quotes? ...the real promise of the computer can be discovered only through a systematic empiricism which opens as many channels between the world and the ideas of the composer as possible. A method which allows for at least a few direct, non symbolic relations to the process of composition has a tremendous utility.105 : Joel Ryan, As if by Magic In working with computers, there is a tendency to drift towards one of the extremes—the allure of potential promised by the virtual or the satisfaction of production guaranteed by the actual. But attempts to produce ‘everything machines’ invariably lead back to the environments in which one is working, and overly specific systems become stagnant and predictable. For the computer to be transformed into an expressive agent, one must create rules and interfaces that allow for movement between these extremes. Real world handles for manipulating specific variables of virtual processes give the player something to play with. Rules are important to free expression. I shall go even farther: my freedom will be so much the greater and more meaningful the more narrowly I limit my field of action and the more I surround myself with obstacles. Whatever diminishes constraint diminishes strength. The more constraints one imposes, the more one frees oneself of the chains that shackle the spirit.106 : Igor Stravinsky, Poetics of Music 117 With the aid of electronic computers, the composer becomes a sort of pilot: pressing buttons, introducing coordinates, and supervising the controls of a cosmic vessel sailing in the space of sound, across sonic constellations and galaxies that could formerly be glimpsed only as a distant dream. Now they can be explored at ease, seated in an armchair.107 : Iannis Xenakis, Formalized Music We might dream of a direct connection to the digital realm—an immediate space of pure transmission where human desire is projected without the hindrance or distortion of material influence. But in this space of infinite potential, would there be no communication. Just as sound can not exist without disturbing a medium, expression requires something to play with. Often, the same structures that make transmission imprecise, also make it possible. So I must ask Xenakis, what is this armchair in which you sit? This liminal space of transformation, where the abstract melts into the concrete, is the digital’s fuzzy boundary, and whatever vitality, whatever creativity inheres in digital technologies, it will be found in this interstitial zone.108 : Aden Evens, Sound Ideas The specific character of the computer lies not in its ability to produce efficiently, but in its ability to process differently. By resisting the metaphorical, and programming rules or procedures that animate between—between forms, between the determinate and the infinite, the known and the unknown, the focused and the enactive—what is expressed is the interplay of a system. In this interstitial space of interfacing, the computer becomes an extension of life. The animating interface gives us “the ability to see the machine as more than when you were first led up to it, that you can make it more.”109 118 FIGURING OUT { Systems for Play } Simon says, pick up a handful of parts. Test them. Are they fair, intact, gleaming? This is a laboratory, so experiment. Simon says, revise this story, or make a better one. Title it “Simon Says.” 110 ~ Shelley Jackson, Simon Says Donna Haraway cuts straight to the heart of the matter, “Late twentieth century machines have made thoroughly ambiguous the difference between natural and artificial, mind and body, self- developing and externally designed, and many other distinctions that used to apply to organisms and machines. Our machines are disturbingly lively, and we ourselves frighteningly inert.”111 It was never the goal to explore the boundary between human and machine. Actually, on the surface, it was, but the idea was to focus on the surface. I set out to explore interfaces for playing music with computers, but unexpected figures threw the navigation off course. It turns out surfaces are not just surfaces—surfaces run deep. To capture in words a method designed to be played with hands, ears, and eyes is a struggle that underscores this entire project. Words reinforce certain dualisms that are inherent to the structure of language—subject/object, mind/body, animate/inanimate. We can play with words as a material themselves, twisting them into knots in an attempt to obscure these binaries, but they resist certain types of thinking. Words have been instrumental in shaping our worldview. 119 But there are other options. We can extend words through figuring. By making binaries visible we can consciously push against them. But figures have their own tendencies and biases. They do not escape linguistic structures, they only patch between them, folding the texture of language to hold multiple variables in a set of relations. Some writers are able to build these patch point into their words, but they do it despite words, not because of them. The computer allows for a different kind of engagement with information. Instead of describing the results of our figuring, we can program procedures that simulate it. We can build structures that are dynamic and models that allow for the manipulation of ideas in time. The computer provides the means to play with our worldviews. Yet, we still engage with computers primarily though symbolic structures: metaphorical interfaces that wrap the possibility space of the digital within icons and words that allow for an ease of use. In translating familiar communication paradigms to the digital realm, we reproduce a worldview that effectively hides the possibility of new perspectives that computers might allow. There is also a larger issue here about power and control. Those that control the current social, economic, political and cultural systems are invested in maintaining control, which requires a preservation of worldviews predicated on symbolic representation. Money is a symbol of wealth, but money is not wealth. But to realize that in a deep way might change how and where people spend money. Efficiency and productivity—faster, and with less effort—are what drive our new technologies, but why? Who actually wants simply faster and easier? I am arguing for slower and stranger. Less predictable, more ambiguous and mysterious and curious and obstinate and less efficient—perhaps even unproductive, at least from the perspective of those in charge. 120 The ndial speaks to this desire. It is not a new perspective in any grand sense, but it generates new perspectives in a small way. It presents a method of navigating our stockpiled music that produces new patterns within familiar materials, rather than pushing us to find familiar patterns in new materials. It resists metaphorical interfacing to allow meaningful dialogue between program and player through non-symbolic, enactive engagement with information. The ndial also forces one to confront their desire for mastery in order to unlock a space for experimentation and play. …we’re now living in a world with exponential and continuous change. We’re not just living through a transition; we have transitioned into always transitioning… That is to say, we have to be willing to constantly regrind our conceptual lenses. In order to continually make sense of this changing world, we need the freedom to explore, mess up, pull back, reflect, and try again.112 : John Seely Brown The protocol that Simon transmitted to millions of children in the 80s is best articulated by computer scientist Alan Perlis, “see the machine as more than when you were first led up to it, that you can make it more.” The computer is not merely a tool or a medium. It is both. It is also much more. It is a medium for making tools and a tool for creating mediums. It is a means of control and provides control over means. In connecting people, people become connected to it. Systems are seductive. They promise to do a hard job faster, better, and more easily than you could do it by yourself. But if you set up a system, you are likely to find your time and effort now being consumed in the care and feeding of the system itself. New problems are created by its very presence. Once set up, it won't go away, it grows and encroaches. It begins to do strange and wonderful things. Breaks down in ways you never thought possible. It kicks back, gets in the way, and opposes its own proper function. Your own perspective becomes distorted by being in the system. You become anxious and push on it to make it work. Eventually you come to believe that the misbegotten product it so grudgingly delivers is what you really wanted all the time. At that point encroachment has become complete... you have become absorbed... you are now a systems person! : John Gale, Systemantics113 121 The computer is not unique in its bi-directionality but it does not simply amplify pre-existing modes either. By animating interfaces I am not attempting to show that interfaces are alive, but that they are active and that they matter—to bring them to life. Our digital interfaces are already a big part of our lives but we engage with them largely as passive boundaries. But they can be so much more. Embedded in our interfaces are philosophies and perspectives, the structures that shape who we are. By animating interfaces I hope to build systems for active collaboration with the aspects of computing that characterize this new digital realm. To develop new modes of thinking that figure in rather than figure out. We return to Pauline Oliveros... It’s about the human/machine relationship or interface—the power of technology to expand the mind. You find solutions to creative problems and those solutions lead you into new territory where new solutions have to be found. 114 Shelley Jackson says “Simon says imitate a human”115 and so I’ll try, but I am not just a human. To “pick up a handful of parts” and “test them. Are they fair, intact, gleaming?” This is a laboratory, so experiment. Simon says, revise this story, or make a better one. Title it “Simon Says.” 116 122 APPENDIX I : BACKGROUND FIGURES Scott McCloud, Understanding Comics Buckminster Fuller, Synergetics A diagram by Stephan Willats Thomas Hirschhorn, Spectre of Evaluation Pauline Oliveros, Wind Horse 123 APPENDIX II : PROCESS MAPPING 124 APPENDIX III : FORMS (BY MICAELA MORRISSETTE) reconsid for eri RELEASE FORM I e system ng the patt Read and initial each disclaimer; completion required for entry. f th ern o in light ____________ I acknowledge and accept the costs of entering the system. ____________ I am strangely comforted, knowing there is no risk, as disaster is certain. Disaster is imminent, assuming that I continue to render my initials, over and over. ____________ I accept and understand that as I enter the system, the system will enter me. I am not initialing a metaphor. I acknowledge that systems reject the metaphorical, this one most of all. This system most of all. This metaphor most of all. I acknowledge my awareness that actual atoms from my body will drift away into the system, and that atoms from the system will infiltrate me. ____________ I understand and acknowledge that those atoms I cede to the system may spin off far, unthinkably far, into the coldest reaches of the cosmos. ____________ Other atoms, after their imprisonment and contamination within the system, may return to me, I understand. They may adhere to my fingertips as I push the door to exit, or cling to the hem of my clothing like orphans. ____________ I acknowledge that those of my atoms taken by the system and jettisoned within it, or flung far and frozen into the cosmos, will nevertheless never forget me. When the atoms of my skin crawl, something within the system will shiver and pucker and crawl. Likewise, when an atom within the system sparks and shrieks, I will spark and shriek. I understand that the system refuses absolutely the application of the term “sympathetic magic” to this phenomenon. ____________ Likewise, “love.” The term is “spooky action,” as per the system. I accept this. ____________ I acknowledge and understand that it is not just a matter of the system, at this point, although the system is the medium by means of which each disastrous and unsunderable atomic twinning shall be accomplished. I understand there are to be other users present in the system as well. ____________ I understand that my atoms will be subsumed into the bodies of other human entities, and I accept that nothing I can ever do or say will succeed in breaking the signal—invisible, inexplicable— between those atoms lost to me in other flesh and those I keep buried, like land mines, within me. ____________ And vice versa, I acknowledge. I will leave here tonight swollen with and coated by a hundred million kidnapped atoms, crying out for their lost twins, and buffeted, reeling with the volume and pitch of the answers they always—yes, always—receive from those twins, no matter how far flung, no matter how cold. 125 reconsid for eri RELEASE FORM II e system ng the patt f th Read and initial each disclaimer; completion required for entry. ern o in light ____________ I consent to being disparate. I am separated from each other system user by impossibly deep wells of darkness, of course. But moreover I consent to my own composition in the form of a field of atoms that circle each other constantly, wolves around fires, fire around wolves, without ever touching, such that I know myself to be not solid but gaseous: a ghost, an eidolon. ____________ I consent to the filming of this event. In particular, I consent to its holographic documentation, such that each pixel of digital image, in addition to composing a part of the image, will contain within itself the entirety of the code for the film as a whole. I consent to my multiplication and my miniaturization in this manner. ____________ I consent to be recorded from many different angles at once. I consent to my digital capture as a figure turned away, hunched and hidden, half out of frame; and as a simultaneous figure whose face will fill the screen like a vast spill of ink that blooms through paper, blotting out words. ____________ I consent to giving up my ghost. For if I am the hologram of the system, by which I consent to mean the universe, and if I encode and mirror it wholly, then I am not fragmentary, am not composed of atoms, ever circling, ever flaming (ever howling). ____________ No, instead, I consent to be space. I consent to be time. I consent to be signal. And I consent to be empty. I consent to simultaneity, ineradicability; I am already future. ____________ I consent to see and hear. I consent to refrain from watching or listening. I consent to cede control. I will be a stage for the system; I will be its stepping stone. With my consent, the system will select my channels of frequency. It will navigate me through infinite potentials of sound and light in spacetime. It will fold me, polyhedric, along its architecture, around its framework. ____________ I will be installed on its platform, and its protocols will run through me. ____________ As a hologram, I consent to repeating in endless seizures of memory everything I see and hear tonight. Indeed, I will be helpless in this matter. I consent to echo echo. I consent to eidetic playback. I consent to play. ____________ I consent to play. I consent to be played with. I consent to be the wind-up doll of the system. ____________ I consent to the performance of certain experiments upon my person. I shall endure certain instruments. I consent to process processes and to locate locations and moments of momentum. Quantum confusion on an n level shall erupt, whatever the danger to myself and others, as the result of my full consent. ____________ I consent to floating in space. I consent to tangled chains of light. I consent to the ecstasy of a million small deaths with each blink, between breaths. 126 reconsid for eri RELEASE FORM III e system ng the patt f th Read and initial each disclaimer; completion required for entry. ern o in light ____________ Even as I scratch my initials into this form, the letters are already erasing, their outlines degrading. Possibly I, too, have already begun to blur. Here at the welcome station, already, I consider my loss of resolution. ____________ I provisionally agree that my particles will stream away from me, magnetized by the electric soul of the system, burrowing into the flesh of other system users, spinning out into the logical fuzz of the cosmos. I provisionally agree to tremble. Provisionally to shiver. ____________ Still, even as I disappear, I will fill up, chokingly, with new atoms. I provisionally agree to these atoms of strangers, to the atoms of the system. Very soon, they will drive me out of myself. I’ll agree to be a mirror in the dark. ____________ It isn’t altogether agreeable here at the welcome station. This falter in my progress. This quantum stuttering. No, it isn’t quite safe, not even here, already. ____________ But perhaps I haven’t been safe for some time now? ____________ And I don’t believe I’ll ever be safe again, after tonight. ____________ I don’t not believe it, either. For if nothing is safe, then nothing is certain. ____________ I agree that if nothing is certain, here in the system, then nothing is true. Then, providentially, nothing is false. Nothing is proven. ____________ I could toy cruelly with 1, if the system would circle round 0. ____________ The system would be neither if I could be nor. ____________ Without the slightest provocation, the system might step on its own head. I would agree to that! Here at the welcome station, already, I’ve stepped into the outside of the inside-out system. ____________ The system reserves the right to seat me apart from my atoms. The system disclaims responsibility for any faces I see in the noise. ____________ I agree to dissolve. To be lost and dissolute. ____________ I agree to enter the system without my star charts and without provisions, for I agree never to return from it. ____________ Not with these molecules, not through this vertex of the multiverse. ____________ Agreed: The system ≈ nourishment if and not only if I am < > ravenous. ____________ Agreed: Within the system will not be light or dark but bleeding veins of spectral color. ____________ Agreed: The system will be sound and silence; also/rather/someday it will be the thrumming of wind forcing its way down my lungs. ____________ NOT light = (1 - truth(light)). ____________ Light AND sound = minimum(truth(light), truth(sound)). ____________ Light OR sound = maximum(truth(light), truth(sound)). ____________ LIGHT = (1 + truth(light)). SOUND = 42. ____________ I am pretty sure that although truth may agree to be measured in degrees, the fiction of the system is absolute. 127 reconsid for eri RELEASE FORM IV e system ng the patt f th Read and initial each disclaimer; completion required for entry. ern o in light ____________ I can almost remember initialing this release at the start (or the end) of every new circuit. ____________ Although I can never quite remember the system even once I am inside it, my initials provoke a queasy déjà-vu on each occasion. ____________ Perhaps because they are the only act of will I am permitted as I circle through system time. ____________ (Not so much a free action of will as the mimetic acting out of freedom, perhaps.) ____________ Perhaps also because of the relief of being able to sign the release, again and again. The relief of continuing to find myself here in the welcome station, intact enough to initial: ____________ I confirm that I have made another circuit through the system, unharmed. ____________ I confirm that I am still alive and, to the best of my knowledge, an individual. Even if I am only my memory of my existence, I confirm that to be the eidetic ghost of myself is good enough for me. ____________ I confirm my expectation that, once I have initialed the release in full, the system will take me again through its protocols, so that I can remember what happened to me a moment ago. ____________ Not that I will remember that moment ago, even once I am inside the system. But at least I can confirm, however briefly, the details of where I have just been and what I have been doing. Me or whoever that was. ____________ Not that undergoing for the nth time a thing perpetually forgotten is the same as recollecting it. ____________ And not that my next circuit will bear any real resemblance to the moment ago I’m now here in search of. If time travel did not destroy memory, I would have confirmed by now that no two circuits through the same point in time are ever identical. Not inside the system, anyway. Where time is unbearably delicate. Where time is crushed to powder under the weight of my gaze on it. Where amnesia is the cost of destroying time by observing it. Where nothing can ever be remembered because there is no such thing as repetition. ____________ As the system burns time to a crisp, everything grows very different from itself. ____________ So, are these my initials? ____________ Well, I confirm that they are. ____________ Or someone does. ____________ I confirm that I prefer a false memory to a real one. ____________ I may not remember the system, but at least I know there’s something I’m forgetting. ____________ I confirm that as long as I continue to initial this release, I can neither remember the system nor forget it. It is always not in my thoughts. I am in limbo. Perhaps I am inside the system already. ____________ The beautiful strings begin to extrude; they tangle painfully across the threshold of the welcome station. ____________ I am reeling backward in time from the moment of my death, and I am here to marvel at the contrails. Everything I fear has already ravaged me. I’m a slave to my freedom. ____________ The system will burn so bright tonight that all I was ever able to see there was light. Seeing everything is just like being blind. Seeing nothing, I begin to believe there are shapes in the dark. I can almost remember them. ____________ It’s time to return to some new place again, forever. 128 NOTES 1 The Institute For Figuring. Accessed March 30, 2014. http://theiff.org/. 2 Wark, McKenzie. A Hacker Manifesto. Cambridge, MA: Harvard University Press, 2004. 3 Watts, Alan. "The Veil of Thoughts." Recorded Lecture. Accessed: http://youtu.be/fE5OGBjtTVU 4 de gustibus non disputandum est. Accessed March 30, 2014. http://sampliciter.tumblr.com/. 5 Stein, Gertrude, and Judy Grahn. Really Reading Gertrude Stein: A Selected Anthology with Essays by Judy Grahn. Freedom, Calif: Crossing Press, 1989. 6 Attali, Jacques. Noise: the political economy of music. (Minneapolis: U. of Minnesota Press, 1985), 115. 7 Baby Names by ThinkBabyNames.com. "Simon - meaning of Simon name." Accessed March 31, 2014. http://www.thinkbabynames.com/meaning/1/Simon. 8 Brickman, Marshall. Simon. 1980. USA: Warner Bros., Film. 9 Berkeley, Edmund C. and Robert A. Jensen, “World's Smallest Electric Brain.” Radio-Electronics (1950): 1, 29 10 Berkeley, Edmund C., "Simple Simon." Scientific American (1950): 1. 11 Berkeley, Edmund C., Giant Brains; Or, Machines That Think. New York: Wiley, 1949. Print. 12 Berkeley, Edmund C., "Simple Simon." Scientific American (1950): 43. 13 Berkeley, Edmund C., “IBM Fact Sheet” Interview, Columbia University, May 18, 1950 14 “Simon Commercial from 1978,” YouTube video, :32, posted by "Gamez Gear," July 9, 2013, https://www.youtube.com/watch?v=yF0ZUXclW8Y. 15 McCloud, Scott. Understanding Comics: The Invisible Art. New York: HarperPerennial, 1994. 16 Sobchack, Vivian. "Child/Alien/Father: Patriarchal Crisis and Generic Exchange." Camera Obscura 15 (1986): 7-34. 17 Spielberg, Steven. Close Encounters of the Third Kind. 1977. California. : Columbia Pictures, Film. 18 Sobchack, Vivian Carol, and Vivian Carol Sobchack. Screening Space: The American Science Fiction Film. (New York: Ungar, 1987): 284. 19 McWhorter, Diane. "Electronic Shock in Toyland." (Boston, 1978): 104 20 Simon Instruction Manual, 1978. Milton Bradley. 21 McWhorter, Diane. "Electronic Shock in Toyland." (Boston, 1978): 164 22 Ibid., 104 23 Brautigan, Richard. All Watched Over by Machines of Loving Grace. San Francisco: Communication Company, 1967. 24 Abelson, Harold, Gerald Jay Sussman, and Julie Sussman. Structure and Interpretation of Computer Programs. Cambridge, Mass: MIT Press, 1996. 25 Found using a picture of Simon in a Google Image Search. (August 23, 2013) 26 Baer, Ralph H. Videogames: In the Beginning. (Springfield, NJ: Rolenta Press, 2005), 323. 27 Ibid,. 324. 28 Galloway, Alexander R. The Interface Effect. Cambridge, UK: (Polity, 2012): 10 29 Arcangel, Cory. “Pauline Oliveros.” BOMB 107. Spring 2009. Accessed 12 March 2014. http://bombmagazine.org/article/3268/pauline-oliveros 30 Urban Dictionary. "Urban Dictionary: onism." Accessed March 31, 2014. http://www.urbandictionary.com/define.php?term=onism. 31 The Dictionary of Obscure Sorrows. "The Dictionary of Obscure Sorrows — onism." Accessed March 31, 2014. http://www.dictionaryofobscuresorrows.com/post/73524850764/onism. 129 32 Sparkfun UBS32 featuring the PIC32MX795 IC based on the work of Brian Schmalz 33 Fuller, R. Buckminster, and E. J. Applewhite. Synergetics: Explorations in the Geometry of Thinking. (New York: Macmillan, 1975), 527.711-12. 34 Attali. Noise. (1985), 115. 35 Johnson, Steven. Interface Culture: How New Technology Transforms the Way We Create and Communicate. (San Francisco: HarperEdge, 1997). 36 Hayles, Katherine. How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics. (Chicago, Ill: University of Chicago Press, 1999), 290. 37 Warne, “Listening and Reflection in an Electroacoustic Compositional Practice.” (2013), 12. 38 Attali. Noise. (1985), 145. 39 Ibid., 134. 40 Ibid., 115. 41 Tanaka, Atau. “Interaction, Experience and the Future of Music." in Consuming Music Together: Social and Collaborative Aspects of Music ed. Kenton O'Hara et al. (Dordrecht: Springer, 2006), 284. 42 Huxley, Aldous. Ends and Means: An Enquiry into the Nature of Ideals and into the Methods Employed for Their Realization. (London: Chatto & Windus, 1937.) 43 Porcello, Thomas. “"Tails out": Social Phenomenology and the Ethnographic Representation of Technology in Music-Making” Ethnomusicology : Journal of the Society for Ethnomusicology. Vol. 42, No. 3 (Autumn, 1998), 501. 44 Oliveros, Pauline. Deep Listening: A Composer's Sound Practice. (New York: iUniverse, Inc, 2005.), xxiii. 45 Kittler, Friedrich A. Gramophone, Film, Typewriter. (Stanford, Calif: Stanford Univ. Press, 2006), 4. 46 Schaeffer, Pierre. "Little Bang Theory." Harpers, April, 2013. 47 Evens. Sound Ideas. (2005), 88. 48 Bangs, Lester. “Interview With Kraftwerk.” Creem Magazine (1975) 49 Holmes, Thom. Electronic and Experimental Music. New York: Scribner's, 1985. 50 "Kraftwerk - TV Documentary 2001 (Full Version)." YouTube. 1:45 – 2:15 https://www.youtube.com/watch?v=ssZ4hcalI6U. 51 Gibson, William. “Rocket Radio.” Rolling Stone (June 15, 1989) 52 Katz, Mark. Capturing Sound: How Technology Has Changed Music. (Berkeley: University of California Press, 2004), 139. 53 Ibid., 140-141 54 McLeod, Kembrew. “Owning Culture: Authorship, Ownership, and Intellectual Property Law.” Popular Culture and Everyday Life. Vol. 1. (New York: P. Lang, 2001), 39. 55 López, Francisco. “Profound Listening and Environment Sound Matter” in Audio Culture: Readings in Modern Music. Ed. Cox et al. (New York: Continuum, 2004), 85. 56 McCloud. Understanding Comics: The Invisible Art. (1994), 66. 57 Hamman. “From Symbol to Semiotic.” (1999), 102. 58 Ibid., 102. 59 Laske, Otto. "Composition theory: An enrichment of music theory." Journal of New Music Research (1989), 46. 60 Eno, Brian. “Generating and Organizing Variety in the Arts.” in Audio Culture: Readings in Modern Music. Ed. Cox et al. (New York: Continuum, 2004), 227. 61 Hamman. “From Symbol to Semiotic.” (1999), 102. 62 McLuhan, Marshall. Understanding Media: The Extensions of Man. (London: 1964), 7. 63 Abelson, Harold, Gerald Jay Sussman, and Julie Sussman. Structure and Interpretation of Computer Programs. Cambridge, Mass: (MIT Press, 1996), 1. 130 64 Hamman. From Symbol to Semiotic. (1999), 102. 65 Bartee, Jordan. “Ontological Toys.” (PhD Diss., Brown University, 2014), 1. 66 chris novello | networks, playthings, design. "About." Accessed March 31, 2014. http://www.paperkettle.com/home/about/. 67 Huizinga, Johan. Homo Ludens: A Study of the Play-Element in Culture. (Boston: Beacon, 1955), 10. 68 Imprecise quote from conversation with Brian House a few weeks after the performance. 69 Holmberg, Arthur. The Theatre Of Robert Wilson. (Cambridge: Cambridge, 1996), 121. 70 Burnham, Jack. ‘Systems Aesthetics,’ in Great Western Salt Works. Essays on the Meaning of Postformalist Art (New York, 1973), 16. 71 Burnham. ‘Systems Aesthetics.’ (1973), 18. 72 Giles, Steve. Theorizing Modernism: Essays in Critical Theory. (London: Routledge, 1993), 28. 73 McGonigal, Jane. “The Curious Interface: A Design Manifesto in Favor of Play.” Accessed March 12, 2014. http://www.avantgame.com/The%20Curious%20Interface.pdf 74 Schell, Jesse. The Art of Game Design: A Book of Lenses. (Amsterdam: Morgan Kaufmann, 2008): 222 75 Hamman, Michael. “The technical as aesthetic: Technology, composition, interpretation.” (2002): 3 76 Galloway, Alexander R. The Interface Effect. Cambridge, UK: (Polity, 2012): 99 77 Evens, Aden. Sound Ideas: Music, Machines, and Experience. Minneapolis: (University of Minnesota Press, 2005): 168 78 Rovan, Joseph. “Living on the Edge: Alternate Controllers and the Obstinate Interface” in Mapping Landscapes for Performance As Research: Scholarly Acts and Creative Cartographies. Ed. Riley et al. (Basingstoke: Palgrave Macmillan, 2009), 253. 79 McGonigal, Jane. “The Curious Interface: A Design Manifesto in Favor of Play.” Accessed March 12, 2014. http://www.avantgame.com/The%20Curious%20Interface.pdf 80 Evens, Aden. Sound Ideas: Music, Machines, and Experience. Minneapolis: (University of Minnesota Press, 2005): 162 81 Ibid,. 164 82 Lewis, George. "Interacting with latter-day musical automata." Contemporary Music Review (1999): 108 83 Hamman, Michael. “The technical as aesthetic: Technology, composition, interpretation.” (2002): 7 84 Sobchack, (Screening Space): 285 85 Pold, Soren. "Interface Realisms: The Interface as Aesthetic Form." Postmodern Culture 15.2 (2005), 1. 86 Katie Salen and Eric Zimmerman. Rules of Play: Game Design Fundamentals. Cambridge, Mass: (MIT Press, 2003): 87 Ibid,. 305 88 Flanagan, Mary. Critical Play Radical Game Design. Cambridge, (Mass: MIT Press, 2009), 1 89 Warne, Matthew. Listening and Reflection in an Electroacoustic Compositional Practice. (PhD diss., Brown University, 2013), 53. 90 Joel Ryan. “As if by Magic.” Website: http://jr.home.xs4all.nl/MusicInstDesign.htm 91 Cascone, Kim. "The Aesthetics of Failure: Post-Digital Tendencies in Contemporary Computer Music." Computer Music Journal (2000), 1 92 Ibid., 3 93 Evens. Sound Ideas. (2005), 123 94 Armstrong, Newton. “An Enactive Approach to Digital Musical Instrument Design.” (PhD diss., Princeton University, 2006), 4 95 Anecdote relayed by Stephan Moore in 2013, from conversations with Lou Harrison. Second anecdote by Todd Winkler in 2014, from conversations with Earl Brown. 96 Noë, Alva, and Tony Smith. "Experience and Experiment in Art." Journal of Consciousness Studies, 7, 131 No. 8–9, (2000), 123–35 97 The Notebooks of Leonardo Da Vinci. Volume No. 9, Section 508 98 Markle, Donald T. “The magic that binds us: Magical thinking and inclusive fitness.” Journal of Social, Evolutionary, and Cultural Psychology, Vol 4(1), (2010) 18-33. 99 Brecht, George. “Chance Imagery.” Originally published as a Great Bear Pamphlet by Something Else Press, (1966), 23. 100 Hamman, Michael. “From symbol to semiotic: Representation, signification, and the composition of music interaction.” Journal of New Music Research 28 (1999), 90-104. 101 Evens. Sound Ideas. (2005), 164. 102 Ibid., 7. 103 Ibid., 82. 104 Ibid., 130. 105 Ryan. “As if by Magic.” 11 106 Stravinsky, Igor. Poetics of music in the form of six lessons. Cambridge (Harvard U. Press, 1970), 53. 107 Xenakis Iannis. Formalized Music. Thought and Mathematics in Music. New York: Pendragon Press (Revised Edition), 1992. {updated to be gender neutral} 108 Evens. Sound Ideas. (2005), 9. 109 Abelson, Harold, Gerald Jay Sussman, and Julie Sussman. Structure and Interpretation of Computer Programs. Cambridge, Mass: (MIT Press, 1996), 1. 110 Jackson, Shelley. "Simon Says." Fence, (2013), 22. 111 Haraway, Donna Jeanne. The Haraway Reader. (New York: Routledge, 2003), 12. 112 Seely-Brown, John. Web Interview. Accessed 12 March 2013. http://spotlight.macfound.org/featured- stories/entry/john-seely-brown-on-interest-driven-learning-mentors-and-play/ 113 Gall, John. Systemantics: How Systems Work and Especially How They Fail. New York: Quadrangle/New York Times Book Co, 1977. 114 Arcangel, Cory. “Pauline Oliveros.” BOMB 107. Spring 2009. Accessed 12 March 2014. http://bombmagazine.org/article/3268/pauline-oliveros 115 Ibid., 12 116 Jackson, Shelley. "Simon Says." Fence, (2013), 22. 132 BIBLIOGRAPHY Abelson, Harold, Gerald Jay Sussman, and Julie Sussman. Structure and interpretation of computer programs. 2nd ed. Cambridge, Mass.: MIT Press ;, 1996. Arcangel, Cory. “Pauline Oliveros.” BOMB 107, Spring 2009. Armstrong, Newton. “An Enactive Approach to Digital Musical Instrument Design.” PhD diss., Princeton University, 2006. Attali, Jacques. Noise: the political economy of music. Minneapolis: University of Minnesota Press, 1985. Baer, Ralph H.. Videogames: in the beginning. Springfield, NJ: Rolenta Press, 2005. Bangs, Lester. “Interview With Kraftwerk.” Creem Magazine, 1975. Bartee, Jordan. “Ontological Toys.” PhD Diss., Brown University, 2014. Berkeley, Edmund. Giant brains; or, Machines that think. New York: Wiley, 1949. Berkeley, Edmund C. and Robert A. Jensen, “World's Smallest Electric Brain.” Radio-Electronics, 1950. Berkeley, Edmund C., "Simple Simon." Scientific American, 1950. Berkeley, Edmund C., “IBM Fact Sheet” Interview, Columbia University, May 18, 1950. Bishop, Claire. Artificial hells: participatory art and the politics of spectatorship. London: Verso Books, 2012. Brautigan, Richard. All Watched Over by Machines of Loving Grace. San Francisco: Communication Company, 1967. Bull, Michael, and Les Back. The auditory culture reader. Oxford, UK: Berg, 2003. Burnham, Jack. ‘Systems Aesthetics,’ in Great Western Salt Works. Essays on the Meaning of Postformalist Art. New York, 1973. Brecht, George. “Chance Imagery.” Originally published as a Great Bear Pamphlet by Something Else Press, 1966. Brickman, Marshall. Simon. 1980. USA: Warner Bros., Film. Cascone, Kim. "The Aesthetics of Failure: Post-Digital Tendencies in Contemporary Computer Music." Computer Music Journal, 2000. Chadabe, Joel. Electric sound: the past and promise of electronic music. Upper Saddle River, N.J.: Prentice Hall, 1997. Cox, Christoph, and Daniel Warner. Audio culture: readings in modern music. New York: Continuum, 2004. 133 Cycling ’74. Max. San Francisco, CA: Cycling ’74, 1997. Deleuze, Gilles and Félix Guattari. A thousand plateaus: capitalism and schizophrenia. Minneapolis: University of Minnesota Press, 1987. Deleuze, Gilles. The fold: Leibniz and the Baroque. Minneapolis: University of Minnesota Press, 1993. Demers, Joanna Teresa. Listening through the noise: the aesthetics of experimental electronic music. Oxford; New York: Oxford University Press, 2010. Evens, Aden. Sound ideas: music, machines, and experience. Minneapolis: University of Minnesota Press, 2005. Flanagan, Mary. Critical play radical game design. Cambridge, Mass.: MIT Press, 2009. Foucault, Michel. The order of things: an archaeology of the human sciences. New York: Pantheon Books, 1971. Fuller, R. Buckminster, and E. J. Applewhite. Synergetics: explorations in the geometry of thinking. New York: Macmillan, 1979. Gall, John. Systemantics: how systems work and especially how they fail. New York: Quadrangle/New York Times Book Co., 19771975. Galloway, Alexander R.. The interface effect. Cambridge, UK: Polity, 2012. Gibson, William. “Rocket Radio.” Rolling Stone, June 15, 1989 Giles, Steve. Theorizing modernism: essays in critical theory. London: Routledge, 1993. Hamman, Michael. "On Technology and Art: Xenakis at Work." Journal of New Music Research 33, no., 2004. Hamman, Michael. “The technical as aesthetic: Technology, composition, interpretation.”, 2002. Hamman, Michael. “From symbol to semiotic: Representation, signification, and the composition of music interaction.” Journal of New Music Research 28, 1999. Haraway, Donna Jeanne. The Haraway reader. New York: Routledge, 2004. Hayles, N. Katherine. How we became posthuman: virtual bodies in cybernetics, literature, and informatics. Chicago, Ill.: University of Chicago Press, 1999. Heidegger, Martin. Being and time. New York: Harper, 1962. Holmberg, Arthur. The theatre of Robert Wilson. Cambridge: Cambridge University Press, 1996. Holmes, Thom. Electronic and experimental music: technology, music, and culture. 3rd ed. New York: Routledge, 2008. Huizinga, Johan. Homo ludens: a study of the play-element in culture.. Boston: Beacon Press, 1955. Jackson, Shelley. "Simon Says." Fence, 2013. 134 Johnson, Steven. Interface culture: how new technology transforms the way we create and communicate. San Francisco: Harper Edge, 1997. Katz, Mark. Capturing sound: how technology has changed music. Berkeley: University of California Press, 2004. Kittler, Friedrich A.. Gramophone, film, typewriter. Stanford, Calif.: Stanford University Press, 1999. Kraushaar, Otto F., and Aldous Huxley. "Ends and Means.." American Sociological Review 3, no. 2 (1938): 259. Laske, Otto. "Composition theory: An enrichment of music theory." Interface 18, no. 1-2, 1989. Lewis, George. "Interacting with latter-day musical automata." Contemporary Music Review 18, no. 3, 1999. Lewis, George. “Mobilitas Animi: Improvising Technologies, Intending Chance.” Parallax 13.4, 2007. Lysloff, René T. A., and Leslie C. Gay. Music and Technoculture. Middletown, Conn.: Wesleyan University Press, 2003. Markle, D. Thomas. "The magic that binds us: Magical thinking and inclusive fitness.." Journal of Social, Evolutionary, and Cultural Psychology 4, no. 1, 2010. McCloud, Scott. Understanding comics: the invisible art. New York: Harper Perennial, 1994. McGonigal, Jane. “The Curious Interface: A Design Manifesto in Favor of Play.” Accessed March 12, 2014. http://www.avantgame.com/The%20Curious%20Interface.pdf McLeod, Kembrew. “Owning Culture: Authorship, Ownership, and Intellectual Property Law.” Popular Culture and Everyday Life. Vol. 1., New York, 2001. McLuhan, Marshall. Understanding Media: The Extensions of Man. London, 1964. McWhorter, Diane. "Electronic Shock in Toyland." Boston, 1978. Noë, Alva, and Tony Smith. "Experience and Experiment in Art." Journal of Consciousness Studies, 7, No. 8–9, 2000. Novello, Chris. networks, playthings, design. "About." Accessed March 31, 2014. http://www.paperkettle.com/home/about/. Oliveros, Pauline. Deep listening: a composer's sound practice. New York: iUniverse, Inc., 2005. Penley, Constance. Close encounters: film, feminism, and science fiction. Minneapolis: University of Minnesota Press, 1991. Pold, Soren. "Interface Realisms: The Interface as Aesthetic Form." Postmodern Culture 15, no. 2 (2005) Ryan, Joel. “As if by Magic.” Website: http://jr.home.xs4all.nl/MusicInstDesign.htm Porcello, Thomas. ""Tails out": Social Phenomenology and the Ethnographic Representation of Technology in Music-Making." Ethnomusicology 42, no. 3, 1998. 135 Rovan, Joseph. “Living on the Edge: Alternate Controllers and the Obstinate Interface” in Mapping Landscapes for Performance As Research: Scholarly Acts and Creative Cartographies. Ed. Riley et al. Basingstoke: Palgrave Macmillan, 2009. Salen, Katie, and Eric Zimmerman. Rules of play: game design fundamentals. Cambridge, Mass.: MIT Press, 2003. Salter, Chris. Entangled: technology and the transformation of performance. Cambridge, Mass.: MIT Press, 2010. Schaeffer, Pierre. "Little Bang Theory." Harpers, April, 2013. Schell, Jesse. The art of game design: a book of lenses. Amsterdam: Elsevier/Morgan Kaufmann, 2008. Seely-Brown, John. Web Interview. Accessed 12 March 2013. http://spotlight.macfound.org/featured- stories/entry/john-seely-brown-on-interest-driven-learning-mentors-and-play/ “Simon Commercial from 1978,” YouTube video, :32, posted by "Gamez Gear," July 9, 2013, https://www.youtube.com/watch?v=yF0ZUXclW8Y. Spielberg, Steven. Close Encounters of the Third Kind. 1977. California. : Columbia Pictures, Film. Sobchack, Vivian. Screening space: the American science fiction film. 2nd, enl. ed. New York: Ungar, 1987. Sobchack, Vivian. "Child/Alien/Father: Patriarchal Crisis and Generic Exchange." Camera Obscura 15, 1986. Stein, Gertrude, and Judy Grahn. Really reading Gertrude Stein: a selected anthology with essays by Judy Grahn.. Freedom, Calif.: Crossing Press, 1989. Stravinsky, Igor. Poetics of music in the form of six lessons. Bilingual ed. Cambridge, Mass.: Harvard University Press, 1970. Tanaka, Atau. “Interaction, Experience and the Future of Music." in Consuming Music Together: Social and Collaborative Aspects of Music ed. Kenton O'Hara et al. Dordrecht: Springer, 2006. The Institute For Figuring. Accessed March 30, 2014. http://theiff.org/. Wark, McKenzie. A hacker manifesto. Cambridge, MA: Harvard University Press, 2004. Warne, Matthew. Listening and Reflection in an Electroacoustic Compositional Practice. PhD diss., Brown University, 2013. Watts, Alan. "The Veil of Thoughts." Recorded Lecture. Accessed: http://youtu.be/fE5OGBjtTVU Xenakis, Iannis. Formalized music; thought and mathematics in composition. Bloomington: Indiana University Press, 1971. 136