Digital Musical Instruments as Probes: How
computation changes the mode-of-being
of musical instruments
KORAY TAHİ ROĞ LU
1
, THOR MAGNUSSON
2
, ADAM PARKINSON
3
,
IRIS GARRELFS
4
and ATAU TANAKA
4
1
2
3
4
This article explores how computation opens up possibilities for
new musical practices to emerge through technology design.
Using the notion of the cultural probe as a lens, we consider the
digital musical instrument as an experimental device that yields
findings across the fields of music, sociology and acoustics.
As part of an artistic-research methodology, the instrumental
object as a probe is offered as a means for artists to answer
questions that are often formulated outside semantic language.
This article considers how computation plays an important role
in the authors’ personal performance practices in different
ways, which reflect the changed mode-of-being of new musical
instruments and our individual and collective relations
with them.
1. INTRODUCTION
Computation has been central to the development
of new interfaces for musical expression, and this
has been followed with new musical practices.
During the last half-century, we have witnessed devel-
opments in the field of digital musical instrument
(DMI) design and practice. However, much of the
knowledge and many of the methods that have
emerged in experimental music practices have been
fragmentary, often responding to individual and spe-
cific artistic and musical problems. These fragments
partially revolve around academic communities such
as those assembled for NIME conferences since
2002, and institutions such as STEIM, established
in the Netherlands in 1959. Whilst this largely hetero-
geneous assemblage of practices and ideas among
connected communities has addressed different musical,
social and technological elements of DMI culture, how
computation has changed the mode-of-being of such
musical instruments more generally has been less widely
realised or discussed.
A DMI has been defined as a musical instrument
where the digital sound generator is separate from
the control interface, the two relatable via mappings
(Malloch, Birnbaum, Sinyor and Wanderley 2006: 49).
Whist commercial digital keyboards can fit this
definition, we focus on those DMIs which have novel
interfaces and mapping strategies. These DMIs
are increasingly decoupled from the established
relationships we have with more traditional musical
instruments. Such DMIs are no longer like traditional
musical instruments, such as the saxophone, sitar or
piano, for which many different compositions have
been written and which have established themselves
as cultural icons, references and constants. Rather
than being a generic type, these DMI move towards
becoming a different category of musical instrument;
often embodying certain theoretical approaches and
affording specific practices. Today’s technological
context makes it easier for composers and performers
alike to develop their own new instruments and
systems, tailored to each new musical context, be
that a single composition, a band or an ensemble, or
an interactive installation or software-based work.
In this, computation is an inherent part of the DMI,
much as electricity is part of rock instrumentation,
and mechanical buttons and reeds are fundamental
to woodwind instruments. Computation therefore
shapes our relationship with DMIs and also transforms
our musical norms, habits, language and intentions; it is
the DMI’s unique mode-of-being in a new performance
practice. Furthermore, computation impacts widely
upon the nature of musical activities; how music is
performed, experienced, shared and distributed.
In this article we seek to provide a new examination
of our relationships with such new musical instruments.
Our primary questions are: How has computation
changed the mode-of-being of musical instruments?
Can we think of DMIs as having an embodied quality
given their digital materiality? What do today’s
instruments inspire us to do differently? Can we apply
the idea of a cultural probe (Gaver, Dunne and Pacenti
1999) to enquire further into their agency as far as
musical composition and performance are concerned?
Through considering the computational nature of new
musical instruments, we question what has changed in
Organised Sound 25(1): 64–74 © The Author(s) 2020. This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence
(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is
properly cited. doi:10.1017/S1355771819000475
https://doi.org/10.1017/S1355771819000475 Published online by Cambridge University Press
our relationships with musical instruments, composi-
tions, performances and musical experiences. This
perspective gives us, the current authors as practising
sonic artists, a lens through which to explain how
computation influences the methods and processes
of our musical work(s). In the following sections, we
explore these questions through a phenomenology
of embodied musical performance with the conceptual
tool of ‘cultural probes’. This article is based on ref-
lections around our creative processes when working
with computational devices, and are the outcome of a
workshop held at Goldsmiths, University of London,
in early 2019.
2. PROBING: AN INTERCONNECTED
ECOSYSTEM
Bill Gaver, Tony Dunne and Elena Pacenti (Gaver
et al. 1999) introduced cultural probes as a tool for
Human Computer Interaction (HCI) research and
development in the late 1990s. Gaver and his team
had been tasked to design projects for various commu-
nities, and order to initiate a dialogue with the users,
the team created toolkits that enabled people to
document their lives in a creative and lively fashion.
As Gaver (1999: 22) describes: ‘The cultural probes—
these packages of maps, postcards, and other
materials—were designed to provoke inspirational
responses from elderly people in diverse communities.’
The ‘probe returns’ of postcards, maps and other arte-
facts were used to start a dialogue so that the designers
felt they were almost ‘designing for friends’ (Gaver ,
Boucher, Pennington and Walker 2004: 54). The idea
of a cultural probe began to resonate throughout
the HCI community, and over the next few years this
idea was adopted by various designers and researchers,
becoming a valuable tool in co-design and related
design practices.
We propose that cultural probes can also be used
as a theoretical toolkit to explore how we relate with
DMIs. This is not to say that DMIs are necessarily
cultural probes per se, but rather that thinking through
DMIs and the greater context in which DMIs are used
in terms of cultural probes can be fruitful in discussing
the changed mode-of-being of these musical instru-
ments. It should be noted that cultural probes, as
used by Gaver et al., refer to specific packages of mate-
rials. For our purposes here, we would like to imagine
a probe and its returns to hav e broader media and
materiality. Furthermore, the idea of cultural probes
has been an interesting ‘probe’ into the HCI commu-
nity itself, stimulating debates around methodology,
uncertainty and the role of ‘the scientific process’
in design, something addressed by Gaver himself
(Gaver et al. 2004; Hutchinson et al. 2003). We believe
that it is in the spirit of cultural probes to adapt the
concept and apply it to DMIs. Gaver et al (2004:53)
note that the probes demonstrated that ‘research
questions could be packaged as multiple, rich, and
engaging tasks that people could engage with by
choice and over time’. What research questions, then,
can be asked through DMIs and the contexts in which
they are used?
When performing with a DMI, we likely depart to
some degree from traditional musical performance
practice. This can begin a dialogue about what consti-
tutes music and performance, and open up discussions
about what ideologies we want to embed in the instru-
ment and inscribe in its computation. This might range
from the extremes of being told (hopefully without too
much cruelty or aggression) that what we are doing
‘isn’t music’ or that our chosen tool ‘isn’t a musical
instrument’, throug h to more positive responses,
for example that a performance or instrument is
‘expressive’. Whereas Gaver et al.’s ‘probe returns’
constituted feedback in terms of photographs, anno-
tated maps and postcards, the returns we receive
from our probes come in the form of bewildered faces,
dancing bodies, applause, heckles, camera phone
footage and perhaps the occasional piece of rotten
fruit. For the most part, we informally digest and
process this feedback, and respond to it in (future)
performances.
In this way we can consider DMIs as cultural probes
insofar as they are a means by which we, as musicians
and researchers, seek to provoke and gather responses
from audiences and listeners in a dialogue about what
constitutes music and musical performance (Figure 1).
To design a DMI is to probe musical history and to
ask what musical ideas and ideologies we subscribe to.
In many instances, the end result of the design
process surrounding a DMI is best understood as
the design of a musical experience. In this way, the
musical concert itself can be conceived of as a cultural
probe. It refers not just to a single performance with
a DMI, but also to the larger context within which per-
formers, promoters and audiences are conceptualising
and experiencing musica l performances. This includes
everything from the stage, lighting, the type of venue
or soundsystem, to whether people are seated or
dancing. This can be conceived of as an ecosystem
within which musical experiences involving the DMI
are collectively imagined and instantiated. Concerts
are the means by which performers, curators and audi-
ences interrogate a new instrument and the performance
practice around it. Therefore, the way in which we
choose to frame the presentation and reception of the
instrument can do as much ‘probing’ as the DMI itself
and can be a way of packaging a research question.
When instruments are put in the hands of per-
formers, a further probing occurs. Musici ans often
use musical instruments in ways that the original
Digital Musical Instruments as Probes: How computation changes the mode-of-being of musical instruments 65
https://doi.org/10.1017/S1355771819000475 Published online by Cambridge University Press
designers never intended, probing for hidden affordan-
ces. An oft-cited example is the Roland TB 303, a
bassline synthesiser that was designed to provide back-
ing tracks for bass players. It turned out to be awkward
to programme and essentially ‘failed’ at what the design-
ers wanted it to do. However, a few years after its
release, DJ Pierre was experimenting with one of these
devices and accidentally stumbled upon the ‘squelchy’
sound that came to characterise, and effectively
invented, the Acid House genre which spread across
North America and the UK (Reynolds 2013). Taking
another example, saxophonists such as Evan Parker
and John Butcher reveal ‘hidden affordances’ (Gaver
1991) that were never deliberately designed into the
instrument, using extended techniques to produce multi-
phonics and ‘unintended’ sounds from the instruments
that have come to define the style of these musicians.
There are many similar cases, all of which undermine,
or at least problematise, the notion that even if an instru-
ment can be designed according to a specified set of
criteria, it can never be fully limited to those definitions.
DMIs as cultural probes have specific functions in
this context; by giving an undefined musical technology,
albeit from a specific cultural context, to an instrumen-
talist the designer is able to observe the musician’s
response and how ideas emerge through the use of
the instrument. This has been explored by instrument
makers, such as Ulfarsson (2019), whose halldorophone
instrument has been adopted by various performers
around the world (Hildur Guðnad´ottir plays the halldor-
ophone on the Sunn O))) album Life Metal). The results
have been interesting: the instrument shapes the musi-
cian as much as the musician shapes the music, but
this is not a simple picture and innumerable parameters
come into play. Through the instrument as a probe
given to the world, to other users, the designers of the
DMI are able to explore the character of the instrument
in important aspects that are not available when using
the instrument themselves, as they are too familiar with
its functions. It is only when the instrument is released,
or set free, that we begin to see its alternative potentials.
One criticism of DMIs is that, in many instances,
they are always being updated and evolving, prevent-
ing the performer from acquiring the sort of virtuosity
we associ ate with instruments of the past. However,
if we see DMIs as probes, it makes sense that they
are always in development, and that they are part of
an ongoing design process, forming one element of
a multi-sided conversation. The relationship between
audience feedback and instrument design is not such
that the DMI performer necessarily responds directly
to every criticism uttered, much in the way that Gaver
et al. (1999: 29) note ‘Although the probes were central
to our understanding of the sites, they didn’t directly
lead to our designs. They were invaluable in making
us aware of the detailed texture of the sites, allowing
us to shape proposals to fit them.’ Likewise, this probe
becomes part of the DMI’s performative ecosystem to
which we respond.
The design of DMIs is often rooted in rich, multimodal
and conversational design processes that resonate with
the cultural probe’s methodology, as opposed to a more
‘scientific’ process. Attempts to assess the ‘succe ss’
or ‘failure’
of a new DMI can prove to be difficult.
Knowing that a DMI enables a performer to reach a dis-
tant octave in a short time period may not be as valuable
as knowing that a DMI made people dance or enjoy a
concert, but the latter is likely to be harder to measure
Figure 1. In 4 Hands Iphone performance series, Atau Tanaka and Adam Parkinson re-appropriate the iPhone and its
advanced technical capabilities to transform the consumer and iconic object into an expressive DMI.
66 Koray Tahiroğlu, Thor Magnusson, Adam Parkinson, Iris Garrelfs and Atau Tanaka
https://doi.org/10.1017/S1355771819000475 Published online by Cambridge University Press
and quantify. Having considered the DMI as a probe in a
larger ecosystem, next we consider how phenomenology
has looked at the ways we probe our world through
technology.
3. PHENOMENOLOGY IN DMI
PERFORMANCE
Phenomenology is a wide-ranging philosophical
movement that studies human experience. It looks
at how we make sense of the world and takes an
approach that rejects Cartesian dualism. At the begin-
ning of phenomenological thought, Husserl proposes
ascientific,‘systematic enquiry into our conscious men-
tal processes without regard to their non-mental causes
and consequences’ (Inwood 1999: 2). Heidegger breaks
with Husserl’s neo-Kantian view and considers the
philosophical question of being, and extends the word
Dasein to think about ways of ‘being-in-the-world’.
He coins the terms vorhanden (present-at-hand) and
zuhanden (ready-to-hand) to distinguish whether an
object is the focus of attention or is transparent to the
action that is carried out through it (Inwood 1999).
These concepts have been taken up by HCI researchers,
including Dourish (2004).
Different aspects of phenomenology have been
applied to studies of sound and music, and of musical
instruments. Cox (2001) takes an embodied cognition
approach to studying musical meaning and proposes a
mimetic hypothesis where a listener naturally internally
imitates a performance they experience. De Souza
(2017), in his Music at Hand combines music theory
and phenomenology to study ways in which traditional
musical instruments become creative prostheses, technol-
ogies that condition and involve the body in otherwise
seemingly cerebral tasks such as notated composition.
Instrumentalists are familiar with the experience of sens-
ing the unique character of an instrument, as if each one
is a probe into unknown musical possibilities.
Ihde (2007) carries out a phenomenological study
of sound and uses the philosophical frameworks put
forth by Husserl, Heidegger and Merleau-Ponty
to unwrap the act of listening. By emphasising phe-
nomenology not as philosophy, but as an activity,
we might liken Idhe’s gesture to Christopher Small’s
transformation of music into the act of musicking
(Small 1998). If doing phenomenology becomes a
way to study our experiences of the world, we can
deploy DMIs as scientific instruments or probes in
which we further seek to understand our human
condition through active musical perception (Noë
2004). Phenomenology serves here as the theoretical
foundation of the instrumental probes we have in our
methodological toolbox.
Instruments can function as phenomenological
probes, as the example of David Sudnow’s Ways of
the Hands (1993) demonstrates. For Sudnow, learning
the piano late in his life became an investigation into
a phenomenological cond ition between human and
instrument, and writing the book became a reporting
mechanism, akin to the postcards and artefacts with
Gaver’s subjects with which the journey and its
insights are captured. By bringing together insights
from De Souza (2017) on the embodied cognition
view of musical instrument practice with Dourish’s
(2004) phenomenological understanding of human –
computer interaction, we can ask how does the
computational nature of DMIs reframe the phenome-
nological probe? What does it mean to ‘think’ with a
probe, to utilise an instrument that might change
or evolve over time? Are there modes of feedback –
visual, haptic or sonorous – that can strengthen the
performer’s focus such that new insights emerge?
(Figure 2).
However, a phenomenology of digital musical
instrument performance cannot reside solely in the
performer relationship with the instrument. If the per-
formance takes place in a public setting, the experience
is not complete until it is seen and heard by an audi-
ence. In the The Meaning of the Body, Mark Johnson
describes forms of intersubjectivity that are useful for
us to consider how a performer’s experience is commu-
nicated (Johnson 2008). Using phenomenology to
probe a DMI performance, we can study the experi-
ences of performers and audiences. They can then
be combined to form a holistic understanding of per-
formance and reception of a concert taking place
on digital musical instrument technology. The non-
dualist orientation of post-Husserlian phenomenology
helps us to consider visceral and mental musical
experience together as an intertwined whole. Studying
performer and audience together may allow us to
understand the experiential space that is afforded by
interactive instrument design. Beyond the instrument,
it may help us to understand the ways in which the
performance experience is transfused through different
bodies – the performer’s body, the audience’sbodies
and the social body through which a moment is shared.
4. DIFFERENT BODIES
In thinking about phenomenological probes and
bodies, how do we as active participants in the ecology
of sonic-making relate to this, especially in a perfor-
mance setting? How do we feature as bodies in this
becoming-practice? What does this ecosystem of body
and computation in digital instruments have to say
about the music that is being produced?
From a phenomenological perspective, Ihde speaks
about the significance of ‘listening to the voiced
character of the sounds of the World’ (Ihde 2007:
147), equally inner imagined sounds and auditory
Digital Musical Instruments as Probes: How computation changes the mode-of-being of musical instruments 67
https://doi.org/10.1017/S1355771819000475 Published online by Cambridge University Press
phenomena experienced in hearing. In performance
this relates to the sounds we create, and how we imag-
ine our audience to experience them. This probing-as-
listening connects the performer’s body with the
instrumental body, the body of the audience, the archi-
tectural body and, crucially, the sonic body that
emerges from their interplay. The instrumental body
in this configuration is an interesting one with far-
reaching computational implications on the sound
that is being created, and how performers and audien-
ces might relate to them by active listening. In the
twentieth century, the encounter between new technol-
ogies and new philosophies gave us new musics and
new listening perspectives, such as musique concrète
and reduced listening. Ihde surmised that ‘perceptions
are concretely situated within an emerging metaphor,
a newly oriented system’ (Ihde 2007: 233). If we hear
differently it is because we exist in the world differ-
ently. At the same time, within both the creation of
and the listening to the sonic body, past memories
emerge and are encoded continually, both culturally
(Castells 1996) and personally speaking (Demers
2010); the same applies to the design of DMIs.
Regardless of how the past influences auditory percep-
tion, it always takes place in the present. Even if
technology shapes our listening and making of music,
these are clearly deeply embodied and intertwined pro-
cesses. For DMIs to work fluidly there needs to be an
immediacy of interaction between body and interface.
A short response time between imagining sound, execut-
ing sound and listening to the sound made, enables the
performer ’s body to feel intimately connected with the
sonic output of the instrument played.
In Western art music, composition and performance
have become seen as separate activities: according to
this model a composer encodes her ideas in musical
notation, and the performer then executes this score
in sound. Later, some of these separate activities began
to overlap as sounds could be ‘sculpted’ according to
the composers wishes (Ihde 2007: 262); for example,
via editing and manipulating tapes (and later desktop
computers), guided by the probing ears. Further down
the digital timeline, and in regard to the augmented
voice pioneer Pamela Z, the ‘body becomes redefined
as both a source of data and as a bidirectional, perme-
able membrane of transference’ (Lewis 2007: 77).
In short, any envisaged ‘instrumentation’ becomes
part of the composition process, imagination activated
by the body and vice versa. This ‘instrumentation’ also
activates us and contributes to the music that unfolds.
And so, while we make music with/through an instru-
ment, each instrument we engage with also leads us to
create certain sounds, or at least favour some (Figure 3).
This applies to both haptic properties and computa-
tional parameters, and their joint ‘workflow’ (Brown,
Eldridge and McCormack 2009).Atthecoreofthe
composition process is a system which ties together
the performer, instrument, audience, performance space
and sound. However, this system is never completely
fixed, and to boot many of these demarcation lines have
become blurred. It requires listening as a probe to retain
balance in flux.
Thinking back to Gaver’s notion of probes, we
can extend this to the probing of spaces. Considered
in conjunction with Castells’s observation that spaces
express society (Castells 1996
), perhaps we also need
Figure 2. Atau Tanaka performing the piece Myogram that explores the digital music
performance with sensors, networks, mobility and performer body.
68 Koray Tahiroğlu, Thor Magnusson, Adam Parkinson, Iris Garrelfs and Atau Tanaka
https://doi.org/10.1017/S1355771819000475 Published online by Cambridge University Press
to re-think the spaces that we want to inhabit as musi-
cians and sounding artists, or how we inhabit them.
For instance, by creating mobile performances or
using wearable speaker systems that allow sounds to
emanate from the body that produces it, by which
sound is re-combined with body, re-focusing schizo-
phonic experience. The complexity of contemporary
music tech nology, and how it influences composition,
also includes aspects of co-creation where generative
processes contribute to the music and offer the performer
an opportunity for response. Probing-as-listening then
also connects the performer’s body with this computa-
tional body, accessed through sound.
5. MULTIDIMENSIONAL RELATIONS
WITH DMIS
The above contexts represent the diverse interrelations
of bodies and the materiality of sound, offering us
compelling possibilities for rediscovering our relation-
ships with DMIs. These relations are mediated
by digital technologies and are centred around the
different dimensions in which the changed nature of
embodiment may become manifest (Ihde 2010: 41).
The word ‘dimensions’ is used here to mean the range
of musical activities we undertake with surrounding
technologies and our interactions with them. They
are the ways in which our embodied relationships
with DMIs may vary and take on diversity. The
features of DMIs have traditionally been developed
by novel technological configurations, at times embrac-
ing design constraints. Here, we explore how different
aspects of human–technology interaction alter the
mode-of-being of musical instruments and change our
relationships with them. In order to do this, we will look
at human–technology relations through Don Ihde’s four
phenomenological modes of technological mediation:
embodiment relations, hermeneutic relations, alterity
relations and background relations (Ihde 1990).
In the first mode, embodiment relations, technology
remains perceptually transparent representing a kind
of isomorphism between our bodily actions and
perceptions (Ihde 1990: 82). We can think of this as
being consistent with Heidegger’s zuhanden. In this
perspective the user of the technology is fully engaged
with interactions with technological artefacts where
the artefact itself slips into the background, disappear-
ing from conscious thought. In this mode, technology
becomes part of life, not distinct from it. It becomes an
extension of our body and we use it focusing on the
content and not the technology. When music becomes
mediated through an instrument in an embodiment
relation, we do not focus on the instrument as a
distinct object but rather on the music itself.
Another aspect of embodiment relations is that tech-
nology brings into perception that which was not
previously apparent. Examples given by Ihde are the
magnifying glass or the telescope, because these scien-
tific instruments mediate perception, illuminating that
which was invisible to the human eye in earlier times
(Ihde 1990: 49). By mediating perception, technology
shapes and transforms our relationship with the world
around us. Ihde points out that once the sight of the
mountains of the moon became visible, visual technol-
ogy transformed the moon’s earlier existing spiritual
context in human history into a visual perception that
Figure 3. An example for improvised augmented voice performance by Iris Garrelfs. Performance took place at the
Barbican during the 2013 HackTheBarbican event, curated by Music Hackspace.
Digital Musical Instruments as Probes: How computation changes the mode-of-being of musical instruments 69
https://doi.org/10.1017/S1355771819000475 Published online by Cambridge University Press
has changed our relationship with it. The moon is not
part of the heavens any more, but it is an astronomical
object, a body that orbits the earth.
Digital technologies result in deep transformations
in embodied musical practices, enhancing the differ-
ences in comparison to acoustic musica l instruments.
Computation brings new methods of generating music
with our bodies and new methods of producing sound.
The quantity of differences in terms of the speed or
accuracy of the control that we can have evolves
into qualitative differences in being able to specify at
various levels how sound develops in a composition
(Rowe 2008). In this way, computation reveals new
possibilities for sound. This results in digital tech-
nologies transforming the musical behaviour of new
musical instruments. Digital technologies magnify
our listening behaviour, and result in the acquisition
of new skills in understanding the music, ultimately
allowing us to experience new aesthetics in music.
DMIs become the probes that bring in a new
embodied player–instrument relation with computa-
tionally enhanced variations. Figure 4 illustrates the
performance setup of the Notion of Participative and
Enacting Sonic Interaction – PESI interactive music
system (Tahiroğlu, Correia and Espada 2013). This is
an example of a DMI that enables embodied musical
exploration in co-located collaborative performance,
encouraging reflections about space and movements
(Goddard and Tahiroğlu 2013; Parkinson and
Tahiroğlu 2013).
The strength of our embodied interaction with
the world around us gives us the capacity to reflect
upon the ‘engaged phenomena’ that we encounter in
the form of technology. Any technology could chal-
lenge us in a similar way to how an artwork might
change our view of the world – when we look at a great
painting the world opens up as we act on it (Noë 2012:2).
There are times in embodiment relations where the
technology becomes too easy, too transparent, and
even prescribes actions through the abilities it bestows
upon us. Designers can remove the possibility of
challenging users to critically respond to a technology.
By making everything ready at hand (vorhanden),
ready to use and easy to facilitate, the technological
world empowers itself. Contrary to such a relation-
ship, computational technologies can give us great
technical advancements when designing musical
instruments; as the user can redesign it through pro-
gramming or hardware hacking. Such a relationship
might empower the user in their use (and misuse)
of techno logy, allowing us to discuss the changed
mode-of-being of a musical instrument through other
relations we have with technology. This is what Ihde
calls hermeneutic relations (Ihde 1990: 80).
In hermeneutic relations, users of a technology rely
on their interpretive capabilities to ‘read’ some aspect
of the world through that technology. In this type of a
relation, the technology itself ‘becomes the object of
perception while simultaneously referring beyond itself
to what is not immediately seen’ (Ihde 1990: 82).
The computational nature of the DMI brings in
such hermeneutic relations, another type of digital
transformation, that takes our interpretive sense
of embodiment (perception) to refer to the possible ways
Figure 4. Koray Tahiroğlu performing the piece No More Together written for the PESI system and three-musicians.
PESI provides co-located, embodied and spatial opportunities for musical exploration with on-body mobile
instruments and in-space surround speakers with motion-tracking modules.
70 Koray Tahiroğlu, Thor Magnusson, Adam Parkinson, Iris Garrelfs and Atau Tanaka
https://doi.org/10.1017/S1355771819000475 Published online by Cambridge University Press
in which we interpret sound and music. Such interpreta-
tion appears as a common method of practice in
building and rebuilding DMIs through which we reflect
and think about computational tools (Magnusson
2009). In this hermeneutic relation with computational
tools, DMIs present themselves as probes through
the ways musical sound is transformed into musical
object. Ihde (1990: 96) calls this transformation ‘instru-
mentation’, comprising our actions through technology,
something that also exists in our relationship with
acoustic instruments and is therefore common to
all musical instruments (Vasquez, Tahiroğlu , and
Kildal 2017).
The computational features of DMIs represent
hermeneutic qualities that are new to the domain of
musical instruments. Figure 5 shows the ‘live-coding’
performance in practice, in which coding becomes an
embodied hermeneutic technic, resulting in interpreta-
tions of music not only perceived by the performer but
also by the audience. Taking Ihde’s ideas further, we
could argue that certain sound-generation features in
computer-generated music, such as random processes
applied to audio synthesis algorithms, or generative
algorithms in musical composition, further unfold into
sets of relations that turn the DMI into a probe and
cause it to become the other, constituting a phenome-
nological mode Ihde terms as alterity relations (Ihde
1990: 97).
The alternative and otherness relationship (Ihde
1990: 98) can take place where there exists a unique
type of otherness in which autonomous and intelligent
behaviours are applied to music generation; for exam-
ple, with machine learning and artificial intelligence
tools (Briot and Pachet 2017; Tahiroğlu, Svedström
and Wikström 2015). These advanced computational
technologies have been applied to some DMIs,
forming technologies that are alternative to human
musicians and to commonly practised musical instru-
ments. The computational features of automatic
response behaviours potentially turn a DMI into
an entity, generating relevant musical actions to be
performed, monitoring the music space, performing
musical events that would challenge the human musi-
cal discourse (Tahiroğlu et al. 2015). These advanced
automation features on musical responses bring
independent decision-making technologies into our
relationship with musical instruments. One could
argue over whether the otherness of the DMI is less
strong than the otherness found in an alternative
human musician. The new relationship with DMIs
provides such alterity relations with an equal sense
of interacting with something other than me as dis-
cussed by Ihde, not in the sense of a competitor but
more in line with a dialogue in the co-creation sense.
The otherness for alterity relations with DMIs probes
the possibilities of alternatives to the relationship we
have with musical instruments that are designed as
tools to serve musical objectives. The DMI might
come to be regar ded as having its own objectives
and self-purpose within itself: ‘having a life of its own’.
These relationships we have with technology affect
to different degrees the ways in which music made with
Figure 5. Thor Magnusson live coding at a festival in Bristol’s Arnolfini. The code is written in real time and presented
on the screen, which opens ups for further interpretation by the audience.
Digital Musical Instruments as Probes: How computation changes the mode-of-being of musical instruments 71
https://doi.org/10.1017/S1355771819000475 Published online by Cambridge University Press
DMIs is experienced. Some phenomeno logical modes
may bring certain effects to the fore more directly than
other relations. Ihde introduces the term technological
texturing in background relations to describe the ways
that technological interference is linked to our lived
experience (Ihde 1990: 109). Computational technolo-
gies texture music performance environments in such
a way that an ‘absent presence’ (Ihde 1990:112)oftech-
nology in background relations may transform music
perception. For example, surround sound systems
afford a particular type of musical performance and
perception that opens new dimensions of human–
technology relations. Concert spaces in their ‘absent
presence’ role where their acoustics have a prominent
effect upon the way music is experienced but it is
rarely the object of focus. The environment in which
the music is presented reflects our background relations
with music – something that can be studied by the use of
probes in real-time experiments.
6. KNOWLEDGE IN DMIS
The previous sections explored our relationships with
our instruments, and how we form bonds and interpret
them as extensions of our physical body, as an aug-
mented body-image. We can also explore the qualities
that bestow instruments with this potential. What is it
that gives instruments their character, depth and person-
ality? Asking such a question will quickly take us to
a realisation that the materialities of our twenty-first-
century instruments are heterogeneous and resisting
clear organological categorisations (Magnusson 2017).
We find that acoustic instruments embody certain
qualities of reliable behaviour and continuity, offering
standards in performance and composition. We also
discover the incredible flourishing of experimentation
in instrument design and musical media brought with
electricity, electronic and computational technologies.
Here, our instruments bring new modes of thinking,
performing and understanding sound and music.
Digital technologies are a revolution in this sense, as
algorithmic control makes design flexible, fluid and
redefinable through code. Hardware is here written, like
functional poetry.
Conceiving of the development of instruments
in these three distinct, yet overlapping, material
substrata of the acoustic, electric and digital, we also
find that the instruments incorporate and embody
certain musical qualities. Tunings, scales, time signatures,
tempi and other musical elements are increasingly written
into our tools. The 16-step paradigms of electronic
sequencers and drum-machines, or the 4/4 time signature
and 120 beats-per-minute setups of most DAWs, implic-
itly direct the way we conceive of music-making in ways
that acoustic instruments never did. The result is that
music has become more homogeneous and standardised,
following a technological normatisation in composition,
performance and mixing-mastering techniques (Percino,
Klimek and Thurner 2014).
Mark Fisher recently asked, how might it be, with
the powerful music technologies we have available,
that so much of today’s music has become so homog-
enous and similar to the music of the 1990s (Fisher
2014). We can here consider digital music technologies
as consisting of two strands: the first one is a simula-
tion of historical technologies (notation, piano rolls,
mixers, tape, outboard devices) and aimed at music
production. In the design of these tools, we have
abstracted technologies and practices and implemented
them in the digital domain as software. Through this
abstraction process, processes and musical objects are
amplified, transformed, reduced, rejected, or simply
forgotten. Many of the techniques that were possible
with a multitrack tape machine cannot be done with
a DAW, and some of those were never intentionally
design, or indeed part of the device’s manual. In the
world of acoustic and electronic instruments, the acci-
dental was often embraced and there was a strong
quality of discoverability in these systems, a quality that
is reduced in digital systems.
The other strand of digital musical design could
be conceived of as one of performance (as opposed
to production). Here digital materials (MIDI cont-
rollers, DIY electronics and coding) are applied as
sources for unique assemblages of musical purpose.
Installations, software, apps, instruments, tools and
other systems are made for specific ends in mind, often
uniquely shaped to the creator
’s interests to the degree
that the digital system becomes a piece, a composition
on its own. Here, what previously could be character-
ised as composing work has now transformed into
inventing a system (Magnusson 2019). With the pleth-
ora of diverse ingredients available (sensors, motors,
controllers, code libraries, etc.) an approach to
composition with these materials as compositional
elements becomes more diverse and less streamlined
in alignment with the tools of the commercial software
houses.
Considering how these two strands of design
approaches have influenced today’s music, we see
how the production software strand has homogenised
music (Fisher 2014), but in the performance real-time
strand a new world of musical creativity is emerging.
Fisher’s quest for new music and new musical practices
is less likely to be rewarded through music streamed
from musical subscription providers, but rather to be
experienced in live settings across the world in new
music events, festivals, hacklabs and art contexts.
New music embraces the performative, liveness, materi-
ality and system design and these new musical practices
do not travel as well as an MP3 bitstream on the
internet.
72 Koray Tahiroğlu, Thor Magnusson, Adam Parkinson, Iris Garrelfs and Atau Tanaka
https://doi.org/10.1017/S1355771819000475 Published online by Cambridge University Press
What, then, are these new qualities of digital mate-
rials? How, and to what degree, can we define them as
containing an epistemic dimension: one that functions
differently from the phenomenological perspective
than acoustic instruments? Any designer of a new
DMI will be familiar with how the process of design-
ing the instrument inevitably involves musical
decisions. These might involve deciding upon a tuning
system, a scale, a metric system and other musical
parameters that will affect the music performed. The
violin and the Theremin do not have a scale or a fixed
tuning, but in a digital system any continuous dimen-
sion could easily be reticulated into discrete steps.
Furthermore, those discrete steps (e.g., a scale) could
be changed by the press of a button, thus completely
transforming the quality of the instrument. This can,
of course, also be done with acoustic instruments.
We have movable frets on a sitar and we can use alter-
native tunings on the guitar, but these are relatively
major operations that are rarely done, at least in the
case of the guitar. It is hard to change the tuning of
a flute, clarinet or oboe. The digital brings a certain
fluidity and sculptability to our instruments: through
algorithmic definitions we can transform the function
of the material object with a press of a button.
The openness and lack of definition of digital mate-
rials (a sensor can equally be used in military, sport,
games and music equipment) means that anything is
possible and that, at least this point in time, most
approaches are characteristically unique to the author
or the designer. This blank technological space offers
a method of probing musical possibilities through new
instruments making new music in new contexts. There
is a lot of inventiveness in the field of new musical
interfaces and it is not limited to instrument design.
Rather, what is happening is that musical composition
is moving from operating with symbolic writing of
dots on staff lines or signal writing on phonographic
media to new digital systems that merge the externali-
sation of thought in material, symbolic and signal
technologies. With the democratisation of hardware
and software design through open source and maker
culture, we also open up new performance contexts
and cultural spaces for new music to emerge in.
Inevitably this design is of such high level of function
that it involves new definition of musical spaces, where
technology sets the constraints of the possible, but
at a heightened musical dimension compared with
the constraints we are accustomed to with acoustic
instruments.
7. CONCLUSION
In this article we have pointed to how DMIs can be
used as probes into the nature of new music and
related practices. The context of music-making has
been transformed with computational technologies
and networked com puters, and we must consider
our instrument design as real-time experiments that
sometimes succeed and sometimes fail in shaping the
nature of future music. The text has described how
we, as active practitioners in the sonic arts, reflect
upon the new computational materiality of the tech-
nologies we work with and how they pose a new
mode of being in the performing arts. The figures in
the article demonstrate the diverse practices of the five
authors, which supports the individual sections.
We have applied the notion of cultural probes in
the context of instrument design, as we find that the
design process is a cultural experiment where we gi ve
something out into the world, whether that be an
instrument, installation, code library or the music
itself, and in that process we observe how the work
is received and gain invaluable feedback or return.
The instrument as a cultural probe is equally a test
into the potential of new musical expression, body–
instrument relations, sound in space technology,
performer–audience relationship and many more
aspects of contemporary musicking in a network of
reciprocal relationships. These methods can be formal,
as often happens in NIME development at academic
levels, or they can equally be implicit in the work pro-
cesses only rarely emerging explicitly, yet informally,
through discussion in clubs and hacklabs. These inten-
sive methods of personalising technology through
iterative design approach can be beneficial to other
fields of product design, as music presents a unique
and highly intensive performance form (in terms of
numbers of parameters, bodily training, timeliness,
embodiment, etc.). We have long-term relationships
with musical instruments. To fully appreciate the role
of these designed objects in performance may require
durational research studies. By studying DMIs in this
way, we can think of them as computati onal sonic
entities, as the objects that are fluid, never resting
and continually opening themselves up for new defini-
tions and usage.
Acknowledgements
This work is supported by the Academy of Finland
(project 319946) and the AHRC (UK Arts and
Humanities Research Council - AH/R002657/1).
REFERENCES
Briot, J. P. and Pachet, F. 2017. Music Generation by Deep
Learning-Challenges and Directions. arXiv preprint
arXiv:1712.04371.
Brown, O., Eldridge, A. and McCormack, J. 2009. Under-
standing Interaction in Contemporary Digital Music:
Digital Musical Instruments as Probes: How computation changes the mode-of-being of musical instruments 73
https://doi.org/10.1017/S1355771819000475 Published online by Cambridge University Press
From Instruments to Behavioural Objects. Organised
Sound 14(2): 188–96.
Castells, M. 1996. The Rise of the Network Society. Oxford:
Blackwell.
Cox, A. 2001. The Mimetic Hypothesis and Embodied
Musical Meaning. Musicae scientiae 5(2): 195–212.
Demers, J. 2010. Listening through the Noise: The Aesthetics
of Experimental Electronic Music. Oxford: Oxford
University Press.
De Souza, J. 2017. Music at Hand: Instruments, Bodies, and
Cognition. Oxford: Oxford University Press.
Dourish, P. 2004. Where the Action Is: The Foundations of
Embodied Interaction. Cambridge, MA: MIT Press.
Fisher, M. 2014. Ghosts of my LIFE: Writings on Depression,
Hauntology and Lost Futures. Winchester and Washington,
DC: Zero Books.
Gaver, B., Dunne, T. and Pacenti, E. 1999. Design: Cultural
Probes. Interactions 6(1): 21–9.
Gaver, W. W. 1991. Technology Affordances. Proceedings of
the SIGCHI Conference on Human Factors in Computing
Systems: New Orleans, Louisiana: ACM, 79–84.
Gaver, W., Boucher, A., Pennington, S. and Walker, B.
2004. Cultural Probes and the Value of Uncertainty.
Interactions-Funology 11(5): 53–6.
Goddard, C. and Tahiroğlu, K. 2013. Situating the
Performer and the Instrument in a Rich Social Context
with PESI Extended System. Proceedings of the Sound
and Music Computing Conference. Stockholm, Sweden,
368–75.
Hutchinson, H., Mackay, W., Westerlund, B., Bederson, B. B.,
Druin, A., Plaisant, C., et al. 2003. Technology Probes:
Inspiring Design for and with Families. Proceedings of
the SIGCHI Conference on Human Factors in Computing
Systems. Ft. Lauderdale, Florida: ACM, 17–24.
Ihde, D. 1990. Technology and the Lifeworld: From garden to
Earth (no. 560). Indianapolis: Indiana University Press.
Ihde, D. 2007. Listening and Voice: Phenomenologies
of Sound, 2nd edn. New York: State University of New
York Press.
Ihde, D. 2010. Embodied Technics. Copenhagen: Automatic
Press/VIP.
Inwood, M. (ed.) 1999. A Heidegger Dictionary. Oxford:
Wiley-Blackwell.
Johnson, M. 2008. The Meaning of the Bo dy: Aesthetics of
Human Understanding. Chicago: University of Chicago Press.
Lewis, G. 2007. The Virtual Discourses of Pamela Z. Journal
of the Society for American Music 1(1): 57–77.
Magnusson, T. 2009. Of Epistemic Tools: Musical Instru-
ments as Cognitive Extensions. Organised Sound 14(2):
168–76.
Magnusson, T. 2017. Musical Organics: A Heterarchical
Approach to Digital Organology, Journal of New Music
Research 46(3): 286–303.
Magnusson, T. 2019. Sonic Writing: Technologies of
Material, Symbolic, and Signal Inscriptions. New York:
Bloomsbury Academic.
Malloch, J., Birnbaum, D., Sinyor, E. and Wanderley,
M. M. 2006. Towards a New Conceptual Framework
for Digital Musical Instruments. Proceedings of the
9th International Conference on Digital Audio Effects .
Montreal, Canada, 49–52.
Noë, A. 2004. Action in Perception. Cambridge, MA:
MIT Press.
Noë, A. 2012. Varieties of Presence. Cambridge, MA:
Harvard University Press.
Parkinson, A. and Tahiroğlu, K. 2013. Composing Social
Interactions for an Interactive-Spatial Performance
System. Proceedings of the 2013 International Sound and
Music Computing Conference. Stockholm: KTH Royal
Institute of Technology, 347–52.
Percino, G., Klimek, P. and Thurner, S. 2014.
Instrumentational Complexity of Music Genres and
Why Simplicity Sells. PloS one 9(12): e115255.
Reynolds, S. 2013. Energy Flash: A Journey through Rave
Music and Dance Culture. London: Faber & Faber.
Rowe, R. 2008. Personal Communication. Doctoral
defence, 12 December.
Small, C. 1998. Musicking: The Meanings of Performing and
Listening. Middletown, CT: Wesleyan University Press.
Sudnow, D. 1993. Ways of the Hand: The Organization of
Improvised Conduct. Cambridge, MA: MIT Press.
Tahiroğlu, K., Correia, N. N. and Espada, M. 2013. PESI
Extended System: In Space, On Body, with 3 Musicians.
Proceedings of the 2013 International New Interfaces
for Musical Expression Conference. Daejeon, Republic of
Korea, 35–40.
Tahiroğlu, K., Svedström, T. and Wikström, V. 2015. Noisa:
A Novel Intelligent System Facilitating Smart Interaction.
Proceedings of the 33rd Annual ACM Conference Extended
Abstracts on Human Factors in Computing Systems.Seoul,
Republic of Korea, 279–82.
Ulfarsson, H. 2019. Feedback Mayhem. Compositional
Affordances of the Halldorophone Discussed by its Users.
Proceedings of the 2019 International Computer Music
Conference. New York: ICMC/NYCEMF.
Vasquez, J., Tahiroğlu,K.andKildal,J.2017.Idiomatic
Composition Practices for New Musical Instruments:
Context, Background and Current Applications. Pro-
ceedings of the International Conference on New Interfaces
for Musical Expression. Copenhagen, Denmark , 174–9.
74 Koray Tahiroğlu, Thor Magnusson, Adam Parkinson, Iris Garrelfs and Atau Tanaka
https://doi.org/10.1017/S1355771819000475 Published online by Cambridge University Press
