technology platform
sonic experience
This prototype implements our design decisions up to a certain resolution + research process and serves as a platform for iterative prototyping. It consists of a small laptop, a BasicX-24 microprocessor, a USB-MIDI converter as well as biometric and environmental sensors worn by the user. Sensor input is collected by a BasicX-24 microcontroller, which sends them in MIDI format to a small laptop computer (carried in a shoulder-bag) via a USB-MIDI converter. The data is then reconverted and processed for context and action recognition of "if-then" type, and mapped to musical parameters in a modular program created in the interactive environment PD: urban sounds captured by the microphone are processed in real time and turned into music based on sensor input. The music is output through headphones as the user is walking. More information about the prototype is also available on the Viktoria Institute's Sonic-City project webpage.
dataflow
HARDWARE
Current sensors used in this implementation are a metal detector, an IR-sensor measuring proximity o walls and objects, a light intensity sensor, a microphone measuring sound level, and an accelerometer sensing stops, starts, and the starting user pace that determines the music tempo of a whole session. We have also experimented with sensing pollution and temperature and plan on adding a heart-rate sensor. Low-level sensor input such as light intensity or presence of metal are continuously measured and mapped to the music, whereas the context recognition of high-level parameters such as "standing still at night" are updated every other beat. The sensors can be plugged in and out of the platform, thus individual sensors are easily isolated, combinations quickly tested, and damaged components simply replaced. The way the sensors are worn and positioned on the body can influence on what is being sensed and thus on the kinds of musical patterns generated. Therefore, their positioning is designed to be flexible in order to allow for more user control and expression through customisation. In the final version of the system, the sensors will be wireless and the context recognition done on hardware level, allowing for more flexibility and robustness, as well as the possibility to conduct  for more long term evaluation with user in context.
hardware   proximity sensor
SOFTWARE
Structurally, the PD program is composed of small modular units that construct the music algorithmically according to what factors trigger them, and to the values of the data mapped to them. Concrete urban sounds input through the microphone take manifold parallel and serial paths through sound processing objects. These paths can be flexibly deviated and redirected in response to multiple incoming sensor values, resulting in a highly dynamic sonic output. This modularity and the flexibility of the mapping models enable us to easily test various types of musical output. + sound design & mapping.
PD program
sonic experience