nature forms II

Nature Forms II (2015) explores the possibility of recursive re-interrogation of a field recording through visualization and resonification/resynthesis via machine and performative means. In Nature Forms II a range of forms of representation are  used including semantic graphical notation, percussion notation and a hybrid form of sonogram notation. This most literal form of representation is the process I have previously termed the “spectral trace” in which notation is drawn directly onto the spectrogram to represent features of the field recording to be performed by an ensemble of flute (orange), clarinet (red), viola (green) and cello (blue).

“Spectral Trace” notation from Nature Forms II

The graphical symbols are presented on a proportional staff created to solve the problem that horizontally, the spectral trace is temporally proportional and performing with precision can be achieved through a scrolling score, however it is also vertically proportional – each pitch occupies a distinct vertical spatial position.

Vertically proportional stave showing a chromatic scale from E3 to F4.

The system was developed for the work here, apparently, there was time for everything [2015], and attempted to more-or-less retain the topographical layout of the traditional stave, while adding coloured lines to accommodate non-natural notes. A second approach draws on the concept and techniques developed in EVP (2011) and Lyrebird: Environment Player (2014).

The amplitude of the frequency of the single strongest detected sinusoidal peak in a recording is represented by the size of the rectangles drawn on a scrolling LCD obect (in this case jit.lcd). ​The “lyrebird” generative score process was also employed in Nature Forms II, to create high, middle and low frequency visualisations of the field recording.

Detail from the high frequency “lyrebird” score of Nature Forms II [2015] (excerpt).

Percussion part for Nature Forms II created using the generative score software “Semantics of Redaction” (excerpt).

​The final form of visualization in Nature Forms II is derived from the work Semantic of Redaction (2014) in which notation in generated by using accents detected in a speech recording in real-time to generate graphical symbols of varying vertical position, size and colour, determined by the frequency, amplitude and timbre of a speech recording at the accent point. In Nature Forms II the software detects accepts created by sharp attacks such as the chirping of crickets.

​A number of approaches to sonfication/resynthesis are used in Nature Forms II. The first is an additive synthesis approach using a patch called Sinereader to resonify greyscale spectrogram images .

Spectrogram of additive synthesis from Nature Forms II (excerpt).

Spectrogram of Ring Modulation synthesis from Nature Forms II (excerpt).

​Processes were also developed to resynthesise sounds using spectral analysis. In the first process, the strongest sinusoidal component detected each 40ms of the recording were resynthesized with a sinewave that was ring-modulated according to the currently detected brightness of the recording.  The aim was to retain the amplitude of sonic features of field recording while maintaining and equivalent brightness.

​This method is also used in portions of the performance to ring modulated the instruments via live processing to mimic the parametric brightness of the field recording by subtracting the brightness of the live instruments from that of the field recording to derive a ring modulation value.

Spectrogram of subtractive synthesis from Nature Forms II (excerpt).

​Subtractive synthesis was also employed by using frequency and amplitude detected in the recording to bandpass white-noise. At the opening of Nature Forms II “coloured noise” performed by the instrumentalists is gradually shaped into the sonic structure of the field recording using subtractive synthesis, and then cross-faded with the source recording.

Nature Forms II explores the notion of eco-structuralism, maintaining what Opie and Brown (2006) term the “primary rules” of “environmentally-based musical composition”: that “structures must be derived from natural sound sources” and that “structural data must remain in series”.
The structure of the original work is conserved using the approach discussed in the miracle of the rose, where the temporal proportionality of the recording is retained by aligning multiple notation and resynthesis versions of the recording in visual representations that can be alternated or combined in the creation process of the score, processing and fixed media.
The work also uses “spectral freezing” of components of the field recording to create spectrally derived chords from features of the recording bird sounds and a rusty gate which are then transcribed into notation for the instrumentalists and temporal manipulation of the recording to allow complex bird calls to be emulated in a human time-scale.

Visual representation of temporally proportional alignment of multiple resynthesis (a.- e.) and notation (e. – h) versions of the recording in Nature Forms II (excerpt): a. field recording spectrogram; b. additive synthesis spectrogram; c. ring modulation resythesis spectrogram; d. subtractive synthesis spectrogram; e. spectral “freeze” sonogram/score f. “lyre- bird score; g. “semantics of redaction” score; and h. “spectral trace” score. 

Recording generated with data from the source field recording using additive, subtractive and ring modulation resynthesis.