Audio Visual

Research

Paul Klee

My friend recommended to look into Paul Klee after we discussed sound notation in writing. He is known for his expressionism paintings. From wikipedia: “(Expressionism) typical trait is to present the world solely from a subjective perspective, distorting it radically for emotional effect in order to evoke moods or ideas.”

I find warmth and peace in this artwork. There’s a lot of shapes with flow and curves that deform and stretch or pull.

Max Cooper collaborations

An electronic artist called Max Cooper often commissions audiovisual videos for his music pieces.

Max Cooper - Symphony in Acid (Official video by Ksawery Komputery)

This audio visual piece can actually be played online in browser at symphonyinacid.net

Max Cooper - Void (Official Video by Jessica In)

I really like this video, entirely monochrome and harsh visuals but does put me in a sort of trance.

Max Cooper - Cyclic (official video by Numbercult)

Work Paul recommended

Ryoji Ikeda

Ryoji Ikeda does sound work that is represented with very digital looking art. His works often installed in giant spaces using mainly monochromatic colours, strobing, and electronic sounds.

I am a fan of their album Dataplex. The album is comprised of very harsh electronic sounds that has occasional rhythms in some parts. It is harsh to listen to at first but I realized it could soothe me and help me be productive.

Test pattern is displayed in such a huge huge space, transporting anyone that goes in to an alien world seperate entirely from the world outside.

Ryoichi Kurokawa

I remember seeing s.asmbli on Instagram and being infatuated. It’s a projection on 5 inner walls of different environments (I think maybe point clouds??) that are appear and move, paired with sounds that perfectly match the visuals. The environments are almost being etched into reality at some points.

Looking further at their work I discovered node 5:5, also using this “etching” sort of thing with lasers and incredible sounds.

Idea forming

Brief ideas that appeared

Experiencing ambient sound from around where sit with effects to enhance.
Moving electronics (like printer heads) with Arduino using the sound from around.
Visualise recorded audio from outdoors with words and colour to convey my experiences.
A sound piece to show journey of being trans. From euphoria to dysphoria to growing as a person.
- I want to exhibit something at the GSASA’s Iridescence event which raises money for trans healthcare needs. I would like my project to be personal in some sense.

Developing idea

Ryoichi Kurokawa’s s.asmbli has inspired me lots to experiment similarly. I like the way the audio sounds like the work is being etched into the world with the sharp industrial tones. I’d like to do something similar for etching words onto a screen.

I experimented with type in the Typography project, Extended Reality project and also just personally. I want to explore this further.

Over the past few months I have started writing more to capture ideas I want, I find it a great way to capture emotions and transport yourself back to times when you felt especially present. I have also been taking photos a digital camera from the early 2000s for a while and would like to explore how to use these.

Development

Resources used

Software

Geomerative - Typography library for Processing
The Midibus - MIDI library for Processing
Voxglitch - VCV Rack 2 modules for sampling audio files
BetterDisplay - Virtual screen of any resolution
Calligraphr - Turn handwriting into fonts
FontForge - Used to merge Calligraphr fonts to get passed paid character limit An idea was to etch out the letters of words. I can do this by deconstructing
OBS - Used to capture the Processing output

Help

“Drawing part of a Bézier curve by reusing a basic Bézier-curve-function?” - Figuring out how to draw part of a bezier curve with variables Geomerative provides
Media Studio BrightSign & CRT Monitor - Media Studio Padlet
Exporting Video for BrightSign / USB stick from Adobe Premiere Pro - Media Studio Padlet

Process

Font

Geomerative

I spent sometime looking through the documentation and figured out how to get the commands used to draw the lettering. I drew part of it by calculating the total length of the word (the perimeter of the lines?) and exiting out drawing it when I reached a progress point.

I implemented using my own handwriting as a font.

VCV Rack 2

I used The MidiBus library to link the sketch up to VCV Rack 2. I trigger CC events when the text is drawn and when photos change in the background.

Field recordings

I captured a bunch of recordings of outside, both walking and with a tripod, using a Zoom H1N recorder. I used these in my patch by having them in a folder and loading them with the Voxglitch modules.

Photo sequence

The photos are loaded from a folder and displayed at random intervals in the sketch. I had some problems with crashing during the program, it turned out to be it was trying to show “.DS_Store” file as a photo, a hidden mac system file in every folder. Bit annoying!

Choosing the sentences

I have to choose what sentences to show during the program. I have sourced many writings. Some are from my writing notebook, others are some song lyrics that I like. I also draw symbols sometimes too, which were drawn on a whim which I feel is needed for a project about presentness.

It loads them from a text file called sentences.txt and shuffles them and then writes them out. I had some more issues with the audio getting hung, which I found was due to empty lines in the text file!

Getting the equipment

I have borrowed different equipment from the media studio to display my project through a CRT TV and hear it through multiple headphones.

Capturing the video

OBS

I used OBS to capture the audio from VCV Rack and the video from Processing.

OBS allows me to easily output the video at 1920x1080 (needed for the Brightsign) and center the Processing output (1440x1080, matches 4:3 of CRT TV and digital camera) with the use of BetterDisplay to easily do this. It also allows me to easily use H.264 (not entirely sure what this is) and a Constant Bit Rate of 25 mbps which is also required for the Brightsign.

Autorun file

Following the padlets for videos on the Brightsign, I had to put a file on the SD Card that automatically scales and loops videos for 4:3 CRT TV. One edit I did in the file was change the view mode so that it chops off edges of 16:9 video to just the 4:3 video so it displays correctly on the CRT TV.

Setup

Photos

The word Peacelove. — The word "Peacelove."

Conclusion

I feel proud of what I accomplished in this two week project. I do want to work on it further and figure out how I’d “dress up” the exhibition, so it’s not just a CRT TV on a plynth, but maybe a nice homely table or a rug pattern similar to my family living room at home.

I will probably also explore different ways the photos can fade out, to see if more “drama” suits the project or not, or just to refine the elements that are already there.

February 2024