[Csnd] [semi-OT] "synthesizing" graphics from audio

Hey hey,
just today someone asked, if I couldn't add a little more interesting image content to my music on Youtube. I had been thinking about that before.

Does anyone here know of a text programmable tool that can create abstract images from music? Based on some kind of audio analysis perhaps? Can Csound be used to achieve that? I was thinking of abstract geometric shapes in varying colours. It's been a while since I've seen a desktop audio player app on an actual screen, but some of them used to have similar features...

If you can think of something, it could be a Python library, that can load audio and produce and save moving images, I'd be grateful.x!

Thanks and best wishes,

Jeanette

Hi,

I don't know any out-of-the-box solutions but you can use PIL to draw graphics as frames and then string those together into video using ffmpeg or similar. I used this synchronised with Csound for a couple of experimental videos - https://www.youtube.com/watch?v=BTTvcESpaq4 ; https://www.youtube.com/watch?v=qW-Igs75fEE

The image opcodes in Csound (imagecreate) could be used similarly to create individual frames, but are very low level - not used them much.

moviepy (MoviePy documentation — MoviePy documentation) is a useful python library for creating more detailed videos as well but more geared towards manipulating existing video.

Also I've created an experimental plugin for SDL graphics within Csound - this outputs to the screen rather than a file, but can be quite easily used for basic shapes etc.

With all these, there's work to do in mapping audio features to visual features though..

Hi Jeanette

A few web-based options:

You could explore p5.Csound (https://rorywalsh.github.io/p5.Csound/#/) so you can run your Csound patches and have bi-directional communication with p5.js

Another Csound wasm option to explore is CABLES.gl if you fancy something a bit more visual programming like Pure Data.

Hydrasynth is really cool, though I’ve yet to see a direct integration, but I think something could be done. You can however send midi events to have visuals react :slight_smile:

Other than that:

Openframeworks (bit more involved), Touchdesigner, Game engines: Unity, Unreal — but a bit more involved

Sorry — Hydrasynth = Hydra (https://hydra.ojack.xyz/)

The former is an actual synth

Hi Richard and S.,
thanks both of your for your suggestions.

@Richard: I've taken a look at PIL (Python Pillow), which comes closest. Though I think with that particular interface, I might almost return to POV-Ray and work that way. Perhaps not that reactive in the end. :slight_smile:

@smills: sorry all those are out. Can't use that GUI stuff, which is why I particularly asked for text-programmable stuff. I should have made myself clearer, in saying that I need something that can be developed and run purely from the commandline.

I love the Hydrasynth, though haven't come across the other, even though I'm aware of a few video synthesizers, mostly in hardware, they get shown at synth events now, which is great in itself.

Best wishes,

Jeanette

I did not know that cables has a csound implementation yet. Are there some example patches?

Hydrasynth does look quite cool.

I do actually love POV-ray - in a nostalgic and perhaps slightly masochistic way!
Incidentally there's a Python interface for it - GitHub - Zulko/vapory: Photo-realistic 3D rendering with Python and POV-Ray - which may be of use.

Hi Jeanette,
you could also try processing + csound via OSC. I’ve attached a simple example that should work on the fly.
Rory Walsh made some years ago an integration of processing and csound.
Hope it helps.
Best,
Luís

Csound mailing list Send bugs reports to Discussions of bugs and features can be posted here

visualmusic.zip (3.15 KB)

Hi Richard,
thanks for the Python POV-Ray library. that might really be the way forwards. So I can combiine two things that I know. I see why you might remember it with some frustration. But if you want to do some graphics design while being blind - or as close as makes no difference - POV-Ray is absolutely ACE! :slight_smile:

Best wishes,

Jeanette

Ah great, I am glad that may be helpful.
And yes, I can absolutely understand, while 3D modelling has moved to be quite visual in design, for text scene description and scripting, POV-Ray is fantastic!

Hi Luis,
thanks for the example. Do you know how you could run Processing on the commandline? The OSC control is a nice feature.

Best wishes,

Jeanette

Yes of course, sorry https://cables.gl/p/tjO3wp

It took some finagling - mainly importing the csound js

I just need to abstract out the setup to make the op less tied to the patch