(Other) Fractal sound synthesis (Theory and practice)

  • 10 Replies
  • 784 Views

0 Members and 1 Guest are viewing this topic.

Offline vasyan

  • *
  • Strange Attractor
  • ******
  • Posts: 90
    • stone voices
« on: April 14, 2019, 05:25:33 PM »
Greetings to all like-minded people in this area!
For decades I have been concerned with the subject of creating sound from fractal images.
But unfortunately, experiments in this direction did not produce the desired results.
Quite by chance I recently stumbled upon one interesting scientific direction - cymatics,
founded by Hans Jenny.
The essence of this phenomenon is the visualization of sound vibrations in liquids, powders and liquid paste.
As shown in the pictures:



I do not exclude that there are computer programs for simulating this phenomenon (I have not studied this question yet).
Although the following images are clearly obtained using computer programs:



This prompted the idea that if there is such a conversion of sound into a picture, then there should be an inverse transformation of the picture into sound. I did experiments on converting images into sound through FFT, like the one used in the Murzin's ANS synthesizer, but in this case such a solution is unacceptable.

As such images use the generation of fractals. As intuition suggests, deep zoom of fractals that look like Buddhist mandalas should be fine:


The following videos is a look on the subject of cymatics:




I will be glad to like-minded people and interesting ideas.  :thumbs:

Linkback: https://fractalforums.org/other-artforms/20/fractal-sound-synthesis-theory-and-practice/2750/

Offline claude

  • *
  • 3f
  • ******
  • Posts: 1546
    • mathr.co.uk
« Reply #1 on: April 14, 2019, 06:59:04 PM »
I do not exclude that there are computer programs for simulating this phenomenon (I have not studied this question yet).
I toyed with similar things a bit a few years ago https://mathr.co.uk/blog/2015-03-25_modes_on_a_plate.html

Quote
This prompted the idea that if there is such a conversion of sound into a picture, then there should be an inverse transformation of the picture into sound.
Only special functions have unique inverses.  And not all pictures may be realizable from sound. But you might be able to construct a mesh-based physical model and see how it behaves, constrained by the light parts being fixed in place?

Offline gerrit

  • *
  • 3f
  • ******
  • Posts: 2191
« Reply #2 on: April 14, 2019, 09:24:20 PM »
Standing waves in 2D structures have been investigated for centuries. The circular membrane has an analystic solution in terms of Bessel functions. No matter what shape, the result will be a set of (inharmonic) eigen frequencies which you can auralize as bell of gong like tones.

General shapes require a numerical solution, and as this is very important for structural engineering, codes and full programs to compute that kind of stuff (usually using Finite Element Modeling) are widely available.

It's possible to transform a human to an image using a camera, but it's not generally possible to convert any image into a human. But you can have fun with face recognition/synthesis software by taking it outside it's domain of application, so your idea could work, but I think right now it's more an idea for an idea.

I would try to interpret the fractal image as a some material structure (as per Claude's suggestion), then take that material object as sound generator using some simulation code. I don't see how that would make interesting sounds, but there may be a way if you're more creative than me.

Have you ever seen the MATLAB logo and wondered what it is? Way back in my student days I was doing stuff like that and had computed the modes of various objects and inserted them in an interactive game-like graphics program where you could touch the objects to make their sounds. It featured an L-shaped coffee table.

A bit later little me got a visit from the big Cleve Moler, the founder of MATLAB. Why? Turns out the L-shape is particularly difficult to compute accurately due to numerical issues of the wave equation at the 270 degrees corner, and this guy worked on that when he was young and never lost his obsession with that shape. To the extent that he put an L-membrane eigenmode on the MATLAB logo where it still is to this day.

More here:
https://www.mathworks.com/content/dam/mathworks/tag-team/Objects/t/72943_92021v00Cleve_L_Shaped_Membrane_Nov_2003.pdf

PS. You may wonder if each 2D membrane shape has a unique frequency spectrum. This turned out to be not the case but counterexamples were not found till I think the 1980-ies. You can find the reference somewhere.

PPS. From each spectrum you can make an infinite number of sounds by summing them with different amplitudes. Some of them correspond to physical interactions like hitting the membrane/plate at some specific location.

Offline vasyan

  • *
  • Strange Attractor
  • ******
  • Posts: 90
    • stone voices
« Reply #3 on: April 15, 2019, 02:46:12 AM »
I toyed with similar things a bit a few years ago https://mathr.co.uk/blog/2015-03-25_modes_on_a_plate.html
Тнаnks!
Do you have source codes to implement this idea?

nly special functions have unique inverses.  And not all pictures may be realizable from sound. But you might be able to construct a mesh-based physical model and see how it behaves, constrained by the light parts being fixed in place?
In my opinion, it is possible in many cases to find an inverse transform having a direct, for example, Fourier transform.
Long live the force of mathematics!
I am not looking for easy ways and am willing to spend years for these studies.  ;)

Offline vasyan

  • *
  • Strange Attractor
  • ******
  • Posts: 90
    • stone voices
« Reply #4 on: April 15, 2019, 03:04:06 AM »
I would try to interpret the fractal image as a some material structure (as per Claude's suggestion), then take that material object as sound generator using some simulation code. I don't see how that would make interesting sounds, but there may be a way if you're more creative than me.
Sometimes very unusual ideas can be the basis for building a sound synthesizer based on them. One time ago, I researched one kind of modulation noise, which resulted in a virtual VSTi synthesizer Marazmator, that creating unusual sound forms. The subject of sounds strikes with the variety, it both sound space plots, and the natural phenomena, sounds of technical character (engines, mechanisms, the tool and тп), different voices, and sound objects which are simply not giving in to any description.


Offline vasyan

  • *
  • Strange Attractor
  • ******
  • Posts: 90
    • stone voices
« Reply #5 on: December 23, 2019, 12:23:14 PM »
I managed to make fractal sound synthesis!
This fractal was taken as a basis:


I unfolded the Mandelbrot fractal in polar coordinates into Cartesian coordinates. Then the resulting image was converted into shades of gray. After that, I converted this image to sound through my Sound Art program.


Sounds like the sound of a winter blizzard ;)

I attached a png file with a fractal image and a conversion profile built into it. In this case, anyone can repeat my experiment.  :thumbs:

Offline iRyanBell

  • *
  • Fractal Fanatic
  • ***
  • Posts: 36
« Reply #6 on: June 25, 2020, 11:20:19 PM »
I gave this a try, using log-scaled sine waves at 100hz - 10khz with the smoothed escape values from one axis used as the wave amplitude evolving across the other axis as time.

Mandelwave (Julia @ -.79 + .15i)
https://clyp.it/k5yfawqg

The results have that sci-fi quality, but aren't too much more interesting than any other random 2D plot -> 1D FFT.  I'd be curious to explore other ways to interpret the equation that produce a more natural audio sample.

Offline C0ryMcG

  • *
  • Uploader
  • *
  • Posts: 215
« Reply #7 on: June 27, 2020, 07:07:14 AM »
Interesting ideas! And interesting sounds too.. At the moment they don't sound like anything specific, but it's still pleasing... Maybe too much chaos?

But I do have another thought. Your approach has been to take a formula, turn that into a 2d plot of points, and turn that into a sound wave. Well, sound is already 2 dimensional, the dimensions being pressure and time. Since we experience sound as a system of simultaneous tones, sometimes sound is shown in 3 dimensions, tone, amplitude, and time (spectral analysis of sound is an example, midi piano roll could be argued as another, although usually quite quantized in the 'amplitude' dimension)

I suspect your program is taking the spectral, 3-d approach, looking at pixels along a column and checking their brightness. A chaotic image would result in a noise-like sound. I'd be interested in hearing the results of a cleaner image, like a border coloring. I imagine we'd be able to much more clearly mentally pick out the different parts as the boundary of the shapes came in and out of view.

I also think it would be worth trying a more direct form of data usage... Something that won't sound very musical, maybe, but might still be interesting is to consider Time as a real axis, a constant for Imaginary, and the escape time of that point (normally used for chosing a color in visual rendering) would indicate the tone, maybe with a bit of logarithm action to stop it from just being high-pitched clicks all the way through.

Offline iRyanBell

  • *
  • Fractal Fanatic
  • ***
  • Posts: 36
« Reply #8 on: June 29, 2020, 10:38:32 AM »
This was another version of the same approach, adding noise cancellation with EQ for the harshness and reverb to enhance the general weirdness of it. https://clyp.it/cru2dk10

The way I'm doing the sine-summing adds a bunch of noise in the process, where that would need to be switched out with a better FFT algorithm. But, I'm not sold on this idea of listening to an image as the best way to translate a fractal into audio.

I did one experiment using tones, but it sounds identical to using a random seed to start some sort of sequencer. This could be improved into a much better sounding composition with high quality samples, but seems fairly removed from what might be the sound of the actual thing.

Another idea was to average columns across a very high-resolution wide render and try to interpret that in some way as audio -- as pitches, it sounded crazy, and as amplitudes, it was just a sort of rhythmic white noise.  :-\

Offline iRyanBell

  • *
  • Fractal Fanatic
  • ***
  • Posts: 36
« Reply #9 on: July 05, 2020, 06:33:39 AM »
I'd be interested in hearing the results of a cleaner image, like a border coloring. I imagine we'd be able to much more clearly mentally pick out the different parts as the boundary of the shapes came in and out of view.
Here's a cleaner implementation using Griffin-Lim phase reconstruction to render an inverse spectrogram of an eclipsed plot with the infinite escapes subtracted.
https://clyp.it/vg3bxh4e

Offline C0ryMcG

  • *
  • Uploader
  • *
  • Posts: 215
« Reply #10 on: July 05, 2020, 09:38:19 PM »
That is great! Has a little bit of rhythm in it in places, I assume where small bulbs show up regularly placed, it has discernible pitches that rise and fall, and simple parts and complex parts. I think this is a success.
 I'm sure there are still other ways of exploring the idea, too, of course...

Thanks for posting that example.


xx
"Time Span"

Started by cricke49 on Fractal Image Gallery

0 Replies
577 Views
Last post August 02, 2018, 07:05:21 AM
by cricke49
lamp
Sound Art — free picture to sound and sound to picture converter

Started by vasyan on Other Artforms

7 Replies
692 Views
Last post June 06, 2019, 06:38:58 AM
by vasyan
xx
OpenGL font rendering

Started by sjhalayka on Programming

2 Replies
178 Views
Last post April 20, 2020, 05:00:47 PM
by sjhalayka
clip
Neural Style Transfer with Fractal Art

Started by reallybigname on Other Artforms

1 Replies
453 Views
Last post July 20, 2019, 04:25:41 PM
by reallybigname
xx
Practice Your Craft Daily

Started by 1Maniac on Mandelbulb Maniacs

2 Replies
467 Views
Last post September 10, 2017, 04:10:26 PM
by 1Maniac