Mandelbulber - VFX Feature Ideas

  • 17 Replies
  • 554 Views

0 Members and 1 Guest are viewing this topic.

Offline stilikon

  • *
  • Fractal Fanatic
  • ***
  • Posts: 30
« on: October 23, 2018, 02:56:30 PM »
Hey Guys,
me once more.

I thought as I am working my way deeper into Mandelbulber and the way to use it in our VFX Pipeline, I write a Post that summarizes my thoughts and wishes for Mandelbulber features.

Me and my Team are working on quite a big Cinematic VR Experience (prerendered Stereo 360░, 4K, 7min, Full-CG), called "Strands of Mind". (if you want to have some more detailed info I can give a project portfolio with PM)
It is a 2 year graduation project at the Animation Institute in germany.
A big part in the film are psychedelic Fractal Worlds, where we chose Mandelbulber as our weapon of choice. All the rest of our Pipeline is Houdini and Nuke.

So we are trying to incoorporate the Fractals into our VFX Pipeline as much as possible.
And of course we are also happy to share the stuff we develop for Mandelbulber with the cummunity.

This includes Farm Rendering on our Renderfarm (20.000 Ghz, 230 clients).
This is almost done. Its rather specific to our Renderfarm though. (native NetRender is not practical for our renderfarm)

The other big part is combining it with other 3D and 2D elements.
For this I have a couple of wishes and ideas, that would be great to have in Mandelbulber.


 I know you guys are all doing this as a hobby and I really appreciate your work!
So I am totally aware that some of the stuff I have in mind might be too time consuming or hard to implement. But I suggest it anyways, maybe some might work out (-;


Looking forward to what you think of all this!
Best!
Adrian




_______________________________________________________________
- Proper 32 Bit EXR Output

This looks like it has already been fixed, thanks so much


_______________________________________________________________
- Fix latlong poles for equirectangular stereo rendering. Including "Pole Merging".

When rendering stereo latlongs, there is two problems that arise. On the poles there are swirly artefacts, maybe from some problems in the lens shader.
The other is, the stereo effect (offset of the cameras) needs to be "merged", meaning smoothly reduced down to 0 on the top and bottom poles.
As stereo can only be renderd in one "stereo axis". This is done in all other 3D Rendering engines, and called pole merging.
From the stereo latlong renderings I tested with Mandelbulber its seems there are some problems with this, or its not implemented?
Maybe you can sahre some insights on how you approched this so far?


_______________________________________________________________
- Implement Stereo Rendering in OpenCL

This one would actually just be a nice to have, but not so important, as we are mainly rendering the fractals on the farm on CPU anyways.
But still i think it would be great to have it for Mandelbulber.


_______________________________________________________________
- Camera Export

It is super imprtant to us to get the Camera Animation from Mandelbulber.
Right now we would try to "hack out" the keyframe values out of the scene file, with python, then rebuild the camera from that data in Houdini and export as Alembic (we would have to know what kind of interpolation you use though).
This is a bit hacky though and we dont know yet if it would work well. Of course it would be nicer if there was a build in option to export the camera as FBX or Alembic.


_______________________________________________________________
- AOVs / Enable to export more image layers then Color, Normal, and ZDepth.

For compositing it is crucial to have different layers (AOVs) from the render.
As I understand right these layers must be present internally of Mandelbulber anyways.
It would be great to be able to export them seperately.

The most important here would be a "World Position" pass. (This would enable us to do a complete relighting in compositing)
Other very usefull ones:
Diffuse, Specular, Ambient Occlusion, Emission, Fog


_______________________________________________________________
- Fractal Noises (pseudorandom gradient noises like Perlin / Alligator etc.)

I guess this one is a bit of wish thinking here, but it would be such an enrichement for Mandelbulber to also have "normal fractal noises" available.
They could be mainly used in the Fog Shading. But would also be great as "World Space Deformers", transform functions for the fractals.
So you could for instance add a bit of organic looking Alligator Noise on top of your Amazing Surf surface or whatever, or distorting the coordinate space of it.
Don't know how tough that would be in the real fractal world :-P



_______________________________________________________________
- And finally the probably very wishfull thinking: Timeline Keyframe Animation.

To be honest, the Keyframe and Flight Animation workflows in the fractal world are somewhat strange and unflexible for people coming from the VFX world :-P
Being able to have a normal timeline and then set keyframes freely on this timeline would blast the fun of working with animation in Mandelbulber.

Online claude

  • *
  • 3d
  • ****
  • Posts: 986
    • mathr.co.uk
« Reply #1 on: October 23, 2018, 03:09:53 PM »
Quote
we would have to know what kind of interpolation you use though
I only tried briefly, but iirc the interpolation can be chosen, and I was surprised the default wasn't Catmull-Rom...

Offline mclarekin

  • *
  • Fractal Frogurt
  • ******
  • Posts: 450
« Reply #2 on: October 24, 2018, 04:15:30 AM »
Quote
Camera Export
For navigation the default is akima, but you can change to catmull_rom or linear.
I export in CSV , and import into spreadsheet.


I am thinking about interpolation at the moment. But there can be problems with using cubics. (as shown)

In respect to morphing parameters, there are situations where none of the existing interpolation methods produce what I want, so I am evaluating other methods.


Quote
- And finally the probably very wishfull thinking: Timeline Keyframe Animation.

yes, later. Maybe everything will be accessable in an editable table format

Offline zebastian

  • *
  • Fractal Friend
  • **
  • Posts: 14
« Reply #3 on: October 25, 2018, 10:54:10 PM »
Hi Adrian,

here are my thoughts on your feature suggestions:

> ...I can give a project portfolio with PM)...
That would be interesting!

> .... Animation Institute in germany. ...
Cool, where in germany are you? I am from the linken Niederrhein ;-)

Transforming perspecitive is buddhis topic.  So the topics here are:
- artifacts in pole area
- pole merging

Implement Stereo Rendering in OpenCL
Also mostly buddhis topic...

Camera Export
I can look into this, sounds interesting.
Which data would be needed in the output? I would tend to use alembic, since its an open standard: http://www.alembic.io/
tracked here: https://github.com/buddhi1980/mandelbulber2/issues/575

- AOVs / Enable to export more image layers then Color, Normal, and ZDepth.
Yes. In the old mandelbubler V1 it was possible. Krzysztof and I also prepared the code in v2  for arbitrary image data channels when we implemented surface normals.
I can look into fitting these loose strings. and add possible more channels, like lighting, specular, ...
tracked here: https://github.com/buddhi1980/mandelbulber2/issues/576

- Fractal Noises (pseudorandom gradient noises like Perlin / Alligator etc.)
This is probably a topic for graeme and buddhi :D

- And finally the probably very wishfull thinking: Timeline Keyframe Animation.
Yes, that would be great. Like its done in blender. But this will require a LOT of work. It would be great if we could find an incorporatable  library for the visual part...

Cheers!
« Last Edit: October 27, 2018, 03:13:16 PM by zebastian »

Offline zebastian

  • *
  • Fractal Friend
  • **
  • Posts: 14
« Reply #4 on: October 27, 2018, 03:53:12 PM »
@adrian:
Can you tell me what is the recommended way to incorporate the additional image channels (specular, diffuse, ...) into EXR?
Is there a naming convention for this? For example the naming for normals is n.X, n.Y, n.Z

Offline stilikon

  • *
  • Fractal Fanatic
  • ***
  • Posts: 30
« Reply #5 on: October 29, 2018, 08:21:28 PM »
@adrian:
Can you tell me what is the recommended way to incorporate the additional image channels (specular, diffuse, ...) into EXR?
Is there a naming convention for this? For example the naming for normals is n.X, n.Y, n.Z

Sure, awesome that your on it.
there's not a strict convention thats similar in all DCCs.
But mostly it's:

depth (Z)
n (X.Y.Z)
p (X.Y.Z) {world position}
diffuse (R.G.B)
specular (R.G.B)
reflection (R.G.B)
refraction (R.G.B)
emission (R.G.B)
ao (R.G.B) {ambient occlusion}
volume (R.G.B) {fog}




Offline stilikon

  • *
  • Fractal Fanatic
  • ***
  • Posts: 30
« Reply #6 on: October 29, 2018, 08:48:43 PM »
Hey Sebastian,

> ...I can give a project portfolio with PM)...
...That would be interesting!

Sure, I'll send you a PM tomorrow.



> .... Animation Institute in germany. ...
...Cool, where in germany are you? I am from the linken Niederrhein ;-)

Oh cool. Are there even more german Mandelbulber developers ?
It's in Ludwigsburg, close to Stuttgart.
https://animationsinstitut.de/en/



...Transforming perspecitive is buddhis topic.  So the topics here are:
...- artifacts in pole area
...- pole merging

What do you think buddhi? (-;



...Implement Stereo Rendering in OpenCL
...Also mostly buddhis topic...



..Camera Export
...I can look into this, sounds interesting.
...Which data would be needed in the output? I would tend to use alembic, since its an open standard: ...http://www.alembic.io/
...tracked here: https://github.com/buddhi1980/mandelbulber2/issues/575

Awesome! This is gonna be quite a game changer!
Yes Alembic is the best way to go and most used across all DCCs.


...- AOVs / Enable to export more image layers then Color, Normal, and ZDepth.
...Yes. In the old mandelbubler V1 it was possible. Krzysztof and I also prepared the code in v2  for arbitrary image ...data channels when we implemented surface normals.
...I can look into fitting these loose strings. and add possible more channels, like lighting, specular, ...
...tracked here: https://github.com/buddhi1980/mandelbulber2/issues/576

Game Changer #2 (-:
So cool, I was afraid it would be to hard to do. Already replied you concerning the naming conventions.

Two additional ideas here (just some stuff which would make this even more awesome, but I'm already happy if the basic AOVs like position diffuse and specular work, world position actually being most important):
In almost all DCCs, also in Blender, you can export so called "per light AOVs".
That means each AOV, like diffuse, gets split up into different channels for each light (or light groups), containing just the diffuse information lit be each separate light.
The channels of the EXR then look something like this:
diffuse_light1
diffuse_light2
specular_light1
specular_light2
...
This enables you to recolor the light in post, adjust the brightness and animate it (like a strobe for instance), which is really awesome and gives you a lot of create freedom without rendering.

The other idea:
(we implemented this into our Houdini fractals and it was quite practical)
Being able to export the orbit trap (or other method) used for coloring as a seperate AOV, as a black to white gradient. This also enables you to recolor the fractal in post, and do very controllable animations of the coloring in post.



...- Fractal Noises (pseudorandom gradient noises like Perlin / Alligator etc.)
...This is probably a topic for graeme and buddhi :D

Any thoughts guys?



...- And finally the probably very wishfull thinking: Timeline Keyframe Animation.
...Yes, that would be great. Like its done in blender. But this will require a LOT of work. It would be great if we could ...find an incorporatable  library for the visual part...

Yeah, I guessed it would be a lot of work. But yeah, exactly like in Blender. This system just makes a lot of sense.
If you want I can investigate a bit if there is some library for the visual timeline part.



Thanks so much, looking very much forward to a more VFX friendly fractal world (-;

Offline stilikon

  • *
  • Fractal Fanatic
  • ***
  • Posts: 30
« Reply #7 on: October 29, 2018, 08:57:03 PM »
PS.
it would be great if you could let me know if you think it will work out exporting the "world_position" (p.XYZ) AOV.
If not, we will try to come up with something to recreate it out of the depth.Z AOV and the camera position and start with that. But of course it would be way more convenient to have it directly and precise.

Offline stilikon

  • *
  • Fractal Fanatic
  • ***
  • Posts: 30
« Reply #8 on: November 05, 2018, 11:32:02 AM »
Hey guys,
any news? (-;

Thanks
Adrian

Offline zebastian

  • *
  • Fractal Friend
  • **
  • Posts: 14
« Reply #9 on: November 05, 2018, 11:42:25 AM »
Hi Adrian,

we are right now in feature stop, since we are right before releasing the next stable version.
After the release we will start with the next heavy development cycle.

Hopefully we can finish the next release this week.

Offline stilikon

  • *
  • Fractal Fanatic
  • ***
  • Posts: 30
« Reply #10 on: November 05, 2018, 01:22:14 PM »
Alright, sounds good!
Good to know though.
Thx

Offline stilikon

  • *
  • Fractal Fanatic
  • ***
  • Posts: 30
« Reply #11 on: November 20, 2018, 11:20:40 AM »
Hey guys,
thanks for the 2.15 release.
Saw the .EXR linear issue is already adressed. Nice!

Let me know when you start working on some of the other feature ideas. In case you have questions or anything.
@buddhi:
zebastian mentioned that some of the suggested topics are your area. Do you have any thoughts on the ideas? Especially the fix for latlong poles and stereo pole merging?

Cheers!
Adrian

Offline buddhi

  • *
  • Strange Attractor
  • ******
  • Posts: 86
    • Mandelbulber GitHub repository
« Reply #12 on: November 20, 2018, 06:55:54 PM »
_______________________________________________________________
- Fix latlong poles for equirectangular stereo rendering. Including "Pole Merging".

This needs some investigation. I think it would be implemented as an option. I will do this when I work on stereoscopic rendering with OpenCL.

_______________________________________________________________
- Implement Stereo Rendering in OpenCL

Will be but after multi-GPU rendering and NetRender for OpenCL. You have to be very patient.

_______________________________________________________________
- Camera Export

It is not complicated so probably will be in 2.16

_______________________________________________________________
- AOVs / Enable to export more image layers then Color, Normal, and ZDepth.

No more layers are rendered in the program. However it could be possible to store results of shaders output in special layers activated on demand (to save memory for regular use). Implementation of it is not trivial. I don't think that it will be possible to reassemble the image from these layers in 100%, because calculation of volumetric shaders cannot be divided into layers. The image will look different.
I will look how much work it would be needed to implement it.

_______________________________________________________________
- Fractal Noises (pseudorandom gradient noises like Perlin / Alligator etc.)

Huge topic. I also had this idea. It need to wait, because now we have a lot of another priorities.

_______________________________________________________________
- And finally the probably very wishfull thinking: Timeline Keyframe Animation.

This is even bigger topic. It would be very good to have something like this but there is a lot of programming to make specialized UI elements and flexible morphing engine.

Offline stilikon

  • *
  • Fractal Fanatic
  • ***
  • Posts: 30
« Reply #13 on: November 21, 2018, 12:55:59 PM »
Thanks for checking on that buddhi!
I will answer below...


_______________________________________________________________
- Fix latlong poles for equirectangular stereo rendering. Including "Pole Merging".

This needs some investigation. I think it would be implemented as an option. I will do this when I work on stereoscopic rendering with OpenCL.

_______________________________________________________________
- Implement Stereo Rendering in OpenCL

Will be but after multi-GPU rendering and NetRender for OpenCL. You have to be very patient.


Cool, it would be awesome if you could investigate on the pole merging and latlong pole errors.
OpenCL Stereo rendering is not that important for me.
But I would guess the latlong problems are just a little modification of the latlong lens shader.
And without this the stereo equirectangular rendering feature is not really usable unfortunately/-:
I researched a bit and found the blender developement page on the topic.
They show all the code they implemented for this. So you might be able to just rip it off (-;
So in case you can manage to address this before the big OpenCL topics, I would be very very happy and thankfull :-P

https://developer.blender.org/D1998
Some further info on the topic:
https://docs.chaosgroup.com/display/VRAY3MAX/Stereo+3D+Camera+%7C+VRayStereoscopic#Stereo3DCamera%7CVRayStereoscopic-Panoramicpolemerging
http://pedrofe.com/oculus-camera/



_______________________________________________________________
- Camera Export

It is not complicated so probably will be in 2.16


Awesome! (-:
Just so we can plan a little, because some other tools that we have to develop depend on this, what is roughly your schedule for 2.16?



_______________________________________________________________
- AOVs / Enable to export more image layers then Color, Normal, and ZDepth.

No more layers are rendered in the program. However it could be possible to store results of shaders output in special layers activated on demand (to save memory for regular use). Implementation of it is not trivial. I don't think that it will be possible to reassemble the image from these layers in 100%, because calculation of volumetric shaders cannot be divided into layers. The image will look different.
I will look how much work it would be needed to implement it.


I think its not that important to be able to re-assemble the image 100%. You still have the normal Color output as well, the other layers would just add a lot of flexibility.
As I mentioned, one of the most important things would be a World Position Pass, without it the existing Normal Pass is not really usable in practice.
(and having the Normal Pass also in world postion (not linked to the camera) would be great as well)



_______________________________________________________________
- Fractal Noises (pseudorandom gradient noises like Perlin / Alligator etc.)

Huge topic. I also had this idea. It need to wait, because now we have a lot of another priorities.


Yeah, thought its a huge topic, well that might have to wait a bit.


_______________________________________________________________
- And finally the probably very wishfull thinking: Timeline Keyframe Animation.

This is even bigger topic. It would be very good to have something like this but there is a lot of programming to make specialized UI elements and flexible morphing engine


Same here.




Thanks for investigating!!

Offline stilikon

  • *
  • Fractal Fanatic
  • ***
  • Posts: 30
« Reply #14 on: January 10, 2019, 11:05:41 AM »
Hey guys,
happy new year!

Just wanted to ask if there is any thoughts or progress on the topics?

Thanks!


xx
feature request ideas

Started by claude on Fragmentarium

33 Replies
716 Views
Last post February 09, 2019, 02:33:15 PM
by claude
question
Just an ideas about Mandelbulb3D

Started by Alef on Mandelbulb3d

0 Replies
140 Views
Last post December 27, 2018, 03:41:01 PM
by Alef
xx
Forum Feature Request - Threaded messages

Started by Sabine62 on Discuss Fractalforums

11 Replies
438 Views
Last post September 10, 2017, 08:13:57 PM
by Sabine62
xx
Feature Request: OpenCL Streoscopic Mode

Started by stilikon on Mandelbulber

4 Replies
294 Views
Last post August 04, 2018, 05:01:10 PM
by stilikon
clip
feature request - make the gamepad support required

Started by JerryS on Mandelbulber

5 Replies
382 Views
Last post June 13, 2018, 04:11:22 AM
by JerryS