Fractalforums

There are a bunch of fractal programs with their own file formats for storing iteration data:

- MandelMachine has MMIT
- Kalles Fraktaler has KFB
etc

It would be cool to have a standard file format that fractal programs can use for interoperability.

I propose using OpenEXR 2.0 https://www.openexr.com/ as it is licensed permissively, and is widely used / tested already.  It is flexible (supports any number of float32 and uint32 channels), with multi-image files, and deep images with variable number of samples per pixel.  The main thing is to standardize on the channel names (which are strings).  I propose these to start with (all channels are optional):

uint32 N integer iteration count
  0xFFFFFFFF is non-escaped before header metadata field int Iterations (or string Iterations, it can exceed the range of int, sigh...)
  0x00000000 is uncalculated/glitch/no-data-available.  If actual iteration values can be zero or negative, add a bias constant to each count and store it in the header metadata field int IterationsBias (or string IterationsBias, it can exceed the range of int).  the bias could be negative, this might allow you to store high iteration counts without necessarily needing two channels if the actual min/max range is small enough)
for images with biased iteration counts above 0xFFFFFFFE, split into two channels:
uint32 N0 least significant 32 bits
uint32 N1 most significant 32 bits; 0xFFFFFFFF is interpreted as non-escaped, regardless of the value of N0 (but it should also be set to 0xFFFFFFFF)
for future supercomputers, this can be extended with N2 etc...

genType NF fractional iteration count, expected to be in [0.0 .. 1.0) for float32 and half (float16), and the full range normalized by dividing by 0x100000000 for uint32. The continuous iteration count (when escaped) is N+NF-IterationsBias.  This is stored separately to avoid losing precision at high iteration counts

floatType DE  signed distance estimate, normalized such that distance to a neighbouring boundary pixel is approximately 1.0 for exterior distance estimate and -1.0 for interior distance estimate.  If there are exterior de and no interior de available (or vice versa) the missing data can be written as 0.0, but readers should also handle NaN in this case.

floatType DEX, floatType DEY directional distance estimate in cartesian form, normalized such that distance to a neighbouring boundary pixel sqrt(DEX^2 + DEY^2) is approximately 1.0.  If some pixels have no direcitonal DE the missing data can be written as (0.0, 0.0), but readers should also handle NaNs in this case.  The vector should point away from the fractal boundary.

floatType ZX, floatType ZY coordinates at the first escaped iteration.  Image should have a header attribute floatType EscapeRadius.

Orbit traps can be handled as layers with TRAP somewhere in the path:

uint32 TRAP.N iteration number at which trap was hit, see above for large iteration counts

floatType TRAP.TD distance to trap (eg an x-axis trap would have |y-coordinate|) (normalized however you like, as deep images may have tiny values (perhaps?), but should be monotonic, it could eg be negative if you take logs for more range than float32)

floatType TRAP.ZX, floatType TRAP.ZY coordinates at the trap iteration, should be normalized so that the magnitude at the edge of the trap is 1.0  (can't use just X and Y because Y is already claimed in EXR for colour luminance).

Deep images could be used to accumulate all the orbit traps hit over the lifetime of the pixel's iteration.  More research needed on this.


Feedback very welcome!  There's not much point to this exercise if the only software implementing it is KF (it doesn't do it yet, but I will work on it soon).


EDITED 2019-09-09 to add notes about string attributes for Iterations and IterationsBias, as well as allowing negative bias; added DEX/DEY spec

Linkback: https://fractalforums.org/programming/11/towards-a-common-file-format-for-2d-escape-time-iteration-data/3017/

https://github.com/openexr/openexr/blob/master/OpenEXR/IlmImfExamples/generalInterfaceExamples.cpp seems to be a good source of simple examples

Can you think of a way to make the format extensible, all the way up to say storing the full orbit at each  pixel or say  color = standard deviation of orbit around mean (just making that up)? A comment section which describes what is being stored?

The OpenEXR file format is inherently extensible, and with Deep Images you can store that, the problem is the metadata, either you engineer some extravagant system for describing everything precisely, or you start a bit ad-hoc and come to some standardization after seeing what works in practice (ideally with more than one implementation)...

Here's my first test code, using basic OpenEXR is very simple.

http://code.mathr.co.uk/kf-extras/blob/be57a7df18f785652ad43f84a555ee6bf30e5861:/kfb-to-exr.cc

$ kfb-to-exr test.kfb test.exr
$ exrheader test.exr

file test.exr:

file format version: 2, flags 0x0
Iterations (type int): 200
channels (type chlist):
    DE, 32-bit floating-point, sampling 1 1
    N, 32-bit unsigned integer, sampling 1 1
    NF, 32-bit floating-point, sampling 1 1
compression (type compression): zip, multi-scanline blocks
dataWindow (type box2i): (0 0) - (639 359)
displayWindow (type box2i): (0 0) - (639 359)
lineOrder (type lineOrder): increasing y
pixelAspectRatio (type float): 1
screenWindowCenter (type v2f): (0 0)
screenWindowWidth (type float): 1
type (type string): "scanlineimage"
Note the iteration count (int 200) stored in the header.  You can store StringAttribute too, for example I will reserve one keyword ("KallesFraktaler2+") to store KFR+KFS parameter strings.  Then think about metadata describing viewing parameters, formulas, etc, but the main focus is on iteration data to start with.  For KF purposes I need to think about adding another channel for glitch status information, because that is lost so far.  Maybe treat it like an orbit trap.

My local KF working branch can now save and reload EXR as described in the first post (only N / NF / (sometimes) DE as KF doesn't do orbit traps yet).

Note that I edited the first post with two things, storing maximum iteration count in the "int Iterations" field in the header, and also applying a bias to iteration counts as the channel is unsigned, but normalized iteration counts can go negative.

Hi claude,

I'm really happy to see some momentum with this.  Great idea going with EXR. 

For my small part, I will add support for my After Effects plug-in once you have worked out the format. 

Maybe in the future, once we build an open-source "Movie Maker", there will be very good reasons for most people to support the format.  I'd suggest that you take the lead on this, and publish a format.

Some minor thoughts...

With DE, can we get an angle, not just a distance?

Nice thoughts on orbit traps (are they coming to KF?)...  Making it extensible enough to store 2 or 3 different user-defined traps in a file would be great.    Can I suggest that you should define how the distance to the trap is normalised (maybe 1.0 is the width of the current frame?)  An angle to the trap might also be useful, but I guess it can be calculated if you have a location.

Side note...  I'm not sure on this, but going with EXR might be a genius idea, because I think some 3D software can read it as texture data.  It would be cool to be able to directly load the DE data as a displacement map.   Not sure if it is possible, but I will investigate once you put out he next version of KF.

the problem is the metadata, either you engineer some extravagant system for describing everything precisely, or you start a bit ad-hoc and come to some standardization after seeing what works in practice

-
How about describe everything in a Doc (like phpDoc) system, where you define annoations in comments? I think this is the most straightforward and standard way of writing metadata to code.

What is the benefit of this compared to KFB? KFB is already used by several other programs and has become kind of a standard. Other programs that use KFB files are:
SeryZone's map visualizer
My program to convert KFB to MMIT
Kalles Fraktaler's own video maker
Tools from Gerrit that I don't have but I know that he uses KFB files
Saka's After Effects plugin (I see you replied here.)

The one other file format you mention that is part of the problem of there being too many is MMIT from mandel machine, which is unlikely to be developed further. There have been no updates for years.

0. KFB file format won't go away for a long time, I just won't add new features to it
1. EXR is extensible in a sane way - can add additonal channels and/or metadata very easily - adding this to KFB would make the files backward incompatible anyway, not much to lose by switching format entirely
2. there is a software library (openexr) under permissive license that can read and write it, which makes it easy to use (see https://mathr.co.uk/exrtact for a tiny example)
3. EXR supports higher bit depth colour images (float16, float32, uint32) - I think you(?) once asked for 16bit PNG but EXR covers that base also (though KF only does float16 (half) so far)
4. other software might be more inclined to support EXR than KFB
5. EXR is part of industry standards for video/films/etc (eg the SMPTE standard ACES uses a subset of EXR in an MXF container)

With DE, can we get an angle, not just a distance?

Should be possible, though KF doesn't yet calculate it when using derivatives; and KF doesn't save a DE channel when using image-space differences (it can be calculated from the N and NF channels).

Perhaps a channel DET with angle in radians -PI .. +PI (for floatTypes) or (0x00000000 .. 0xFFFFFFFF divided by  0x100000000 as turns for UINT)  but I think it might be better to have Cartesian DEX, DEY channels (as floatTypes) to avoid any possible radians vs degrees confusion. Cartesian coordinates are more useful for many algorithms, less trig to do.  Then it just remains to fix the direction of the DE, I guess it should point towards the fractal from the pixel?  So for quadratic Mandelbrot, DEX,DEY at -3+0i would be +1.0, 0.0.  Thoughts?

Okay. Yes, I suggested the high precision PNG.
If I understand it's a format that allows to save image color data in addition to various other information for each pixel like iterationcount and whether it was guessed.

Yes it's a high-bit-depth RGB image (half-float I think is sign 1bit exponent 5bits mantissa 10bits with implicit leading 1 bit) with extra channels.

You can tell glitches (and other not calculated pixels) as they will be 0+0.0 in the N+NF channels (I see you used the glitch status for Trees Revisited Revisited).

Guessing and mirroring is not yet exposed (it is not stored by KF even internally), it would have to be another channel.

What about  different file for each data type ?
Like pfm for floating point data:
https://en.wikibooks.org/wiki/Fractals/Computer_graphic_techniques/2D/gfile#pfm

Also:

https://en.wikipedia.org/wiki/Hierarchical_Data_Format

I think it might be better to have Cartesian DEX, DEY channels (as floatTypes) to avoid any possible radians vs degrees confusion. Cartesian coordinates are more useful for many algorithms, less trig to do.  Then it just remains to fix the direction of the DE, I guess it should point towards the fractal from the pixel?  So for quadratic Mandelbrot, DEX,DEY at -3+0i would be +1.0, 0.0.  Thoughts?

Great, Cartesian is fine, perhaps better.  As long as we can get an angle for "slopes" type shading (analytic would be much cleaner than the current pixel based solution).