(Solved) precision loss when passing a Double from script to uniforms

  • 11 Replies
  • 344 Views

0 Members and 1 Guest are viewing this topic.

Offline claude

  • *
  • 3f
  • ******
  • Posts: 1255
    • mathr.co.uk
« on: January 02, 2019, 09:57:22 PM »
in my script I have

Code: [Select]
app.setParameter("C", -1.942702413856584931e+00, 0.000000000000000000e+00);
but the location displayed is wrong (a few minibrot-lengths out).

Instrumenting the code shows me the following value is passed to OpenGL:

Code: [Select]
glUniform2d(12, -1.942700028419499958e+00, 0.000000000000000000e+00)
This value is different to the value I set in my script :(

Linkback: https://fractalforums.org/fragmentarium/17/precision-loss-when-passing-a-double-from-script-to-uniforms/2508/
« Last Edit: January 18, 2019, 02:23:18 AM by claude, Reason: solved »

Offline 3DickUlus

  • *
  • 3f
  • ******
  • Posts: 1376
    • Digilantism
« Reply #1 on: January 03, 2019, 02:23:22 AM »
I am using doubles for the conversion and find this from the Qt 4.8 documentation...
http://doc.qt.io/archives/qt-4.8/qvector2d.html
Quote
Note: By design values in the QVector2D instance are stored as float. This means that on platforms where the qreal arguments to QVector2D functions are represented by double values, it is possible to lose precision.
in these docs ( 4.8 ) the parameters to QVector2D are listed as qreal type, but in the Qt 5.9.4 docs it is listed as float type and no note about precision.

the QOpenGLShaderProgram Class does not pass double type to uniforms only float, I got around that by using GL calls to directly set the double type uniforms, but this is restricted to 7 and 14 decimal places respectively by the sliders.

the exact place where the conversion is checked is at line 62 in SyntopiaCore/Misc/MiniParser.cpp getDouble() function, if the conversion from QString to doublenum fails for any reason it should emit a warning line to the log window.

the exact place the value is set is lines 209 -219 in Fragmentarium/GUI/VariableWidget.cpp Float2Widget::fromString(QString string) and Float2Widget::setValue(QVector3D v) (you can set a 3D with a 2D it just uses xy and ignores z in this case, 3D is used to avoid compiler errors)....

...while writing this and looking at the code I have a fix (I think)...

existing code in VariableWidget.cpp Float2Widget class...
Code: [Select]
void Float2Widget::fromString(QString string) {
    double f1,f2;
    MiniParser(string).getDouble(f1).getDouble(f2);
    setValue(QVector2D(f1,f2));
}
...change to...
Code: [Select]
void Float2Widget::fromString(QString string) {
    double f1,f2;
    MiniParser(string).getDouble(f1).getDouble(f2);
    comboSlider1->setValue(f1);
    comboSlider2->setValue(f2);
}
...so the fromString function doesn't use QVector2D, this assumes the widget is a dvec2, the spinbox attached to the slider is a double type spinbox and the slider->setValue function is defined as void setValue( double d ).

if the above works I will make the necessary changes to all of the FloatnWidget definitions. if it doesn't then will have to look at the entire path from script call to shader uniform and figure out where the bits go missing... that's all I've got for now  :-\

oh, another thing lines 28 - 30 in VariableWidget.h define the number of decimal places to show in the GUI for float and double, 7 and 14 not enough ??? 9 and 18 better ???
Fragmentarium is not a toy, it is a very versatile tool that can be used to make toys ;)

https://en.wikibooks.org/wiki/Fractals/fragmentarium

Offline claude

  • *
  • 3f
  • ******
  • Posts: 1255
    • mathr.co.uk
« Reply #2 on: January 03, 2019, 02:38:55 PM »
I think I found the issue: QString::arg(double, ...) is only displaying 6 or 7 digits by default.  This seems to be a fix, but it's quite verbose:
Code: [Select]
index ef16e4b..0840cf4 100644
--- a/Fragmentarium-Source/Fragmentarium/GUI/MainWindow.h
+++ b/Fragmentarium-Source/Fragmentarium/GUI/MainWindow.h
@@ -239,7 +239,7 @@ using namespace Imath;
         variableEditor->setSettings( QString("%1 = %2\n").arg(setting).arg(v) );
       };
       void setParameter(QString setting, double x, double y) {
-        variableEditor->setSettings( QString("%1 = %2,%3\n").arg(setting).arg(x).arg(y) );
+        variableEditor->setSettings( QString("%1 = %2,%3\n").arg(setting).arg(x, 0, 'g', DDEC).arg(y, 0, 'g', DDEC) );
       };
       void setParameter(QString setting, double x, double y, double z) {
         variableEditor->setSettings( QString("%1 = %2,%3,%4\n").arg(setting).arg(x).arg(y).arg(z) );

This command reveals 237 lines that need auditing for precision loss:
Code: [Select]
git grep \\.arg\(

Offline 3DickUlus

  • *
  • 3f
  • ******
  • Posts: 1376
    • Digilantism
« Reply #3 on: January 04, 2019, 01:08:41 AM »
oh, another thing lines 28 - 30 in VariableWidget.h define the number of decimal places to show in the GUI for float and double, 7 and 14 not enough ??? 9 and 18 better ???

file Fragmentarium/GUI/VariableWidget.h
Code: [Select]
// decimal places for float and double
#define FDEC 9
#define DDEC 18

Offline 3DickUlus

  • *
  • 3f
  • ******
  • Posts: 1376
    • Digilantism
« Reply #4 on: January 04, 2019, 01:32:29 AM »
the line...
Code: [Select]
git grep \\.arg\(...shows all occurrences of QString::arg() function calls, which may or may not have anything to do with converting numbers to strings.

the important ones look like this...
Code: [Select]
QString Float2Widget::toString() {
    int p = FDEC;
    if(isDouble()) p = DDEC;
    return QString("%1,%2")
           .arg(QString::number(comboSlider1->getValue(),'g',p))
           .arg(QString::number(comboSlider2->getValue(),'g',p));
}
..and use FDEC and DDEC as the precision option for the conversion.

going the other way...
Code: [Select]
void Float2Widget::fromString(QString string) {
    double f1,f2;
    MiniParser(string).getDouble(f1).getDouble(f2);
    comboSlider1->setValue(f1);
    comboSlider2->setValue(f2);
}
... the input values all the way to and including spinner->setValue() are double type, so, as mentioned, FDEC and DDEC need to be bumped up and that alone should fix the issue, if it doesn't? then follow the path from script value to widget value and back again to see exactly where/why it's loosing precision.

edit: ALL of the scripting and double type stuff was added by me, the original Fragmentarium program doesn't support double types or scripting, these "features" are entirely experimental, I'm not a mathematician so any corrections would be most welcomed  :embarrass:

Offline 3DickUlus

  • *
  • 3f
  • ******
  • Posts: 1376
    • Digilantism
« Reply #5 on: January 04, 2019, 03:28:29 AM »
ok, taking some time to re-read and digest, yes, some things need FDEC/DDEC thanks for this claude  :thumbs: nobody else seemed to notice or perhaps it was brought to my attention earlier but got lost in the shuffle on the todo list, either way I will try to get it straightened out...

making changes to setParameter() definitions as you pointed out... does this fix the issue or is precision still lost ???

(some of my rambling is just me figuring it out ;) )

Code: [Select]
      void setParameter(QString setting, double v) {
        variableEditor->setSettings( QString("%1 = %2\n").arg(setting).arg(v, 0, 'g', DDEC) );
      };
      void setParameter(QString setting, double x, double y) {
        variableEditor->setSettings( QString("%1 = %2,%3\n").arg(setting).arg(x, 0, 'g', DDEC).arg(y, 0, 'g', DDEC) );
      };
      void setParameter(QString setting, double x, double y, double z) {
        variableEditor->setSettings( QString("%1 = %2,%3,%4\n").arg(setting).arg(x, 0, 'g', DDEC).arg(y, 0, 'g', DDEC).arg(z, 0, 'g', DDEC) );
      };
      void setParameter(QString setting, double x, double y, double z, double w) {
        variableEditor->setSettings( QString("%1 = %2,%3,%4,%5\n").arg(setting).arg(x, 0, 'g', DDEC).arg(y, 0, 'g', DDEC).arg(z, 0, 'g', DDEC).arg(w, 0, 'g', DDEC) );
      };

this does not take into account if the widget we are setting is float type or double type, reducing from double to float is ok? truncated or rounded?
« Last Edit: January 04, 2019, 03:38:48 AM by 3DickUlus »

Offline 3DickUlus

  • *
  • 3f
  • ******
  • Posts: 1376
    • Digilantism
« Reply #6 on: January 04, 2019, 04:20:37 AM »
this brings up another spot that needs some attention...

Code: [Select]
void FloatWidget::setUserUniform(QOpenGLShaderProgram* shaderProgram) {
    int l = uniformLocation(shaderProgram);
    if (l != -1) {
        shaderProgram->setUniformValue(l, (float)(comboSlider1->getValue()));
    }
}

Qt's shaderProgram->setUniformValue() only accepts float, as mentioned, I get around that by using gl calls (if vers > 4.0)

in the above code, if the value is set manually by twiddling the slider or spinner it may be falling short on precision when setUserUniform() gets called ??? individually on a widget,  however, the gl calls are used in DisplayWidget::setShaderUniforms(shaderProgram);  to set all uniforms enmass before actually drawing with the shader (in glcontext provided by DisplayWidget object) so I think that a test for isDouble() should be used to disable the Qt float only setuniform call ???

testing...

...seems to work in Experimental/DoubleTest.frag  :-\  so will fixup all widgets for this behavior and push to git this weekend

Thank you so much for the input claude !!! (and your patience  ::) )


hmmm this brings up another spot that may need some attention... ColorChooser ColorWidget and FloatColorWidget need to be "double aware" too currently converting to QColor(int,int,int,int) from old Qt code but now in 5.9+ we have more options like QColor::setRgbF(qreal r, qreal g, qreal b, qreal a = 1.0) for float type and QColor::setRgba64(QRgba64 rgba) 4x16bit int type etc...

edit: looks like QVector2/3/4D may have to be scrapped entirely (seemed like a good idea at the time before double types) in order to get full double type compatibility  :-\
« Last Edit: January 04, 2019, 06:22:17 AM by 3DickUlus »

Offline claude

  • *
  • 3f
  • ******
  • Posts: 1255
    • mathr.co.uk
« Reply #7 on: January 04, 2019, 03:12:51 PM »
making changes to setParameter() definitions as you pointed out... does this fix the issue or is precision still lost ???

it fixes it, working in my local copy (witness attached)

edit animgif no longer animated when attaching??

Offline claude

  • *
  • 3f
  • ******
  • Posts: 1255
    • mathr.co.uk
« Reply #8 on: January 04, 2019, 04:59:52 PM »
Qt's shaderProgram->setUniformValue() only accepts float, as mentioned, I get around that by using gl calls (if vers > 4.0)
can you subclass/inherit from it to add the double versions?  don't know much C++/Qt to know how

Quote
I think that a test for isDouble() should be used to disable the Qt float only setuniform call ???
if you do glUniform1f followed by glUniform1d it should work out fine, the first call will lose precision on the way to the GPU but the second restores it...

Offline 3DickUlus

  • *
  • 3f
  • ******
  • Posts: 1376
    • Digilantism
« Reply #9 on: January 04, 2019, 05:54:21 PM »
Will look at subclass/inherit.. that should work.

Would like to avoid setting them twice  :-\

Offline 3DickUlus

  • *
  • 3f
  • ******
  • Posts: 1376
    • Digilantism
« Reply #10 on: January 17, 2019, 02:20:06 AM »
precision loss fixed?

Offline claude

  • *
  • 3f
  • ******
  • Posts: 1255
    • mathr.co.uk
« Reply #11 on: January 18, 2019, 02:23:06 AM »
yes fixed! thanks!


xx
"Time Span"

Started by cricke49 on Fractal Image Gallery

0 Replies
338 Views
Last post August 02, 2018, 07:05:21 AM
by cricke49
question
[Solved] Color interpolation

Started by galac on Programming

5 Replies
293 Views
Last post March 02, 2019, 09:03:11 AM
by mclarekin
xx
can somebody convert this to a color smoothing/whatever script

Started by DJ Saint-Hubert on Programming

1 Replies
265 Views
Last post February 09, 2018, 06:37:24 AM
by WAUthethird
xx
OpenCL double precision in release 2.2.13

Started by piotrv on Mandelbulber

11 Replies
575 Views
Last post April 16, 2018, 02:58:42 PM
by mclarekin
xx
Birdie Style

Started by gannjondal on Fractal Image Gallery

1 Replies
407 Views
Last post May 08, 2018, 02:39:37 PM
by who8mypnuts