gawk: Setting precision
1
1 15.4.4 Setting the Precision
1 ----------------------------
1
1 'gawk' uses a global working precision; it does not keep track of the
1 precision or accuracy of individual numbers. Performing an arithmetic
1 operation or calling a built-in function rounds the result to the
1 current working precision. The default working precision is 53 bits,
1 which you can modify using the predefined variable 'PREC'. You can also
1 set the value to one of the predefined case-insensitive strings shown in
1 ⇒Table 15.4 table-predefined-precision-strings, to emulate an IEEE
1 754 binary format.
1
1 'PREC' IEEE 754 binary format
1 ---------------------------------------------------
1 '"half"' 16-bit half-precision
1 '"single"' Basic 32-bit single precision
1 '"double"' Basic 64-bit double precision
1 '"quad"' Basic 128-bit quadruple precision
1 '"oct"' 256-bit octuple precision
1
1 Table 15.4: Predefined precision strings for 'PREC'
1
1 The following example illustrates the effects of changing precision
1 on arithmetic operations:
1
1 $ gawk -M -v PREC=100 'BEGIN { x = 1.0e-400; print x + 0
1 > PREC = "double"; print x + 0 }'
1 -| 1e-400
1 -| 0
1
1 CAUTION: Be wary of floating-point constants! When reading a
1 floating-point constant from program source code, 'gawk' uses the
1 default precision (that of a C 'double'), unless overridden by an
1 assignment to the special variable 'PREC' on the command line, to
1 store it internally as an MPFR number. Changing the precision
1 using 'PREC' in the program text does _not_ change the precision of
1 a constant.
1
1 If you need to represent a floating-point constant at a higher
1 precision than the default and cannot use a command-line assignment
1 to 'PREC', you should either specify the constant as a string, or
1 as a rational number, whenever possible. The following example
1 illustrates the differences among various ways to print a
1 floating-point constant:
1
1 $ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", 0.1) }'
1 -| 0.1000000000000000055511151
1 $ gawk -M -v PREC=113 'BEGIN { printf("%0.25f\n", 0.1) }'
1 -| 0.1000000000000000000000000
1 $ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", "0.1") }'
1 -| 0.1000000000000000000000000
1 $ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", 1/10) }'
1 -| 0.1000000000000000000000000
1