Cloverleaf 6.1 expr command

[Terry Shelton]

Posting hoping someone can have a logical explanation of the following. This was discovered during CL 6.1 unit testing. tcl code in question is used in an xlate to convert centimeters to inches. Obvious question is “why the different outputs ?”

AIX 6.1, CL 6.1, tcl 8.5

tcl>expr 139.7 / 2.54

54.99999999999999

AIX 6.1, CL 5.8, tcl 8.4

tcl>expr 139.7 / 2.54

55.0

Topic

[Scott Folley]

According to the man page for TCL 8.5:

All internal computations involving integers are done calling on the LibTomMath multiple precision integer library as required so that all integer calculations are performed exactly. Note that in Tcl releases prior to 8.5, integer calculations were performed with one of the C types long int or Tcl_WideInt, causing implicit range truncation in those calculations where values overflowed the range of those types. Any code that relied on these implicit truncations will need to explicitly add int() or wide() function calls to expressions at the points where such truncation is required to take place.

All internal computations involving floating-point are done with the C type double. When converting a string to floating-point, exponent overflow is detected and results in the double value of Inf or -Inf as appropriate. Floating-point overflow and underflow are detected to the degree supported by the hardware, which is generally pretty reliable.

[Max Drown (Infor)]

http://wiki.tcl.tk/1650

$tcl_precision, a built-in Tcl variable, specifies the number of digits to generate when converting floating-point values to strings.

Code:

tcl>echo [expr 139.7 / 2.54]
54.99999999999999
tcl>set tcl_precision 12
12
tcl>echo [expr 139.7 / 2.54]
55.0
tcl>

-- Max Drown (Infor)

[Jim Kosloskey]

Terry,

It might be because in Tcl 8.5 (which comes with Cloverleaf 6) the default precision is 0 where before I think it was 12 (and looks like in your case is 15).

So now the expr returns the least number of decimal places to provide differentiation.

To get more decimal places you might need to use the format command as the default precision is no longer able to be over-ridden.

Of course I could be way off base here too  🙂

email: jim.kosloskey@jim-kosloskey.com 29+ years Cloverleaf, 59 years IT - old fart.

[Author]

Replies