Bug 138795 - CALC FILL Precision error - drag to decrement produces surplus decimals
Summary: CALC FILL Precision error - drag to decrement produces surplus decimals
Status: RESOLVED DUPLICATE of bug 129606
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
7.0.3.1 release
Hardware: All Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-12-10 11:21 UTC by Colin
Modified: 2020-12-10 12:13 UTC (History)
0 users

See Also:
Crash report or crash signature:


Attachments
Simple CALC .ods file (15.54 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-12-10 11:22 UTC, Colin
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Colin 2020-12-10 11:21:08 UTC
Description:
Simple sheet enclosed.
Some of the data elements in the cluster V1 -V12 are decreasing by fixed amounts.
If a number of the elements are selected and then dragged to decrement NOT increment then the generated numbers have surplus decimal places.
It only seems to be affecting "auto-fill" rows NOT columns.
This will produce substantial errors when dealing with large scale multi billion transaction values

Steps to Reproduce:
From the attached spreadsheet;
Select H9:J9
Drag right to N9 in an endeavour to decrement 
Try the same with selection F14:J14
They're all highlighted to avoid confusion ;)
Note that the greater the number of cells selected for the FILL operation, the greater the number of erroneous results.
NOTE: All cells are currently defined as no decimals, this does not change even when auto-filled to 13 decimals


Actual Results:
Faulty precision with 13 decimal places zero-filled to the final digit 1

Expected Results:
1 decimal place if the source defines it - probably more if the source data is so defined


Reproducible: Always


User Profile Reset: No


OpenGL enabled: Yes

Additional Info:
Version: 7.0.3.1 (x64)
Build ID: d7547858d014d4cf69878db179d326fc3483e082
CPU threads: 4; OS: Windows 10.0 Build 19042; UI render: Skia/Raster; VCL: win
Locale: sv-SE (en_GB); UI: en-GB
Calc: threaded
Comment 1 Colin 2020-12-10 11:22:14 UTC
Created attachment 168032 [details]
Simple CALC .ods  file
Comment 2 Colin 2020-12-10 11:35:32 UTC
Correction:
The surplus digits are ignored by subsequent math functions but still produce the surplus zeros + 1 in the rendition of the math result.
I only noticed it because the ready sized "header" cells displayed the overflow indicator and resizing the columns to "optimum" identified the effect.
I left the severity as normal but obviously if developers consider it trivial, it should be amended.
Comment 3 Mike Kaganski 2020-12-10 12:13:40 UTC
This fixed in 7.1.
But: see also https://wiki.documentfoundation.org/Faq/Calc/Accuracy

*** This bug has been marked as a duplicate of bug 129606 ***