Bug 103312 - Using format brush to copy conditional formats to some rows generates thousands (?) additional conditional formats making the spreadsheet unusable slow
Summary: Using format brush to copy conditional formats to some rows generates thousan...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.1 all versions
Hardware: All All
: high major
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: bibisectRequest, perf, regression
Depends on:
Blocks: Conditional-Formatting Clone-Formatting
  Show dependency treegraph
 
Reported: 2016-10-18 12:46 UTC by OfficeUser
Modified: 2018-04-09 20:19 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Slow.ods (710.99 KB, application/vnd.oasis.opendocument.spreadsheet)
2016-10-18 12:46 UTC, OfficeUser
Details
Slow2.ods (12.54 KB, application/vnd.oasis.opendocument.spreadsheet)
2016-10-20 16:10 UTC, OfficeUser
Details

Note You need to log in before you can comment on or make changes to this bug.
Description OfficeUser 2016-10-18 12:46:49 UTC
Created attachment 128058 [details]
Slow.ods

Hi,

- please open the attached spreadsheet "Slow.ods"
- make any change to any cell

Note that Calc does not respond several seconds.

- save the document

Note that Calc does not respond several seconds.

Note the high (compressed) file size of >700 kn for this small spreadsheet.

Also note that it is easy to make the spreadsheet that slow that users think the application has crashed by just copy&paste some of the rows using the format brush to copy the row formattings to addional rows.

From the user experience considerations I suggest to handle this issue with CRASH severity.

Not sure if related with Bug 49764.


Found with
Version: 5.2.2.2
Build-ID: 1:5.2.2-0ubuntu1~trusty0
CPU-Threads: 8; BS-Version: Linux 4.4; UI-Render: Standard; 
Gebietsschema: de-DE (de_DE.UTF-8); Calc: group
Comment 1 OfficeUser 2016-10-18 12:50:13 UTC
@Markus: I hope it is ok that I have added you to the CC list of this one. I found that you already have investigated similar issues.
Comment 2 OfficeUser 2016-10-18 12:51:29 UTC
Additional info:

You will already see the extreme bad performance when opening the attached odt-file.
Comment 4 OfficeUser 2016-10-18 13:07:39 UTC
Additional info:

The spreadsheet has a *EXTREMELY* big list of conditional formattings.

A2:K2
A3
A4
A5
A6
.
.
.
ZZ29

The conditional formattings have been first generated for and applied to A2:K2 (the first list entry) and the copied to the additional rows using the format brush.

Maybe this is the reason for these bad performance that the format brush does not generate optimized conditional formattings like...

A2:K2
A3:K3
A4:K4
.
.
.
Comment 5 OfficeUser 2016-10-20 16:08:37 UTC
Same bug happens if I copy and paste a hole single row:

- open the attached spreadsheet "Slow2.ods"

- click on Format/Conditional Formatting/Manage... and note there is a single range item A2:K2.

- Close the Manage Conditional Formatting window

- select row 2

- press ctrl+c

- paste into row 3

- click on Format/Conditional Formatting/Manage... and note there now hundrets of range items for just one additional rows

When pasting some more rows, the document becomes unusable slow!
Comment 6 OfficeUser 2016-10-20 16:10:48 UTC
Created attachment 128099 [details]
Slow2.ods
Comment 7 m_a_riosv 2017-02-05 22:33:41 UTC
There is an enhancement rquest for this matter.

https://bugs.documentfoundation.org/show_bug.cgi?id=87274
Comment 8 Heiko Tietze 2017-02-07 10:04:15 UTC
I can confirm that opening attachment 1 takes minutes on a fairly fast computer. Any operation in the document such as scrolling is almost impossible.

Version: 5.3.0.3
Build ID: 5.3.0-1
CPU Threads: 8; OS Version: Linux 4.9; UI Render: default; VCL: kde4; Layout Engine: new; 
Locale: de-DE (en_US.UTF-8); Calc: group
Comment 9 Xisco Faulí 2017-02-07 10:31:39 UTC
Reproduced in

Version: 5.4.0.0.alpha0+
Build ID: fc53cce64400430cdc21f79c959d75fb9a26d13d
CPU Threads: 4; OS Version: Linux 4.8; UI Render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); Calc: group

and 

Version 4.1.0.0.alpha0+ (Build ID: efca6f15609322f62a35619619a6d5fe5c9bd5a)

but not in

LibreOffice 3.3.0 
OOO330m19 (Build:6)
tag libreoffice-3.3.0.4
Comment 10 OfficeUser 2017-02-09 08:28:04 UTC
@Xisco Fauli

Thanks. But which behavior can you confirm with that build?

We have to problems here:

a) Format brush and copy paste are bloating the spreadsheets with "thousands" of conditional formattings.

b) An already "bloated" spreadsheet (like the one attached) cannot be used anymore because of EXTREME bad performance.
Comment 11 Xisco Faulí 2017-02-09 10:09:10 UTC
I can confirm back to 4.1.0.0.alpha0+ that calc hangs at import time and it's impossible to work with the attached spreadsheet.
Please report each problem in different bugs.
Comment 12 OfficeUser 2017-02-09 10:30:39 UTC
I am not sure if it a good idea to separate both problems. I don't even know if b) can be fixed or lets say optimized. So fixing a) (the root cause) may be the only solution.

Do you know how Excel performs on b) (opening bloated documents)?
Comment 13 Heiko Tietze 2017-02-09 10:37:15 UTC
(In reply to OfficeUser from comment #12)
> I am not sure if it a good idea to separate both problems.

Let's wait for someone who fixes the problem that opening your examples almost crashes the app. I was trying during the latest hackfest, sitting next to Xisco, and it took some minutes (i7-4712 CPU) while Xiscos PC (i5?) refused to do that. Once the doc is open any operation takes seconds to minutes, for example scrolling. We made just sure that you don't have a large number of data or styles to exclude this reason. So please be patient (and try another way of conditional formatting meanwhile).
Comment 14 m_a_riosv 2017-02-09 11:21:34 UTC
This is one of that matters, really annoying, making an amazing tool like is Conditional Format after a great development nearly to unusable on many cases.

In my order of importance:

1- The break of the CF cell ranges with copy/paste/move/brush, with no tool to reunify.
2- It's slow enter data on cells that are part of a CF, curious it doesn't happen pasting, specially visible on hard spreadsheet, needs autocalculate active.
3- No option to move the conditions down/up
4- No option to copy/duplicate a condition
5- Difficult duplicate a CF, it needs to be done with a workaround.

Perhaps a good candidate for GSoc.
Comment 15 OfficeUser 2017-02-09 11:36:12 UTC
I just have exported an "Slow.xls" and opened it in Excel.

Result: Excel also hangs while opening these bloated files. I had to kill Excel. So in my opinion we have to fix a). But a) is already reported as Bug 87274.

Since this bug report (Bug 103312) has a higher importance and a focus on the impacts of this bug I suggest to set Bug 87274 as duplicate of this bug report (Bug 103312).

Details on how to implement a solution for a) may be copied from Bug 87274 (by m.a.riosv).
Comment 16 Kevin Suo 2017-11-30 06:01:17 UTC
I just see a commit on master:
https://cgit.freedesktop.org/libreoffice/core/commit/?id=ea55492a6e55290d92a59324b3cb31ed958981ab

This commit may be a fix for this issue, but I am not sure. Maybe someone with a self-built master can update the build and test.
Comment 17 Mike Kaganski 2017-11-30 06:13:29 UTC
(In reply to Kevin Suo from comment #16)

That is not a fix for this, rather it's related to bug 87274.
Comment 18 Xisco Faulí 2017-11-30 06:30:49 UTC
(In reply to Kevin Suo from comment #16)
> I just see a commit on master:
> https://cgit.freedesktop.org/libreoffice/core/commit/
> ?id=ea55492a6e55290d92a59324b3cb31ed958981ab
> 
> This commit may be a fix for this issue, but I am not sure. Maybe someone
> with a self-built master can update the build and test.

For the next time, you can also test it in the 'updater' builds from http://dev-builds.libreoffice.org/daily/master/, which are normally updated daily ;-)
Comment 19 Xavier Van Wijmeersch 2018-04-02 18:09:37 UTC
Can not reproduce with

Version: 6.1.0.0.alpha0+
Build ID: 3939e10c515364cb9ef0a089d3db667aed0aa5ea
CPU threads: 8; OS: Linux 4.14; UI render: default; VCL: kde4; 
Locale: nl-BE (en_US.UTF-8); Calc: group

but can reproduce with

Version: 6.0.4.0.0+
Build ID: 4cb868abaf789afe3f2b4bc65454c21c08666a78
CPU threads: 8; OS: Linux 4.14; UI render: default; VCL: kde4; 
Locale: nl-BE (en_US.UTF-8); Calc: group
Comment 20 OfficeUser 2018-04-09 20:19:45 UTC
Can confirm that format brush and copy-and-paste now work as expected with:

Version: 6.1.0.0.alpha0+
Build ID: f3e8d0fb317090aade384784a1e16a8c8afe3437
CPU threads: 8; OS: Linux 4.4; UI render: default; VCL: gtk2; 
TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2018-03-22_08:34:58
Locale: de-DE (de_DE.UTF-8); Calc: group


I expect that it has been fixed by the patch of Comment 16.