Bug 35712 - add polynomial regression type trend line
Summary: add polynomial regression type trend line
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Chart (show other bugs)
Version:
(earliest affected)
3.5.0 Beta1
Hardware: Other All
: highest enhancement
Assignee: Tomaz Vajngerl
URL:
Whiteboard: VOTE target:4.2.0 target:4.3.0
Keywords:
Depends on:
Blocks: 40317
  Show dependency treegraph
 
Reported: 2011-03-27 00:27 UTC by Anonymous Helper
Modified: 2014-01-27 13:32 UTC (History)
9 users (show)

See Also:
Crash report or crash signature:


Attachments
ODF file format modification proposed to OASIS in December 2008 (24.53 KB, application/vnd.oasis.opendocument.text)
2011-08-23 09:46 UTC, Laurent Balland
Details
Proposition of dialog box for user interface (47.91 KB, application/vnd.oasis.opendocument.text)
2011-08-23 09:49 UTC, Laurent Balland
Details
Test case with big values (17.50 KB, application/vnd.ms-excel)
2013-07-04 21:13 UTC, Laurent Balland
Details
Results obtained in Excel 2003 with textXLS.xls test file (14.97 KB, image/png)
2013-07-04 21:14 UTC, Laurent Balland
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Anonymous Helper 2011-03-27 00:27:49 UTC
(this is a copy of this Ooo bug: http://openoffice.org/bugzilla/show_bug.cgi?id=20819
See also: http://wiki.services.openoffice.org/wiki/Chart2/TrendLines#Add_polynomial_regression_type)

There are already offered different types of regression lines (linear regression, exponential regression etc.), but a polynomial regression curve is still missing.

There is an extension: http://oooconv.free.fr/fitoo/fitoo_en.html
Comment 1 Laurent Balland 2011-08-23 09:41:32 UTC
Hello,

To get polynomial regression, you better use CorelPoly macro from Laurent Godard (same page as FitOOo) which is simpler than FitOOo. A correction of this macro is proposed on issue 20819
http://openoffice.org/bugzilla/show_bug.cgi?id=20819

Actually, you may use CorelPolyGUI extension, which a graphic user interface quite convivial to build polynomial regression curve.
http://extensions.services.openoffice.org/en/project/CorelPolyGUI

SLAMLib extension propose also some functions (POLYFIT, POLYVAL) to get a polynomial regression.
http://extensions.services.openoffice.org/en/project/SLAMLib

To solve this issue, file format must be modified to store information about regression: other regressions (linear, exponential,...) use only two terms in their equation. Polynomial equation require a variable number of terms. So do we have to wait that ODF specification is modified, or can LibO extend ODF specification to store this information?

A file format modification has been proposed to OASIS to include these parameter in December 2008, but it had no effect.

It is one of the most wanted feature in Chart. What could help to get this implemented?
Comment 2 Laurent Balland 2011-08-23 09:46:04 UTC
Created attachment 50500 [details]
ODF file format modification proposed to OASIS in December 2008

This file contains file format modification requested to get polynomial regression curve.
Comment 3 Laurent Balland 2011-08-23 09:49:49 UTC
Created attachment 50501 [details]
Proposition of dialog box for user interface

This file summarize different enhancement requested for trend line, and propose a user interface.
Comment 4 Laurent Balland 2011-10-04 01:57:09 UTC
Hello,

New version of CorelPoly (v 1.0!) available on
http://extensions-test.libreoffice.org/extension-center/improved-trend-lines/

Now all classic regression are included.
Comment 5 Björn Michaelsen 2011-12-23 11:46:54 UTC
[This is an automated message.]
This bug was filed before the changes to Bugzilla on 2011-10-16. Thus it
started right out as NEW without ever being explicitly confirmed. The bug is
changed to state NEEDINFO for this reason. To move this bug from NEEDINFO back
to NEW please check if the bug still persists with the 3.5.0 beta1 or beta2 prereleases.
Details on how to test the 3.5.0 beta1 can be found at:
http://wiki.documentfoundation.org/QA/BugHunting_Session_3.5.0.-1

more detail on this bulk operation: http://nabble.documentfoundation.org/RFC-Operation-Spamzilla-tp3607474p3607474.html
Comment 6 Laurent Balland 2011-12-24 01:43:14 UTC
Hello,

This enhancement has not been introduced in LibO 3.5.
Extension CorelPolyGUI (Improved trend lines) try to answer this question but is not integrated in LibO:
http://extensions.libreoffice.org/extension-center/improved-trend-lines
Comment 7 nonlinearx 2013-05-10 13:30:50 UTC
Polynomial regression is a very important feature I am needing for data analysis in my high school physics lessons. An easy to use implementation is needed, so the students can focus on the physics.
Comment 8 ign_christian 2013-06-18 02:49:45 UTC
Could you please retest using LO 3.6.6 or 4.0.3 regarding some updates on extension stated in comment 6 ?
Comment 9 Tomaz Vajngerl 2013-07-04 07:11:06 UTC
Hi
Comment 10 Tomaz Vajngerl 2013-07-04 07:22:42 UTC
Ups.. OK let me try again:

Hi,

I have pushed polynomial regression curve code to the master and is now available in the latest daily build [1]. Implemented features are:
* polynomial regression curve with degree parameter.
* moving average regression curve with period parameter.
* bound regression curve to min and max X values and add forward / backward extrapolation parameter
* force intercept for polynomial and linear regression curve
* support added to ooxml, old MS binary formats, odt
* allow more than one trend line per series

The functionality still needs polish (especially in the GUI) and some clean-up but in general it should work OK. Please test it and provide your opinion. 

Thanks!

Regards, Tomaž

[1]: http://dev-builds.libreoffice.org/daily/master/Win-x86@6/2013-07-04_00.19.19/
Comment 11 Laurent Balland 2013-07-04 21:11:52 UTC
Hello Tomaž

Thanks a lot for your great job :)

Here are some comments about Version: 4.2.0.0.alpha0+
Build ID: e703b429cf448deddc7682a33c3e6221889efe98

- Bug: when values are too big => wrong calculation. See attachment textXLS.XLS for a degree 3 polynomial equation, see ResultsXLS.png for curve and equation obtained in Excel 2003 
- Bug: status bar does not update equation when changing degree: only degree=2, whatever the degree
- Bug: it should not calculate equation if number of points <= degree (if 1 point, you cannot find a unique line going through this point)
- UI: when increasing/decreasing Degree, autoselect polynomial type
- Improvement: equation sould be formatted on several lines if equation is too long (with high degree)
Comment 12 Laurent Balland 2013-07-04 21:13:44 UTC
Created attachment 82048 [details]
Test case with big values

Compare results obtained in LibO with those obtained in Excel 2003 (ResultsXLS.png)
Comment 13 Laurent Balland 2013-07-04 21:14:46 UTC
Created attachment 82049 [details]
Results obtained in Excel 2003 with textXLS.xls test file
Comment 14 Laurent Balland 2013-07-04 21:22:49 UTC
Re,

For TestXLS.xls test case, CorrelPolyGUI obtains similar results as Excel 2003.
Comment 15 ign_christian 2013-07-05 02:38:39 UTC
@nonlinearx@freenet.de, would you mind do testing & giving some opinion regarding Tomaž's work?

Rechanging version: to 3.5.0 Beta1 -> oldest release reported based on comment 6
Comment 16 Stephan van den Akker 2013-07-11 10:00:58 UTC
(In reply to comment #11)

> - Improvement: equation should be formatted on several lines if equation is
> too long (with high degree)

This improvement is especially important because, after leaving the editing mode of the chart, the whole chart gets scaled to make the equation line fit the allotted space for the chart. The result is a very small ("squashed") chart. This may spook users as the cause of this scaling may not be immediately obvious.

Alternatively, the number formatting may be changed to include fewer digits.

More space may be saved by using real superscripted exponents "²" instead of "^2"
Comment 17 bugquestcontri 2013-07-26 00:48:54 UTC
Already thanks to everybody working on this important feature!
Comment 18 Laurent Balland 2013-11-08 14:08:16 UTC
Hi,

Version: 4.2.0.0.alpha0+
Build ID: 3d65dfa3c98e0f79c579cfac23d55092f7554244
TinderBox: Win-x86@47-TDF, Branch:master, Time: 2013-11-05_11:23:58

- Extrapolate Forwad/Backward does not accept scientific value
- Force intercept value does accept scientific value, but treat them as normal value: 1E3 => 13
Comment 19 Commit Notification 2013-11-16 11:28:11 UTC
Laurent Balland-Poirier committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=06d2d3290acb5dd3200481b85baf7c5718ea177c

fdo#35712 Correct update of equation in status bar



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 20 Commit Notification 2013-11-27 06:10:41 UTC
Laurent Balland-Poirier committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=eaa13b8b63f91ce432eea216a9c3afd54bac62a5

fdo#35712 fdo#40315 fdo#40314 Autoselect trendline type



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 21 Commit Notification 2013-11-30 22:33:59 UTC
Laurent Balland-Poirier committed a patch related to this issue.
It has been pushed to "libreoffice-4-2":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=78f227e1371ef4d11bc98e2116a3b95bb881b020&h=libreoffice-4-2

fdo#35712 fdo#40315 fdo#40314 Autoselect trendline type


It will be available in LibreOffice 4.2.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 22 Tomaz Vajngerl 2014-01-27 13:32:55 UTC
I think this can be set as resolved now.