1

Assume that we have two separate tree regressions. I'm interested in understanding whether the product of tree regressions can be represented by a single tree. Would this be possible?

TFT
  • 35
  • 2
  • By "product of tree regressions" do you just mean the pointwise product of the resulting prediction functions? The two regressions have the same features but different targets, or...? – Ben Reiniger Dec 15 '22 at 14:39
  • @BenReiniger Yes, in the case the two tree regressions have the same feature but different targets. – TFT Dec 15 '22 at 16:56

1 Answers1

2

Yes, it is possible to find a tree that represents the product. An easy way to do so is to extend the first tree in each leaf with the second tree.

Example

Assume there are theses two tree:
Tree A with 4 leafs Tree B with 3 leafs

A regression product tree can then be given by:
enter image description here

Notes

Is there a smaller product tree?

Not necessarily (but in some cases there might be). Look at the example (which I chose to make this point): Each leaf has a different output, so a tree with less leaf nodes would not work.

Is it feasable?

In most cases, a representation by two trees is more compact and better suited to many tasks. When it comes to training regression trees - as long as the targets of the two trees are given, training them independently is faster and requires less training data.
Note that the depth of regression product tree is the sum of the depth of both trees and the number of leafs is the product of the number of leafs of single trees.

General Case

Your case is just a special case of a more general property of regression trees: Assume you have $K$ regression trees that take the same features as input. Let's say $f_k(x),\,(k=1,\ldots, K)$.

Given a function $y=g(f_1(x),\ldots,f_K(x))$ that combines these trees (the product in your case), this function $g$ can be represented by a single regression tree.

The construction is done similar to above example.

Broele
  • 1,107
  • 7
  • 13