7

I am having a little trouble understanding the difference between what a "Node" of a tree and a "Leaf" of a tree.

Suppose I am trying to decide the size of coffee a person may like. There are three categories: small, medium, and large based off the peoples age, height, weight, income.

So I have four predictors and 3 possible outcomes. When looking at many gradient boosting algorithms, there are parameters that can increase the number of leaves.

My understanding of this (correct me if wrong), but I will illustrate with a picture. Assuming each yes/no split is 50/50

enter image description here

Does increasing the number of leaves to lets say 3 leaves change it from yes/no aka 50/50 to 33/33/33? This is a little confusing to me. Thank you for any clarification.

Sam

RandomSam
  • 71
  • 1
  • 1
  • 2

2 Answers2

6

Leaf nodes are the nodes of the tree that have no additional nodes coming off them. They don't split the data any further; they simply give a classification for examples that end up in that node. In your example tree diagram, the nodes that say 'Large', 'Medium' or 'Small' are leaf nodes. The other nodes in the tree are interchangeably called split nodes, decision nodes or internal nodes.

In gradient boosting algorithms, a number of decision trees are grown. Each tree is grown until some stopping criteria is met. One kind of stopping criteria is the maximum number of leaves in the tree. At each stage of growing a decision tree, a leaf node is turned into a split node by creating a yes/no question (we call this a binary split), and two new leaf nodes are created which correspond to each side of the split. Once the total number of leaf nodes in the tree reaches the limiting value, the tree building algorithm stops and begins building the next tree.

timleathart
  • 3,900
  • 20
  • 35
0

Leaf nodes are the final nodes of the decision tree after which, decision tree algorithm wont split the data.

If pre-pruning technique is not applied then by default decision tree splits the data till it does not get homogeneous group of data i.e. each leaf represents data splits that belongs to same label (0/1, yes/no).

So by default till the time all data points in the node represents or belongs to same class, tree gets split. The final nodes where all data points are of same label is considered as leaf node and all other intermediate nodes are considered as tree node.

Tree nodes can further be divided into sub nodes that leads to formation of leaf nodes.

CodeMaster GoGo
  • 768
  • 1
  • 6
  • 15