Decision Trees

The decision tree algorithm is one of the best and most widely used algorithms in Machine Learning. It is a supervised learning algorithm. A decision tree uses a tree-like model to make predictions. It resembles an upside-down tree.

A decision tree builds classification or regression models in the form of a tree structure. It breaks down a data set into smaller subsets and the final result is a tree with decision nodes and leaf nodes.

A decision tree typically starts with a single node, which branches into possible outcomes. Each of those outcomes leads to additional nodes, which branch off into other possibilities. This gives it a tree-like shape.

The topmost decision node in a tree corresponds to the best predictor called the root node. Leaf node represents a classification or decision. Decision trees can handle both categorical and numerical data.

The process of partitioning the data set into subsets is known as splitting. Splits are formed on a particular variable. If a test splits the data into two partitions, then it is called a binary decision tree.  If it is more than two partitions, then it is known as a multi-way decision tree.

There are some selection criteria to choose a particular attribute for splitting the data. We should try to split the nodes such that the resulting nodes are as homogenous as possible. More homogeneity will mean that most of the data points in the set belong to the same class label.

If we want to divide the data set into multiple subsets because each subset has an independent trend corresponding to it then a linear regression model cannot be used to make predictions. In such situations, we can use a decision tree model to make predictions. Because a tree regression model has the capability of splitting the data into multiple sets and assigning a linear regression model to each set independently.

The difference between decision tree classification and decision tree regression is that in regression, each leaf represents a linear regression model, as opposed to a class label.

Advantages of Decision Trees

  • Decision trees are easy to interpret.
  • To build a decision tree requires little data preparation from the user- there is no need to normalize data

Disadvantages of Decision Trees

  • Decision trees are likely to overfit noisy data. The probability of overfitting on noise increases as a tree gets deeper.

Comments are closed.