• 欢迎访问速搜资源吧，如果在网站上找不到你需要的资源，可以在留言板上留言，管理员会尽量满足你！

【速搜问答】决策树学习是什么

10个月前 (04-10) 168次浏览

Chinese-English Translation:

Decision tree training in data mining and machine learning uses decision tree as prediction model to predict the class label of samples. This decision tree is also called classification tree or regression tree. In the structure of these trees, the leaf node gives the class label, while the inner node represents an attribute.

Decision tree training in statistics, data mining and machine learning uses decision tree as a prediction model to predict the class label of samples. This decision tree is also called classification tree or regression tree. In the structure of these trees, the leaf node gives the class label, while the inner node represents an attribute.

In decision analysis, a decision tree can express the decision process clearly. In data mining, a decision tree expresses data rather than decision.

extension

Decision tree training is a common method in data mining. The goal is to create a model to predict the target value of the sample.

A decision tree describing the survival of passengers on the Titanic

The training process of a tree is: according to an index, split the training set into several subsets. This process is repeated recursively in the generated subset, namely recursive segmentation. When the class labels of a training subset are the same, recursion stops. The top-down induction (tditd) of decision tree is one of greedy algorithms, and it is also the most commonly used training method so far.

The data are presented as follows:

Types of decision tree

There are two main types of decision tree in data mining

The output of the classification tree is the class label of the sample.

The output of the regression tree is a real number (such as the price of the house, the time of the patient in the hospital, etc.).

The term classification and regression tree (CART) includes the above two decision trees, which were first proposed by Breiman et al. There are some similarities and differences between classification trees and regression trees – for example, dealing with the problem of where to split.

Some integrated methods produce multiple trees

Bagging is an early integration method, which uses put back sampling method to train multiple decision trees, and the final result is generated by voting method.

Random forest uses multiple decision trees to improve classification performance.

Boosting tree can be used for regression analysis and classification decision.

Rotation forest – principal component analysis (PCA) is used to train each tree.

There are many other decision tree algorithms

ID3 算法

ID3 algorithm

C4.5 算法

C4.5 algorithm

CHi-squared Automatic Interaction Detector (CHAID)， 在生成树的过程中用多层分裂。

Chi squared automatic interaction detector (CHAID) uses multi-layer splitting in the process of spanning tree.

MARS 可以更好的处理数值型数据。

Mars can deal with numerical data better.

Model expression

When building a decision tree, we usually adopt a top-down approach, choosing the best attribute to split at each step The definition of “best” is to make the training set in the child nodes as pure as possible. Different algorithms use different indicators to define “best”. This section introduces some of the most common indicators.

Purity index of Gini

In cart algorithm, Gini impure represents the probability that a randomly selected sample is wrongly divided in a subset. Gini impure is the probability that the sample will be selected multiplied by the probability that it will be misclassified. When all samples in a node are one class, Gini impure is zero.

Suppose that the possible value of Y is {1, 2 , m}

information gain

ID3，C4.5 和 C5.0 决策树的生成使用信息增益。信息增益 是基于信息论中信息熵的理论。

ID3, C4.5 and C5.0 decision trees are generated using information gain. Information gain is based on the theory of information entropy in information theory.

Compared with other data mining algorithms, decision tree has many advantages

It is easy to understand and explain. People can easily understand the meaning of decision tree.

With little data preparation, other technologies often need data normalization.

That is to say, it can deal with both numerical data and category data. Other technologies tend to deal with only one data type. For example, association rules can only deal with categorical data, while neural networks can only deal with numerical data.

White box model is used. The output is easily explained by the structure of the model. Neural network is a black box model, which is difficult to explain the output results.

Test sets can be used to verify the performance of the model. The stability of the model can be considered.

Robust control. Robust to noise.

It can deal with large-scale data very well.

shortcoming

Training an optimal decision tree is a complete NP problem. Therefore, in practical application, heuristic search algorithm such as greedy algorithm is used to train the decision tree. This algorithm can not get the optimal decision tree.

The excessive complexity of decision tree will lead to the failure to predict the data outside the training set. This is called over fitting. Pruning mechanism can avoid this problem.

Some problems can not be solved by decision tree, such as XOR problem. When solving this problem, the decision tree will become too large. To solve this problem, we can only change the domain of the problem or use other more time-consuming learning algorithms (such as statistical relation learning or inductive logic programming)

For those data with categorical attributes, the information gain will be biased.

extend

Decision chart

In the decision tree, the path from the root node to the leaf node is converged or combined with. In decision graph, minimum message length (MML) can be used to join two or more paths.

Search by evolutionary algorithm

Evolutionary algorithm can be used to avoid the local optimal problem.

[361009623@qq.com]  • 版权声明

本站的文章和资源来自互联网或者站长
的原创，按照 CC BY -NC -SA 3.0 CN
协议发布和共享，转载或引用本站文章
应遵循相同协议。如果有侵犯版权的资
源请尽快联系站长，我们会在24h内删
除有争议的资源。
• 网站驱动

• 友情链接

• 支持主题 