Exploring New Depths in Forests: Applications in Multi-Label Learning

2019-11-20

"Deep Forest" algorithm proposed by Professor Zhou Zhihua's team at Nanjing University has attracted wide attention in the field of artificial intelligence academia and industry, and has been implemented in large-scale internet finance companies. This technology is a method based on non-differentiable units (i.e., trees/tree ensembles), which has the characteristics of being suitable for multi-core chip architecture acceleration, and can better handle discrete or list data than perception based on neural networks. It can achieve maximum optimization using core architecture devices such as Intel Xeon Scalable processors. Recently, Professor Zhou Zhihua's team at Nanjing University has introduced the deep forest algorithm into multi-label learning for the first time, proposing the Multi-Label Deep Forest method (MLDF). Non-neural network model - Deep Forest After the rise of the third wave of artificial intelligence, deep learning technology has become popular, and deep neural network models have become the focus of the industry and academia. However, current deep neural networks have many shortcomings, such as too many hyperparameters, requiring a large amount of training data, difficult theoretical analysis, black box models, and the complexity of the model is usually much greater than the "required" complexity of the task. "Deep Forest" is a major innovation in the field of artificial intelligence, which opens a new door for artificial intelligence algorithms. It is not intended to replace deep learning, because it is itself a form of deep learning and the first deep learning model that does not use the backpropagation algorithm for training. From the perspective of application value, deep neural networks are often not the best choice for many tasks other than image, video, and speech. Many times, they even perform poorly in problems such as symbolic modeling, mixed modeling, and discrete modeling. "Deep Forest" may have better performance in these tasks. Currently, the deep forest model has been applied by large enterprises and achieved good results. Therefore, it not only has academic value but also practical value. Deep Forest adopts a multi-level structure, with each layer consisting of four random forests, two random forests, and two extreme forests. Each extreme forest contains 1000 completely random trees, and each forest is trained on the data and outputs a result, which is called the class vector generated by the forest. To avoid overfitting, the data fed to each forest for training is cross-validated through k-fold cross-validation. Each layer finally generates four class vectors. The next layer trains on the four class vectors from the previous layer and the original data as the new train data, and this process is repeated. The last layer averages the class vectors to obtain the prediction result. By comparison, it is easy to see that this structure is very similar to a neural network. The unit of a neural network is a neuron, while the unit of a deep forest is a random forest. A single random forest is stronger in performance than a single neuron, which is the main reason why deep forests can achieve good results even with few layers and basic forest trees. However, fundamentally speaking, deep forests are completely different from neural networks, which gives them new possibilities and expands the system of deep learning. New exploration - Used for multi-label learning Recently, Professor Zhou Zhihua's team has expanded the application scope of deep forests and used the deep forest method for multi-label learning, which is a completely new and bold attempt. In multi-label learning, each instance is associated with multiple labels, and the key task is how to use label correlation in model construction. Deep neural network methods usually embed feature information and label information into a latent space together to utilize label correlation. However, the success of these methods largely depends on the accurate selection of the model's depth. Deep forests, on the other hand, do not rely on backpropagation. It is probably based on this different characteristic that Professor Zhou Zhihua's team believes that the advantages of the deep forest model are very suitable for solving multi-label problems. They have designed the Multi-Label Deep Forest method using two mechanisms: metric-aware feature reuse and metric-aware layer growth. In multi-label learning, each instance is associated with multiple labels, and the task of multi-label learning is to predict a set of relevant labels for unseen instances. Therefore, it has been widely used in various problems such as text classification, scene classification, functional genomics, video classification, and chemical classification. It is almost ubiquitous in real-world problems and has attracted increasing research attention. Deep forest is an ensemble deep model built on decision trees, and it does not use backpropagation during the training process. The cascaded structure of the deep forest ensemble system enables it to perform representation learning like deep neural models. It is easier to train because it has fewer hyperparameters. Due to the complexity of evaluation in multi-label learning compared to traditional classification tasks, there are multiple performance measurement methods. The new MLDF method can create building blocks of deep forests using different multi-label tree methods and utilize label correlation through layer-by-layer representation learning. It introduces deep forests into multi-label learning for the first time and solves two challenging problems: optimizing different performance metrics according to user requirements and reducing overfitting when utilizing feature correlations across multiple layers.

RECENT AI NEWS

RECENT AI TOOLS