Focus on the Transformation of the AI Industry: Graph Neural Networks Could Be the Next Turning Point

2020-01-17

Graph Neural Networks (GNNs) are one of the hottest research topics in the field of AI, with academia and industry investing significant resources into studying them. GNNs have tremendous potential in causal reasoning and are expected to address a series of problems that deep learning struggles with, such as relational reasoning and interpretability. These problems are considered crucial for substantial advancements in AI. In recent years, GNN technology has received great attention and response from the academic community. Major academic conferences have introduced workshops related to GNNs, and the number of papers with "Graph Network" as a keyword has surged.

A Brief Introduction to Graph Neural Networks

Before discussing GNNs, let's first understand what a graph is. In computer science, a graph is a data structure composed of vertices and edges. Depending on whether there is a directional dependency between vertices, edges can be directed or undirected. Graphs are used to model a set of objects (nodes) and their relationships (edges). In recent years, the analysis of graphs using machine learning methods has gained increasing attention due to the powerful expressive capabilities of graph structures. GNNs are a class of methods based on deep learning that handle information in the graph domain. Due to their good performance and interpretability, GNNs have become widely applied in graph analysis. The motivation behind GNNs originated from Convolutional Neural Networks (CNNs). The widespread application of CNNs brought breakthroughs to the field of machine learning and ushered in a new era of deep learning. However, CNNs can only operate on regular Euclidean data, such as images (2D grids) and text (1D sequences). The challenge was how to apply CNNs to graph structures, which exist in non-Euclidean spaces. This became a key problem that GNN models aimed to solve. Another motivation for GNNs comes from graph embedding, which learns low-dimensional vector representations of nodes, edges, or subgraphs in a graph. Methods like DeepWalk, LINE, and SDNE have achieved great success in the field of network representation learning. However, these methods are computationally complex and not optimal for large-scale graphs. GNNs aim to address these issues. Graph networks combine network science with deep learning. In recent years, numerous excellent works using graph network techniques to model complex systems have emerged, such as graph convolutional networks, graph attention networks, and the GraphSAGE algorithm applied to large-scale data. However, these representative works focus on positive problems in complex networks, such as node classification and graph classification, given the known network structure. In comparison, network reconstruction is more like an "inverse problem," inferring the internal network structure and learning the dynamic rules of the system through observations. In summary, GNNs have powerful computational capabilities and friendly handling of graph-structured data, making them highly favored by researchers. However, there are still many challenges that need to be addressed.

Turning Point of the Hotspot

After a period of incubation and exploration from 2017 to 2018, GNNs have experienced a rapid outbreak since the end of 2018. They have blossomed in both theoretical research and practical applications. In terms of theoretical research, the principles of GNNs, variant models, and their adaptation to various graph data have become mainstream topics. In terms of practical applications, GNNs have demonstrated unprecedented penetration, ranging from visual reasoning to open-ended reading comprehension, from drug molecule development to 5G chip design, and from traffic flow prediction to 3D point cloud data learning. The broad application prospects of GNNs are evident. As an emerging technology, the interpretation of GNN principles and the exploration of various variants and extensions constitute the hotspots of theoretical research. These papers provide good answers to the advantages, disadvantages, and adaptability transformation of GNNs. The first current hotspot in GNN research lies in the theoretical study of its related capabilities. In "How Powerful are Graph Neural Networks?" and "On the equivalence between graph isomorphism testing and function approximation with GNNs," the performance of GNNs on graph isomorphism problems is discussed. Graph isomorphism refers to determining whether two given graphs are identical. The research on variant models of GNNs is another hotspot in the field. These variants enhance the performance of GNNs in certain aspects. We know that the GCN model originates from the Fourier transform on graph signals. The "Graph Wavelet Neural Network" introduces wavelet transform on graph signals to transform the GCN model, enabling convolutional computations within local neighborhoods in the spatial domain. Transforming data representation from Euclidean space to hyperbolic space not only obtains better hierarchical representations but also significantly reduces the dimensionality. "Hyperbolic Graph Convolutional Neural Networks" and "Hyperbolic Attention Networks" extend GNNs to hyperbolic space. In "MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing" and "Diffusion Improves Graph Learning," the definition of neighbors in the original GCN is extended from first-order to higher-order, strengthening the low-pass filtering characteristics of the model, which traditional networks cannot achieve.

RECENT AI NEWS

RECENT AI TOOLS