AI Helps You Pick Reading Materials Online

2019-12-24

With the development of the Internet, billions of books, news reports, and articles are available online, making it the best era for reading enthusiasts. Justin Solomon, an assistant professor at MIT, said, "There is a large amount of text on the Internet, but not all of it is useful. You need to segment and systematically learn from them, in short, select." With the help of the MIT IBM Watson AI Lab and the Geometric Data Processing Group at MIT, Solomon recently proposed a new technique for segmenting large amounts of text at the Neural Information Processing Systems conference. This method combines three popular text analysis tools - topic modeling, word embeddings, and optimal transport - to provide better and faster results than competing methods on popular document classification benchmarks. If an algorithm knows what you have liked in the past, it can scan millions of similar possibilities. With the improvement of natural language processing technology, AI recommendations of "what you might also like" will become faster and more tailored to your preferences. This algorithm can categorize collections (such as books) into topics based on commonly used vocabulary in the collection. Then, it divides each book into 5 to 15 most important topics and estimates the impact of each topic on the entire book. To compare books, researchers also used two other tools: word embeddings and optimal transport. Word embeddings is a technique that converts words into numerical lists to reflect their similarity in popular usage, enabling the comparison of topics across the entire collection and measuring the overlap of common topics between any pair of books. This technology is particularly effective when scanning large volumes of books and lengthy documents. By modeling books based on representative topics rather than individual words, high-level comparisons can be made. Mikhail Yurochkin, the lead author of the study and a researcher at IBM, said, "If you ask someone to compare two books, they might break down each book into easily understandable concepts and compare those concepts instead of directly comparing the books themselves." The researchers compared 1,720 pairs of books in the Gutenberg Project database in just one second, which was more than 800 times faster than previous methods, with improved accuracy. In addition to rapidly and accurately classifying documents, this method also provides insight into the decision-making process of the model. By examining the list of topics that appear, users can see the reasons why the model recommends certain documents.

RECENT AI NEWS

RECENT AI TOOLS