"Deciphering DNA Language Model with Generative Artificial Intelligence"

2023-06-14

Discovering Statistical Patterns in DNA Sequences with DNA Language Models

Large Language Models (LLMs) are trained on massive amounts of data to predict the next content of a phrase by learning the statistical relationships between letters and words. For example, GPT-4, a popular AI program that generates human-like text called ChatGPT, is trained on several petabytes (millions of gigabytes) of text.



Biologists have utilized the capabilities of these LLMs to uncover new insights in genetics by discovering statistical patterns in DNA sequences. Similar to nucleotide language models, DNA language models are trained on extensive DNA sequences.

DNA is often referred to as the "language of life." The genome is a set of DNA sequences that make up the genetic information of an organism. Unlike written languages, DNA only consists of four letters: A, C, G, and T, representing adenine, cytosine, guanine, and thymine, respectively. Despite its seemingly simple genetic language, its grammar remains a mystery to us. DNA language models can help us understand the grammar of the genome, one sequence at a time.

Multi-function Predictions

ChatGPT's ability to handle various tasks, from composing poetry to editing articles, has given it incredible power. DNA language models are also versatile. They can be used to determine the functions of various genomic regions and the interactions between multiple genes. Language models can also infer genomic characteristics and enable new analysis techniques without the need for a "reference genome" by deducing from DNA sequences.

The ability of DNA language models to generate new mutations in genomic sequences allows researchers to predict how these changes occur. For example, researchers have used language models based on genome size to predict and trace the evolution of the SARS-CoV-2 virus.

Exploring Distant Interactions in Genomes with DNA Language Models

Biologists have recently discovered that genomic regions once considered "junk DNA" interact with other parts of the genome in unexpected ways. Using DNA language models is a fast way to understand these hidden interactions. Language models can identify relationships between genes in distant regions of the genome by discovering patterns in long-distance DNA sequences.

Researchers at the University of California, Berkeley, have developed a DNA language model, known as the Genomic Pre-training Network (GPN), which can learn the effects of whole-genome variations. These variations, single-letter changes in the genome that lead to diseases or other physiological effects, are typically discovered through expensive research investigations called whole-genome association studies.

The GPN language model is trained on the genomes of seven species in the mustard family of plants. It can be fine-tuned for any species to identify genomic variations and accurately name the components of these mustard family genomes.

From Illusions to Creativity

The issue of "illusions" arises when language models produce seemingly reasonable but unrealistic outputs. For example, ChatGPT may generate fundamentally incorrect health advice. However, this "creativity" allows language models to effectively develop entirely new proteins in the context of protein design.

To improve the success of deep learning models like AlphaFold in predicting protein folding, researchers also use language models on protein datasets. Folding is a complex process that allows proteins, initially just a chain of amino acids, to take on useful forms. Given that DNA sequences control how proteins fold and obtain their structure and function, we can learn all the knowledge about protein structure and function solely from gene sequences.



Source: https://www.analyticsinsight.net/decrypting-dna-language-models-with-generative-ai/

RECENT AI NEWS

RECENT AI TOOLS