Meta releases open-source DINOv2 visual model for commercial distribution

2023-09-01

Meta has released the DINOv2 computer vision model, which is licensed under Apache 2.0, providing developers and researchers with greater flexibility in downstream tasks. Meta has also released a set of dense prediction models based on DINOv2 for semantic image segmentation and monocular depth estimation.

In addition, Meta has introduced FACET, a benchmark for evaluating fairness in computer vision models, including tasks such as classification and segmentation. The dataset contains 32,000 images from 50,000 individuals and includes demographic attributes such as perceived gender and age group, in addition to physical features.

FACET aims to serve as a standard benchmark for evaluating fairness in computer vision models and encourages the design and development of models that consider a more diverse population.

Meta's DINOv2 serves as the foundational model for computer vision. The company has showcased its advantages and hopes to combine DINOv2 with large-scale language models.

In May 2021, Meta's AI researchers introduced the concept of DINO (unsupervised distillation) as a self-supervised training model for image tasks, including classification or segmentation. Now, Meta has released an significantly improved version of DINOv2.

Like DINO, DINOv2 is a computer vision model trained using self-supervised learning and, according to Meta, outperforms most current state-of-the-art systems on all benchmark tests. Due to its use of self-supervised learning, labeled data is not required, and the DINO model can be trained using large unlabeled image datasets.

DINOv2 serves as the foundational model for all computer vision tasks

Meta states, "DINOv2 provides high-performance features that can be directly used as input for simple linear classifiers." This makes DINOv2 a flexible and versatile tool for a wide range of computer vision tasks, from image-level tasks such as image classification, instance retrieval, and video understanding, to pixel-level tasks such as depth estimation and semantic segmentation.

According to Meta, the DINOv2 model can be applied to various applications, including forest mapping with the World Resources Institute, estimating animal density and population, and biological research such as cell microscopy.

To train the model, Meta collected 1.2 billion images and curated and balanced the dataset. Ultimately, DINOv2 was trained using 142 million images. Like its predecessor, DINOv2 relies on Vision Transformers.

Meta plans to combine DINOv2 with large-scale language models

DINOv2 complements Meta's work in computer vision, particularly "Segment Anything," a recently released zero-shot image segmentation model with prompting capabilities. Meta sees DINOv2 as a building block that can be linked with other classifiers for application in many domains beyond segmentation.

The company has released the code and some models from the DINOv2 series. Now, the company plans to integrate DINOv2 into more complex AI systems, enabling interaction with large-scale language models. "Providing visual backbones that offer rich information about images will allow complex AI systems to perform deeper reasoning on images than just describing them with a single text sentence."

He also stated that models like CLIP, which are trained using image-text pairs, will ultimately be constrained by the captions. "For DINOv2, there is no such built-in limitation."

Summary

  • Meta has released DINOv2, a series of self-supervised computer vision models that can be used for various tasks such as depth estimation or segmentation.
  • The DINOv2 models have been trained on over 142 million images, and Meta considers them as foundational models that can be linked with other classifiers.
  • Meta intends to combine DINOv2 with large-scale language models to create more powerful models for image analysis and processing.

RECENT AI NEWS

RECENT AI TOOLS