Developing Smart City Traffic Management System using OpenUSD and Synthetic Data

2023-08-03

Smart City Traffic Management Systems with OpenUSD and Synthetic Data Smart cities are the future of urban living. However, city planners may face various challenges, especially in the field of transportation. To succeed, all aspects of the city, including the environment, infrastructure, business, and education, must achieve functional integration. This is difficult because managing traffic flow alone is a complex issue, filled with challenges such as congestion, emergency response to accidents, and emissions. To address these challenges, developers are creating artificial intelligence software with on-site programmability and flexibility. These software-defined Internet of Things solutions provide scalable, plug-and-play products for real-time environments, such as traffic management, license plate recognition, smart parking, and accident detection. However, building effective artificial intelligence models is not easy. Common issues in training data, such as missing values, duplicate examples, incorrect labels, and erroneous feature values, can lead to inaccurate models. For autonomous vehicles, inaccurate results can be dangerous and lead to inefficient traffic systems or poor city planning. Real-Time City Traffic Digital Twins SmartCow, an end-to-end artificial intelligence engineering company and NVIDIA Metropolis partner, has created digital twins of traffic scenes on NVIDIA Omniverse. These digital twins generate synthetic datasets and validate the performance of artificial intelligence models. The team addresses common challenges in building optimized artificial intelligence training pipelines by using synthetic data generated with NVIDIA Omniverse Replicator. The foundation of Omniverse is Universal Scene Description (USD), also known as OpenUSD. USD is a powerful exchange format with highly scalable attributes for building virtual worlds. The digital twins of smart cities rely on highly scalable and interoperable USD capabilities to accurately simulate large, high-fidelity scenes of the real world. Omniverse Replicator, the core extension of the Omniverse platform, enables developers to programmatically generate annotated synthetic data to kickstart the training of perception artificial intelligence models. Synthetic data is particularly useful when real datasets are limited or hard to obtain. Using digital twins, the SmartCow team generates synthetic datasets that accurately represent real traffic scenes and violations. These synthetic datasets help validate artificial intelligence models and optimize the training process. Building License Plate Recognition Extensions One of the biggest challenges in intelligent traffic management systems is license plate recognition. Developing a model that works in different countries, municipalities, and environments requires diverse and robust training data. To provide sufficient and diverse training data for the model, SmartCow has developed an extension in Omniverse to generate synthetic data. Extensions in Omniverse are reusable components or tools that provide powerful functionalities to enhance workflows and processes. After building an extension in Omniverse Kit, developers can easily distribute it to clients for use in Omniverse USD Composer, Omniverse USD Presenter, and other applications. SmartCow's extension, called License Plate Synthetic Generator (LP-SDG), uses environment randomizers and physical randomizers to make the synthetic dataset more diverse and realistic. The environment randomizer simulates changes in lighting, weather, and other factors in the digital twin environment, such as rain, snow, fog, or dust. The physical randomizer simulates scratches, dirt, dents, and discoloration that may affect the model's ability to recognize license plate numbers. Generating Synthetic Data with NVIDIA Omniverse Replicator The data generation process starts with creating a 3D environment in Omniverse. The digital twins in Omniverse can be used for many simulation scenarios, including generating synthetic data. The initial 3D scenes are built by SmartCow's in-house technical artists to ensure the digital twins closely match reality. After generating the scenes, domain randomization is used to change the light sources, textures, camera positions, and materials. The entire process is done programmatically using the built-in Omniverse Replicator APIs. The generated data is exported with bounding box annotations and additional output variables for training. Model Training The initial model is trained on 3,000 real images to understand the performance of the baseline model and validate aspects such as correct bounding box sizes and lighting variations. Next, the team conducts a series of experiments comparing benchmark tests on the generated synthetic datasets, including 3,000 samples, 30,000 samples, and 300,000 samples. "Due to the realism obtained through Omniverse, models trained with synthetic data sometimes outperform models trained with real data," said Natalia Mallia, software engineer at SmartCow. "Using synthetic data actually eliminates biases that naturally exist in real image training datasets." To provide accurate benchmarks and comparison results, the data is randomized during training on the three sizes of synthetic datasets, including consistent parameters such as time, scratches, and observation angles. Real-world data is not mixed with synthetic data for training. Each model is validated with a dataset of approximately 1,000 real images. SmartCow integrates the training data from Omniverse LP-SDG extension with NVIDIA TAO, a low-code AI model training toolkit that leverages the power of transfer learning for model fine-tuning. The team uses a pre-trained license plate detection model available in the NGC catalog and fine-tunes it using TAO and NVIDIA DGX A100 systems. Deploying the Model with NVIDIA DeepStream The AI model is then deployed on custom edge devices using the NVIDIA DeepStream SDK. They implement a continuous learning loop involving collecting drift data from edge devices, feeding the data back into Omniverse Replicator to generate retrainable datasets, annotating the datasets with automated labeling tools, and feeding them back into TAO for training. This closed-loop pipeline helps create accurate and effective AI models for automatically detecting traffic directions for each lane and identifying vehicles with abnormally long parking times. Starting with Synthetic Data, Digital Twins, and AI Intelligence in Smart City Traffic Management To build more efficient AI models in the field of smart city traffic, using a workflow that generates synthetic datasets and validates AI model performance with digital twins is an important step. Synthetic datasets overcome the challenges of limited datasets and provide accurate and effective AI models, leading to efficient traffic systems and better city planning. Source: https://developer.nvidia.com/blog/developing-smart-city-traffic-management-systems-with-openusd-and-synthetic-data/

RECENT AI NEWS

RECENT AI TOOLS