Optimization Tools for AI Models: Auptimizer

2019-11-11

In recent years, although there have been numerous open-source tools such as Databricks' AutoML toolkit, Salesforce's transfogrfai, and IBM's Watson Studio AutoAI, fine-tuning machine learning algorithms at a large scale remains a challenge. The task of finding the right hyperparameters (variables in the algorithm that control the overall performance of the model) is often time-consuming, involving job scheduling and data tracking. This is why scientists from LG's Advanced Artificial Intelligence Department developed Auptimizer, an open-source hyperparameter optimization framework designed to assist in fine-tuning and accounting for artificial intelligence models. It can be obtained from GitHub. In a paper introducing Auptimizer, the LG team wrote: Auptimizer simplifies the process of configuring various models and is reproducible. Like all hyperparameter algorithms, it can initialize the search space and configuration before assigning values to hyperparameters, then train the target model and update the results. The process of proposing, training, and updating is repeated until the optimal value is determined. Auptimizer adopts a gradient-based architecture search, where a controller based on AI models can generate "sub-model" strings, with the architecture of these "sub-models" specified by string variables. The controller gradually assigns higher probabilities to structures with higher accuracy of the trained sub-models, thereby improving its search capability. With just a few lines of code, Auptimizer can help users gradually complete experiment-related configuration settings and support switching between different hyperparameter algorithms and computing resources without the need to rewrite training scripts. Furthermore, its design can smoothly scale to other algorithms and resources without significant modifications. Once an experiment is defined and initialized, Auptimizer continuously checks for available resources and hyperparameter data, and runs jobs to determine the best model. After the workload is completed, it initiates an asynchronous function to record and save the results, automatically performing mapping and saving the hyperparameter values to a file for recovery and use in specific jobs. Users can specify the resources used in the experiment configuration, including processors and graphics chips. It is compatible with existing resource management tools and can track available resources and jobs in a database. Additionally, it has a basic tool for visualizing historical results. The team stated that future versions of Auptimizer will support end-to-end model building on edge devices, including model compression and neural structure search. In their paper, they wrote: "Auptimizer not only provides a universal platform for efficient development of new algorithms but also reduces the difficulty for data scientists to optimize hyperparameters in practice. Its scalability enables users to efficiently train models on all available computing resources, allowing practitioners to explore their ideas relatively easily using advanced algorithms."

RECENT AI NEWS

RECENT AI TOOLS