Google Cloud AI Platform Update: Enhanced Training and Inference Capabilities

2019-10-30

Google recently announced an update to its Cloud AI Platform to enhance the training and prediction capabilities of machine learning and deep learning models.


Google Cloud AI Platform is an end-to-end machine learning platform (ML PaaS) for data scientists, ML developers, and AI engineers. Cloud AI Platform provides services to address the entire lifecycle of machine learning models, from data preparation to training to model serving. This update makes training and deploying ML models on the Google Cloud Platform more flexible and powerful.



Model Development


Running custom containers for training models on the Cloud AI Platform is now easier than ever. This feature allows users to run their own Docker container images with any pre-installed ML frameworks or algorithms on the AI platform.


Custom container support eliminates the constraints involved in training large-scale models in the cloud. Customers can now package custom container images with specific versions of language frameworks and tools used in their training pipelines, eliminating the need to choose a specific version of tools for training models on the platform. With custom containers, data scientists and ML developers can bring their own frameworks and libraries to the AI platform, even if they are not natively supported by the platform. Developers can build and test container images locally before deploying them to the cloud. DevOps teams can integrate the AI platform with existing CI/CD pipelines to automate the deployment process.


To simplify the process of selecting the right hardware configuration for training ML models, Google introduced the concept of expansion tiers, which are a set of predefined cluster specifications based on a class of GCE VMs. Each tier is defined based on its suitability for certain types of workloads.


Customers can also choose a custom tier where they can configure the compute resources for the master server, worker servers, and parameter servers. These servers in the cluster help with distributed training to accelerate the training of large datasets.



Model Deployment and Inference


The process of hosting a fully trained model is called inference.


Customers can host trained machine learning models in the Google Cloud AI platform and use the AI platform prediction service to infer the target values for new data. Cloud AI Platform Prediction manages the computational resources in the cloud to run ML models. Developers using ML models can request predictions from the deployed models and receive the predicted target values as a response.


Now, the Cloud AI Platform Prediction service allows customers to choose from a range of Google Compute Engine machine types to run ML models. Customers can add GPUs such as NVIDIA T4 or TPUs to accelerate the inference process. As a managed platform, the service handles provisioning, scaling, and serving without manual intervention. Previously, the online prediction service only supported choosing from one or four vCPU machine types.


GCP customers using the AI platform can now log prediction requests and responses directly to BigQuery for analysis and detecting biases, outliers, or determining the need for retraining to improve model accuracy.


Cloud AI Platform Prediction is supported by Google Kubernetes Engine to provide the required scalability. Since rebranding ML PaaS to AI platform at the Cloud NEXT event, Google has been continuously enhancing the service. The widespread availability of features like custom containers and GKE-based prediction service gives the platform greater flexibility and scalability for training and deploying machine learning models in the cloud.

RECENT AI NEWS

RECENT AI TOOLS