GSP917

Overview

In this lab, you use BigQuery for data processing and exploratory data analysis and the Vertex AI platform to train and deploy a custom TensorFlow Regressor model to predict customer lifetime value. The goal of the lab is to introduce to Vertex AI through a high value real world use case - predictive CLV. You start with a local BigQuery and TensorFlow workflow that you may already be familiar with and progress toward training and deploying your model in the cloud with Vertex AI.

Vertex AI is Google Cloud's next generation, unified platform for machine learning development and the successor to AI Platform announced at Google I/O in May 2021. By developing machine learning solutions on Vertex AI, you can leverage the latest ML pre-built components and AutoML to significantly enhance development productivity, the ability to scale your workflow and decision making with your data, and accelerate time to value.

Objectives

In this lab, you will:

• Train a TensorFlow model locally in a hosted Vertex Notebook.
• Create a managed Tabular dataset artifact for experiment tracking.
• Containerize your training code with Cloud Build and push it to Google Cloud Artifact Registry.
• Run a Vertex AI custom training job with your custom model container.
• Use Vertex TensorBoard to visualize model performance.
• Deploy your trained model to a Vertex Online Prediction Endpoint for serving predictions.
• Request an online prediction and explanation and see the response.

Setup and requirements

Before you click the Start Lab button

Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources will be made available to you.

This hands-on lab lets you do the lab activities yourself in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab.

To complete this lab, you need:

• Access to a standard internet browser (Chrome browser recommended).

Note: Use an Incognito or private browser window to run this lab. This prevents any conflicts between your personal account and the Student account, which may cause extra charges incurred to your personal account.

• Time to complete the lab---remember, once you start, you cannot pause a lab.

Note: If you already have your own personal Google Cloud account or project, do not use it for this lab to avoid extra charges to your account.

How to start your lab and sign in to the Google Cloud console

1. Click the Start Lab button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is the Lab Details panel with the following:

   • The Open Google Cloud console button
   • Time remaining
   • The temporary credentials that you must use for this lab
   • Other information, if needed, to step through this lab

2. Click Open Google Cloud console (or right-click and select Open Link in Incognito Window if you are running the Chrome browser).

   The lab spins up resources, and then opens another tab that shows the Sign in page.

   Tip: Arrange the tabs in separate windows, side-by-side.

   Note: If you see the Choose an account dialog, click Use Another Account.

3. If necessary, copy the Username below and paste it into the Sign in dialog.

   {{{user_0.username | "Username"}}}

   You can also find the Username in the Lab Details panel.

4. Click Next.

5. Copy the Password below and paste it into the Welcome dialog.

   {{{user_0.password | "Password"}}}

   You can also find the Password in the Lab Details panel.

6. Click Next.

   Important: You must use the credentials the lab provides you. Do not use your Google Cloud account credentials. Note: Using your own Google Cloud account for this lab may incur extra charges.

7. Click through the subsequent pages:

   • Accept the terms and conditions.
   • Do not add recovery options or two-factor authentication (because this is a temporary account).
   • Do not sign up for free trials.

After a few moments, the Google Cloud console opens in this tab.

Note: To view a menu with a list of Google Cloud products and services, click the Navigation menu at the top-left.

Activate Cloud Shell

Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.

1. Click Activate Cloud Shell at the top of the Google Cloud console.

When you are connected, you are already authenticated, and the project is set to your Project_ID, . The output contains a line that declares the Project_ID for this session:

Your Cloud Platform project in this session is set to {{{project_0.project_id | "PROJECT_ID"}}}

gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.

2. (Optional) You can list the active account name with this command:

gcloud auth list

3. Click Authorize.

Output:

ACTIVE: * ACCOUNT: {{{user_0.username | "ACCOUNT"}}} To set the active account, run: $ gcloud config set account `ACCOUNT`

4. (Optional) You can list the project ID with this command:

gcloud config list project

Output:

[core] project = {{{project_0.project_id | "PROJECT_ID"}}} Note: For full documentation of gcloud, in Google Cloud, refer to the gcloud CLI overview guide.

Task 1. Enable Google Cloud services

• In Cloud Shell, use gcloud to enable the services used in the lab:

gcloud services enable \ compute.googleapis.com \ iam.googleapis.com \ iamcredentials.googleapis.com \ monitoring.googleapis.com \ logging.googleapis.com \ notebooks.googleapis.com \ aiplatform.googleapis.com \ bigquery.googleapis.com \ artifactregistry.googleapis.com \ cloudbuild.googleapis.com \ container.googleapis.com

Task 2. Create Vertex AI custom service account for Vertex Tensorboard integration

1. Create custom service account:

SERVICE_ACCOUNT_ID=vertex-custom-training-sa gcloud iam service-accounts create $SERVICE_ACCOUNT_ID \ --description="A custom service account for Vertex custom training with Tensorboard" \ --display-name="Vertex AI Custom Training"

2. Grant it access to Cloud Storage for writing and retrieving Tensorboard logs:

PROJECT_ID=$(gcloud config get-value core/project) gcloud projects add-iam-policy-binding $PROJECT_ID \ --member=serviceAccount:$SERVICE_ACCOUNT_ID@$PROJECT_ID.iam.gserviceaccount.com \ --role="roles/storage.admin"

3. Grant it access to your BigQuery data source to read data into your TensorFlow model:

gcloud projects add-iam-policy-binding $PROJECT_ID \ --member=serviceAccount:$SERVICE_ACCOUNT_ID@$PROJECT_ID.iam.gserviceaccount.com \ --role="roles/bigquery.admin"

4. Grant it access to Vertex AI for running model training, deployment, and explanation jobs:

gcloud projects add-iam-policy-binding $PROJECT_ID \ --member=serviceAccount:$SERVICE_ACCOUNT_ID@$PROJECT_ID.iam.gserviceaccount.com \ --role="roles/aiplatform.user"

Task 3. Launch Vertex AI Workbench notebook

To create and launch a Vertex AI Workbench notebook:

1. In the Navigation Menu , click Vertex AI > Workbench.

2. On the Workbench page, click Enable Notebooks API (if it isn't enabled yet).

3. Click on User-Managed Notebooks tab then, click Create New.

4. Name the notebook.

5. Set Region to and Zone to .

6. In the New instance menu, choose the latest version of TensorFlow Enterprise 2.11 in Environment.

7. Click Advanced Options to edit the instance properties.

8. Click Machine type and then select e2-standard-2 for Machine type.

9. Leave the remaining fields at their default and click Create.

After a few minutes, the Workbench page lists your instance, followed by Open JupyterLab.

10. Click Open JupyterLab to open JupyterLab in a new tab. If you get a message saying beatrix jupyterlab needs to be included in the build, just ignore it.

Click Check my progress to verify the objective. Create a Vertex AI Notebook

Task 4. Clone the lab repository

Next you'll clone the training-data-analyst repo to your JupyterLab instance.

To clone the training-data-analyst repository in your JupyterLab instance:

1. In JupyterLab, click the Terminal icon to open a new terminal.

2. At the command-line prompt, type the following command and press ENTER:

git clone --depth=1 https://github.com/GoogleCloudPlatform/training-data-analyst

3. To confirm that you have cloned the repository, in the left panel, double click the training-data-analyst folder to see its contents.

It will take several minutes for the repo to clone.

Click Check my progress to verify the objective. Clone the lab repository

Task 5. Install lab dependencies

• Run the following to go to the training-data-analyst/self-paced-labs/vertex-ai/vertex-ai-qwikstart folder, then pip3 install requirements.txt to install lab dependencies:

cd training-data-analyst/self-paced-labs/vertex-ai/vertex-ai-qwikstart pip3 install --user -r requirements.txt sudo apt -y install python3-pandas pip uninstall openpyxl pip install openpyxl

Navigate to lab notebook

1. In the file browser, navigate to training-data-analyst > self-paced-labs > vertex-ai > vertex-ai-qwikstart, and open lab_exercise.ipynb.

Note: Be sure to open the notebook titled lab_exercise.ipynb to complete this lab.

2. Continue the lab in the notebook, and run each cell by clicking the Run icon at the top of the screen.

Alternatively, you can execute the code in a cell with SHIFT + ENTER.

Read the narrative and make sure you understand what's happening in each cell.

Congratulations!

In this lab, you ran a machine learning experimentation workflow using Google Cloud BigQuery for data storage and analysis and Vertex AI machine learning services to train and deploy a TensorFlow model to predict customer lifetime value. You progressed from training a TensorFlow model locally to training on the cloud with Vertex AI and leveraged several new unified platform capabilities such as Vertex TensorBoard and prediction feature attributions.

Next steps / Learn more

This lab is part of a series of labs called Qwik Starts. These labs are designed to give you a little taste of the many features available with Google Cloud. Search for "Qwik Starts" in the lab catalog to find the next lab you'd like to take!

Google Cloud training and certification

...helps you make the most of Google Cloud technologies. Our classes include technical skills and best practices to help you get up to speed quickly and continue your learning journey. We offer fundamental to advanced level training, with on-demand, live, and virtual options to suit your busy schedule. Certifications help you validate and prove your skill and expertise in Google Cloud technologies.

Manual Last Updated May 16, 2024

Lab Last Tested May 16, 2024

Copyright 2024 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.