Overview

In this lab, you will walk through the authoring of a Cloud Build CI/CD workflow that automatically builds and deploys a Kubeflow Pipeline (KFP). You will also integrate your workflow with GitHub by setting up a trigger that starts the workflow when a new tag is applied to the GitHub repo hosting the pipeline's code.

Objectives

• Create a custom Cloud Build builder to pilote AI Platform Pipelines

• Write a Cloud Build config file to build and push all the artifacts for a KFP

• Setup a Cloud Build Github trigger to rebuild the KFP

Task 0: Setup

For each lab, you get a new Google Cloud project and set of resources for a fixed time at no cost.

1. Make sure you signed into Qwiklabs using an incognito window.

2. Note the lab's access time (for example, and make sure you can finish in that time block.

3. When ready, click .

4. Note your lab credentials. You will use them to sign in to the Google Cloud Console.

5. Click Open Google Console.

6. Click Use another account and copy/paste credentials for this lab into the prompts.

7. Accept the terms and skip the recovery resource page.

Task 1: Create an instance of AI Platform Pipelines

1. From the Navigation menu, scroll down to AI Platform and pin the section for easier access later in the lab.

2. Click Pipelines.

3. Then click New Instance.

4. Click Configure.

5. Check Allow access to the following Cloud APIs, leave the name as is, and then click Create Cluster.

This should take 5-10 minutes to complete. Wait for the cluster to finish before proceeding to the next step. In the first tab opened, you can view the Cluster Creation taking place in the GKE section of the Cloud Console, or see the individual VMs spinning up in the GCE section of the Cloud Console.

6. When the cluster creation is complete, check the Terms of Service box, leave other settings unchanged, and then click Deploy. You will see the individual services of KFP deployed to your GKE cluster. Proceed to the next step while installation occurs.

Click Check my progress to verify the objective. Creating an instance of AI Platform Pipelines

Task 2: Access AI Platform Notebook

An instance of AI Platform Notebooks is used as a primary experimentation/development workbench. The instance is configured using a TensorFlow 2.3 Enterprise image.

In this lab, the AI Notebook instance is created for you at lab start time. Note: It will take approximately 5-10 minutes after starting the lab for the notebook instance to appear in AI Platform Notebooks. Wait for the instance to become available before proceeding.

Click Check my progress to verify the objective. Wait for instance of AI Platform Notebooks

Task 3: Clone the mlops-on-gcp repo within your AI Platform Notebooks instance

To clone the mlops-on-gcp notebook in your JupyterLab instance:

Step 1

In JupyterLab, click the Terminal icon to open a new terminal.

Step 2

At the command-line prompt, type in the following command and press Enter.

git clone https://github.com/GoogleCloudPlatform/mlops-on-gcp

Note

If the cloned repo does not appear in the JupyterLab UI you can use the top line menu and under Git --> Clone a repository, clone the repo (https://github.com/GoogleCloudPlatform/mlops-on-gcp) using the UI.

Step 3

Confirm that you have cloned the repository by double clicking on the mlops-on-gcp directory and ensuring that you can see its contents. The files for all the Jupyter notebook-based labs throughout this course are available in this directory.

Navigate to the mlops-on-gcp notebook

Step 1

In the notebook interface, navigate to mlops-on-gcp > workshops > kfp-caip-sklearn > lab-03-kfp-cicd > exercises and open lab-03.ipynb.

Step 2

In the notebook interface, click on Edit > Clear All Outputs (click on Edit, then in the drop-down menu, select Clear All Outputs).

Carefully read through the notebook instructions and fill in lines marked with #TO DO where you need to complete the code as needed

Tip: To run the current cell you can click the cell and hit shift+enter. Other cell commands are found in the notebook UI under Run.

• Hints may also be provided for the tasks to guide you along. Highlight the text to read the hints (they are in white text).

• If you need more help, you may take a look at the complete solution by navigating to mlops-on-gcp > workshops > kfp-caip-sklearn > lab-03-kfp-cicd and opening lab-03.ipynb.

Congratulations!

In this lab, you've learned how to work with Cloud Build CI/CD workflow that automatically builds and deploys a KFP.

End your lab

When you have completed your lab, click End Lab. Qwiklabs removes the resources you’ve used and cleans the account for you.

You will be given an opportunity to rate the lab experience. Select the applicable number of stars, type a comment, and then click Submit.

The number of stars indicates the following:

• 1 star = Very dissatisfied
• 2 stars = Dissatisfied
• 3 stars = Neutral
• 4 stars = Satisfied
• 5 stars = Very satisfied

You can close the dialog box if you don't want to provide feedback.

For feedback, suggestions, or corrections, please use the Support tab.

©2021 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.