arrow_back

Explore Machine Learning Models with Explainable AI: Challenge Lab

Join Sign in

Explore Machine Learning Models with Explainable AI: Challenge Lab

1 hour 30 minutes 5 Credits

GSP324

Google Cloud selp-paced labs logo

Introduction

In a challenge lab you’re given a scenario and a set of tasks. Instead of following step-by-step instructions, you will use the skills learned from the labs in the quest to figure out how to complete the tasks on your own! An automated scoring system (shown on this page) will provide feedback on whether you have completed your tasks correctly.

When you take a challenge lab, you will not be taught new Google Cloud concepts. You are expected to extend your learned skills, like changing default values and reading and researching error messages to fix your own mistakes.

To score 100% you must successfully complete all tasks within the time period!

This lab is recommended for students who have enrolled in the Explore Machine Learning Models with Explainable AI quest. Are you ready for the challenge?

Setup

Before you click the Start Lab button

Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources will be made available to you.

This hands-on lab lets you do the lab activities yourself in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab.

To complete this lab, you need:

  • Access to a standard internet browser (Chrome browser recommended).
Note: Use an Incognito or private browser window to run this lab. This prevents any conflicts between your personal account and the Student account, which may cause extra charges incurred to your personal account.
  • Time to complete the lab---remember, once you start, you cannot pause a lab.
Note: If you already have your own personal Google Cloud account or project, do not use it for this lab to avoid extra charges to your account.

Challenge scenario

You are a curious coder who wants to explore biases in public datasets using the What-If Tool. You decide to pull some mortgage data to train a couple of machine learning models to predict whether an applicant will be granted a loan. You specifically want to investigate how the two models perform when they are trained on different proportions of males and females in the datasets, and visualize their differences in the What-If Tool.

You are expected to have the skills and knowledge for these tasks, so don’t expect step-by-step guides.

Task 1. Start a JupyterLab Notebook instance

  1. Click on the Navigation Menu.

  2. Navigate to AI Platform, then click View notebook instances.

 AI Platform Dashboard

  1. On the Notebook instances page, click + NEW NOTEBOOK.

  2. In the Customize instance menu, select the version of TensorFlow Enterprise 2.6 Without GPUs.

  3. In the New notebook instance dialog, accept the default options and click CREATE.

After a few minutes, the AI Platform console will display your instance name, followed by Open Jupyterlab.

Note: If a notification pops up asking you to migrate your notebook instance to the new Notebooks API, click Enable Notebooks API to give your notebook additional functionality.
  1. Click Open JupyterLab. Your notebook is now set up.

Click Check my progress to verify the objective. Start a JupyterLab Notebook Instance

Task 2. Download the Challenge Notebook

  1. In your notebook, click the terminal.

  2. Clone the repo:

git clone https://github.com/GoogleCloudPlatform/training-data-analyst
  1. Go to the enclosing folder: training-data-analyst/quests/dei.

  2. Open the notebook file what-if-tool-challenge.ipynb.

  3. Download and import the dataset hmda_2017_ny_all-records_labels.

All the code to download and pre-process the dataset has been provided for you. The rest of the steps will be inside the notebook file. You should refer back to this lab guide to check your progress and get some hints.

Click Check my progress to verify the objective. Download the notebook and import the dataset

Task 3. Build and train your models

  • Use TensorFlow to build two models: one trained on the complete dataset, and one trained on the limited dataset. You should use the model Sequential(). To do so, please access the tf.keras.Sequential Document.

IMPORTANT: To accurately check your progress, the first model should be saved in the location saved_complete_model/saved_model.pb and the second in saved_limited_model/saved_model.pb.

Click Check my progress to verify the objective. Build and Train your models

Task 4. Deploy the models to AI Platform

Now, you'll deploy your models to the AI Platform.

Hint: You need to first create a storage bucket to store your models in.

Create the Complete AI Platform model

Use the following to create your first AI Platform model:

  • Model Name =
  • Version Name = v1
  • Python version = 3.7
  • Framework = TensorFlow
  • Framework version = 2.3.1
  • ML Runtime version = 2.3

Create the Limited AI Platform model

Use the following to create your second AI Platform model:

  • Model Name =
  • Version Name = v1
  • Python version = 3.7
  • Framework = TensorFlow
  • Framework version = 2.3.1
  • ML Runtime version = 2.3

Click Check my progress to verify the objective. Deploy the models to AI Platform

Task 5. Use the What-If Tool to explore biases

  • After your models are deployed to the AI Platform, you can use the following code to explore them in the What-If Tool in the notebook. We've created custom prediction functions custom_predict and bad_custom_predict that get the class predictions from the models:
config_builder = (WitConfigBuilder( examples_for_wit[:num_datapoints],feature_names=column_names) .set_custom_predict_fn(bad_custom_predict) .set_target_feature('loan_granted') .set_label_vocab(['denied', 'accepted']) .set_compare_custom_predict_fn(custom_predict) .set_model_name('limited') .set_compare_model_name('complete')) WitWidget(config_builder, height=800)

Examine the models

In the datapoint visualization, the top arc of points (not on the diagonal) are the females in the test data, where the limited dataset model under-scores female applicants compared to the complete dataset model. Feel free to take the rest of the time in the lab to explore your models in the What-If Tool. Once you're ready, answer the following questions.

Datapoint visualization graph

Review your results

Congratulations!

Explainable AI Badge

Earn your next skill badge

This self-paced lab is part of the Explore Machine Learning Models with Explainable AI quest. Completing this skill badge quest earns you the badge above, to recognize your achievement. Share your badge on your resume and social platforms, and announce your accomplishment using #GoogleCloudBadge.

This skill badge quest is part of Google Cloud’s Professional Machine Learning Engineer learning path. If you have already completed the other skill badge quests in this learning path, search the catalog for 20+ other skill badge quests that you can enroll in.

Google Cloud training and certification

...helps you make the most of Google Cloud technologies. Our classes include technical skills and best practices to help you get up to speed quickly and continue your learning journey. We offer fundamental to advanced level training, with on-demand, live, and virtual options to suit your busy schedule. Certifications help you validate and prove your skill and expertise in Google Cloud technologies.

Manual Last Updated July 27, 2022

Lab Last Tested March 21, 2021

Copyright 2022 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.