Loading...
No results found.

Apply your skills in Google Cloud console

10

Production Machine Learning Systems

Get access to 700+ labs and courses

Distributed Training with Keras

Lab 2 hours universal_currency_alt 5 Credits show_chart Advanced
info This lab may incorporate AI tools to support your learning.
Get access to 700+ labs and courses

Overview

The tf.distribute.Strategy API provides an abstraction for distributing your training across multiple processing units. The goal is to allow users to enable distributed training using existing models and training code, with minimal changes.

This lab uses the tf.distribute.MirroredStrategy, which does in-graph replication with synchronous training on many GPUs on one machine. Essentially, it copies all of the model's variables to each processor. Then, it uses all-reduce to combine the gradients from all processors and applies the combined value to all copies of the model.

MirroredStrategy is one of several distribution strategies available in TensorFlow core. For more information about strategies, see the distribution strategy guide.

Learning objectives

You will learn how to perform the following tasks:

  • Define a distribution strategy and set an input pipeline.
  • Create the Keras model.
  • Define the callbacks.
  • Train and evaluate the model.

Setup

For each lab, you get a new Google Cloud project and set of resources for a fixed time at no cost.

  1. Sign in to Qwiklabs using an incognito window.

  2. Note the lab's access time (for example, 1:15:00), and make sure you can finish within that time.
    There is no pause feature. You can restart if needed, but you have to start at the beginning.

  3. When ready, click Start lab.

  4. Note your lab credentials (Username and Password). You will use them to sign in to the Google Cloud Console.

  5. Click Open Google Console.

  6. Click Use another account and copy/paste credentials for this lab into the prompts.
    If you use other credentials, you'll receive errors or incur charges.

  7. Accept the terms and skip the recovery resource page.

Task 1. Launch Vertex AI Workbench instance

  1. In the Google Cloud console, from the Navigation menu (), select Vertex AI > Dashboard.

  2. Click Enable All Recommended APIs.

  3. In the Navigation menu, click Workbench.

    At the top of the Workbench page, ensure you are in the Instances view.

  4. Click Create New.

  5. Configure the Instance:

    • Name: lab-workbench
    • Region: Set the region to
    • Zone: Set the zone to
    • Advanced Options (Optional): If needed, click "Advanced Options" for further customization (e.g., machine type, disk size).

  1. Click Create.

This will take a few minutes to create the instance. A green checkmark will appear next to its name when it's ready.

  1. Click Open Jupyterlab next to the instance name to launch the JupyterLab interface. This will open a new tab in your browser.

  1. Click the Python 3 icon to launch a new Python notebook.

  1. Right-click on the Untitled.ipynb file in the menu bar and select Rename Notebook to give it a meaningful name.

Your environment is set up. You are now ready to start working with your Vertex AI Workbench notebook.

Click Check my progress to verify the objective. Launch Vertex AI Workbench instance

Task 2. Clone a course repo within your JupyterLab interface

The GitHub repo contains both the lab file and solutions files for the course.

  1. Copy and run the following code in the first cell of your notebook to clone the training-data-analyst repository.
!git clone https://github.com/GoogleCloudPlatform/training-data-analyst

  1. Confirm that you have cloned the repository. Double-click on the training-data-analyst directory and ensure that you can see its contents.

Click Check my progress to verify the objective. Clone a course repo within your JupyterLab interface

Task 3. Distributed training with Keras

  1. In the notebook interface, navigate to training-data-analyst > courses > machine_learning > deepdive2 > production_ml > labs, and open keras.ipynb.

  2. In the Select Kernel dialog, choose TensorFlow 2-11 (Local) from the list of available kernels.

  3. In the notebook interface, click Edit > Clear All Outputs.

Carefully read through the notebook instructions and fill in lines marked with #TODO where you need to complete the code.

Tip: To run the current cell, click the cell and press SHIFT+ENTER. Other cell commands are listed in the notebook UI under Run.

  • Hints may also be provided for the tasks to guide you along. Highlight the text to read the hints (they are in white text).
  • If you need more help, look at the complete solution at training-data-analyst > courses > machine_learning > deepdive2 > production_ml > solutions and open keras.ipynb.

End your lab

When you have completed your lab, click End Lab. Qwiklabs removes the resources you’ve used and cleans the account for you.

You will be given an opportunity to rate the lab experience. Select the applicable number of stars, type a comment, and then click Submit.

The number of stars indicates the following:

  • 1 star = Very dissatisfied
  • 2 stars = Dissatisfied
  • 3 stars = Neutral
  • 4 stars = Satisfied
  • 5 stars = Very satisfied

You can close the dialog box if you don't want to provide feedback.

For feedback, suggestions, or corrections, please use the Support tab.

Copyright 2022 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.

Previous Next

Before you begin

  1. Labs create a Google Cloud project and resources for a fixed time
  2. Labs have a time limit and no pause feature. If you end the lab, you'll have to restart from the beginning.
  3. On the top left of your screen, click Start lab to begin

This content is not currently available

We will notify you via email when it becomes available

Great!

We will contact you via email if it becomes available

One lab at a time

Confirm to end all existing labs and start this one

Use private browsing to run the lab

Use an Incognito or private browser window to run this lab. This prevents any conflicts between your personal account and the Student account, which may cause extra charges incurred to your personal account.
Preview