GSP1260

Overview

Finding the right datasets and assets for machine learning workflows with Vertex AI can be tricky. It may not be easy to find datasets ready to use for model training and when you do, it may not be clear how to use the data.

Dataplex Universal Catalog can help! Dataplex is a centralized platform that unifies data assets distributed across an organization. It also supports data governance without the need to move or transfer data from its original location. Dataplex provides easy-to-use tools to discover, catalog, and manage data to power your analytics and machine learning workflows, while leveraging the security of IAM in Google Cloud.

In this lab, you walk through a typical use case for using Dataplex. Specifically, you catalog and discover data assets that you can use to train Vertex AI models.

Note: For additional support throughout the lab, expand the hint boxes for help. Challenge yourself by expanding hints progressively (or not!), as you need more support.

Pre-created data assets in each project

This lab provides pre-created data assets across two Google Cloud projects. This includes data assets in BigQuery and Cloud Storage, which you use to create a new Vertex AI dataset in the first task.

Project	Project ID	Assigned region	Available Data
1			Cloud Storage bucket named -bucket containing the images of damaged car parts
2			Three BigQuery datasets: damaged_car_ownership (containing a table for owners of the damaged cars); damaged_car_image_info (containing a table with additional image information, such as owner ID and location/timestamp of the image); and damaged_car_image_metadata (an object dataset with metadata derived from the image files in Cloud Storage, such as storage path and updated date)

Multiple projects and users

First, you take on the role of the Data Engineer who is helping their organization get data into usable and accessible formats for machine learning workflows. In Project 1, you create a Vertex AI dataset from images in Cloud Storage, and then you create an aspect type and add it to a Vertex AI dataset (and other potentially useful data in BigQuery), so that it is easy for others in your organization to find it using Dataplex.

Then, you transition to the role of the Data Scientist or Machine Learning Engineer who is looking for existing data assets that they can use to train new models. In Project 2, you use the aspects applied by the Data Engineer to search for relevant data assets in Dataplex. Then, you create a new aspect type for personally identifiable information (PII) and add it to additional assets that can be useful for future modeling efforts.

User	Goal	Username	Primary project
1 - Data Engineer	Create and add aspects to assets to help others find available data for a Vertex AI model to be named damaged_car_parts across multiple projects.
2 - Data Scientist or ML Engineer	Use aspects to search for Vertex AI assets to train the model to be named damaged_car_parts, and create and add a new aspect for PII to protected data assets.

What you'll do

In this lab, you learn how to:

• Create a Vertex AI managed dataset.
• Create aspect types in Dataplex.
• Add aspects to Vertex AI assets.
• Search for Vertex AI assets using filters and aspects.
• View lineage of Vertex AI assets.

Prerequisites

While not required, it is helpful to have some previous knowledge about how Dataplex and Vertex AI are commonly used within Google Cloud workflows. For an introduction to these tools before you begin this lab, complete the following labs:

• Create and Add Aspects to Dataplex Assets
• Get Started with Vertex AI Studio
• Identify Damaged Car Parts with Vertex AutoML Vision

Setup and requirements

Note: To begin this lab, follow the instructions below to log into Project 1:

as Username 1: .

Before you click the Start Lab button

Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources are made available to you.

This hands-on lab lets you do the lab activities in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials you use to sign in and access Google Cloud for the duration of the lab.

To complete this lab, you need:

• Access to a standard internet browser (Chrome browser recommended).

Note: Use an Incognito (recommended) or private browser window to run this lab. This prevents conflicts between your personal account and the student account, which may cause extra charges incurred to your personal account.

• Time to complete the lab—remember, once you start, you cannot pause a lab.

Note: Use only the student account for this lab. If you use a different Google Cloud account, you may incur charges to that account.

How to start your lab and sign in to the Google Cloud console

1. Click the Start Lab button. If you need to pay for the lab, a dialog opens for you to select your payment method. On the left is the Lab Details pane with the following:

   • The Open Google Cloud console button
   • Time remaining
   • The temporary credentials that you must use for this lab
   • Other information, if needed, to step through this lab

2. Click Open Google Cloud console (or right-click and select Open Link in Incognito Window if you are running the Chrome browser).

   The lab spins up resources, and then opens another tab that shows the Sign in page.

   Tip: Arrange the tabs in separate windows, side-by-side.

   Note: If you see the Choose an account dialog, click Use Another Account.

3. If necessary, copy the Username below and paste it into the Sign in dialog.

   {{{user_0.username | "Username"}}}

   You can also find the Username in the Lab Details pane.

4. Click Next.

5. Copy the Password below and paste it into the Welcome dialog.

   {{{user_0.password | "Password"}}}

   You can also find the Password in the Lab Details pane.

6. Click Next.

   Important: You must use the credentials the lab provides you. Do not use your Google Cloud account credentials. Note: Using your own Google Cloud account for this lab may incur extra charges.

7. Click through the subsequent pages:

   • Accept the terms and conditions.
   • Do not add recovery options or two-factor authentication (because this is a temporary account).
   • Do not sign up for free trials.

After a few moments, the Google Cloud console opens in this tab.

Note: To access Google Cloud products and services, click the Navigation menu or type the service or product name in the Search field.

Activate Cloud Shell

Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.

1. Click Activate Cloud Shell at the top of the Google Cloud console.

2. Click through the following windows:

   • Continue through the Cloud Shell information window.
   • Authorize Cloud Shell to use your credentials to make Google Cloud API calls.

When you are connected, you are already authenticated, and the project is set to your Project_ID, . The output contains a line that declares the Project_ID for this session:

Your Cloud Platform project in this session is set to {{{project_0.project_id | "PROJECT_ID"}}}

gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.

3. (Optional) You can list the active account name with this command:

gcloud auth list

4. Click Authorize.

Output:

ACTIVE: * ACCOUNT: {{{user_0.username | "ACCOUNT"}}} To set the active account, run: $ gcloud config set account `ACCOUNT`

5. (Optional) You can list the project ID with this command:

gcloud config list project

Output:

[core] project = {{{project_0.project_id | "PROJECT_ID"}}} Note: For full documentation of gcloud, in Google Cloud, refer to the gcloud CLI overview guide.

Task 1. Create a Vertex AI managed dataset

For this task, be sure that you have logged into Project 1 () as Username 1 ().

In a typical workflow to create a new Vertex AI dataset for image classification, you create a CSV input file, in which each row contains a URL image path to a training image and the associated label for that image.

In this lab, this CSV file has been pre-created for you in Cloud Storage as:

gs://-bucket/damaged-car-images/data.csv

In this task, begin by updating the image paths in data.csv with your bucket name, and then uploading the revised CSV file back to your Cloud Storage bucket. Then, use this file when you create a new Vertex AI dataset to connect the dataset to the training images in that Cloud Storage bucket.

Remember that you get help by expanding the hint boxes as you need them!

Update data.csv with the path to your bucket

Start by downloading the data.csv file locally to Cloud Shell, so you can update the image paths and upload the revised file back to Cloud Storage.

1. In Cloud Shell, create a local copy of data.csv from your pre-created bucket, so that you can easily update the image paths with your bucket name:

gsutil cp gs://{{{project_0.project_id}}}-bucket/damaged-car-images/data.csv .

If prompted, click Authorize.

2. Create a variable for the path to your bucket and subdirectory:

export DIRECTORY={{{project_0.project_id}}}-bucket/damaged-car-images

3. Review data.csv to see that the original paths:

cat ./data.csv

The results display the current path (gs://damaged-car-parts-vertex-dataset/), which you update in the next step.

5. Now update the image paths in data.csv with your bucket name and subdirectory:

sed -i -e "s|damaged-car-parts-vertex-dataset|${DIRECTORY}|g" ./data.csv

6. Review the updated data.csv to see that the paths have been updated to reference your desired bucket and subdirectory:

cat ./data.csv

The results display the updated paths that begin with gs://{{{project_0.project_id}}}-bucket/damaged-car-images/.

7. Upload the revised data.csv back to your Cloud Storage bucket:

gsutil cp ./data.csv gs://${DIRECTORY}

Check that data.csv has been updated in Cloud Storage

Although it's not a required step of this workflow, it's a good idea to check that the data.csv file has been uploaded to the desired location and successfully updated.

Check that data.csv was updated in your bucket using any method you already know, or expand the hint below for some helpful steps.

Full solution (Expand to see all of the steps!)

Create a managed dataset in Vertex AI

Next, create a new managed dataset in Vertex AI for single label image classification.

1. Navigate to Vertex AI in the Google Cloud console by clicking on the __Navigation menu () > Vertex AI > Dashboard.

2. Click Enable All Recommended APIs.

3. In the Vertex AI navigation menu, under Model Development, click Datasets.

These first three steps got you to the appropriate location in your Google Cloud project.

4. Now, on your own, create a Vertex AI dataset named damaged_car_parts for single-label image classification in the assigned region for Project 1 ().

If the option to select a region is null, you can assume that it will default to the assigned for this lab.

Expand the hints below for some helpful steps!

Option 1: Review the docs

Option 2: Adapt steps from a related lab

Full solution (Expand to see all of the steps!)

When you have successfully created the dataset, the page refreshes and provides new options for selecting an import method, which you use in the next section.

Connect your Vertex AI dataset to your training images using data.csv

Now, use your updated data.csv to import the image files of damaged car parts to the Vertex AI dataset.

1. For Select an import method, click Select import files from Cloud Storage.

2. In the Select import files from Cloud Storage section, click Browse, and navigate to data.csv in your Cloud Storage bucket (gs://-bucket).

3. Click Select.

   Once you've selected an appropriate file, a green checkbox appears to the left of the file path.

4. Click Continue.

Note: It can take around 10 to 15 minutes for your images to import and be aligned with their categories. You need to wait for this step to complete before checking your progress for this task.

During this time, you can leave this browser tab open, proceed to Task 2 in a new browser tab, and complete the first section titled Create an aspect type in Dataplex.

Click Check my progress to verify the objective.

Create a Vertex AI dataset

Task 2. Add an aspect to the Vertex AI dataset

For this task, stay logged into Project 1 () as Username 1 ().

Now that you have created the Vertex AI dataset, it is time to add an aspect to it, so that others can easily find it in Dataplex Universal Catalog.

In this task, begin by creating an aspect type for Vertex AI model names. Then, you add that aspect to the Vertex AI dataset and associate it with the model name for damaged_car_parts.

Create an aspect type in Dataplex

Start by creating an aspect type for associated model names.

1. In the Google Cloud console title bar, type Dataplex Universal Catalog in the Search field, and then click Dataplex Universal Catalog from the search results.

2. In the left pane, under Manage metadata, click Catalog.

3. Click Create aspect type.

4. Enter the required information below to define the aspect type:

Property	Value
Display Name	VertexAI_Data
Location

5. In the Template section, click Add field and enter the required information below to add a new field to the aspect type:

Property	Value
Field Display Name	Related Models
Type	Text
Text Type	Rich Text

5. Select the Is Required checkbox.

6. Click Done.

7. Click Save.

Click Check my progress to verify the objective.

Create an aspect type in Dataplex

Search for Vertex AI assets using filters

Note: Before you continue, check that your Vertex AI dataset named damaged_car_parts has been created.

Next, find the Vertex AI dataset that you created using filters for systems and data types in Dataplex.

1. In the Dataplex menu, under Discover, click Search.

Make sure that the search platform is selected as Dataplex Universal Catalog on the top right.

2. For Filters > Systems, enable the checkbox for Vertex AI.

3. Click the damaged_car_parts dataset.

Add an aspect to the Vertex AI dataset

Now, add an aspect to the Vertex AI dataset to associate it with the model name damaged_car_parts.

1. Scroll down to the Tags & aspects section. Next to Optional tags & aspects, click Add.

2. Type VertexAI_Data in the Filter field, and then click the VertexAI_Data aspect from the results.

3. For Related Models, type damaged_car_parts.

4. Click Save.

Remain on this details page for the damaged_car_parts dataset.

Click Check my progress to verify the objective.

Add an aspect to the Vertex AI dataset

Add an overview description to the Vertex AI dataset

Adding an overview for a dataset asset in Dataplex helps others understand the intended purpose of the data and how it can or should be used.

Using the available options on the current page for the damaged_car_parts dataset, add an overview for the dataset that can be useful to others when they discover this dataset.

For example, the text can be something like: This Vertex AI dataset contains images of damaged car parts including bumpers, windshields, etc. Each image has been assigned a single label to categorize the images by these car part types.

Full solution (Expand to see all of the steps!)

Task 3. Apply your skills by adding the same aspect to other assets

For this task, stay logged into Project 1 () as Username 1 ().

In the previous task, you created a reusable aspect type that you can add to multiple data assets with associated model names.

1. Add the same aspect to two additional BigQuery datasets that contain more information about the damaged car part images, so that these datasets are also associated with the model name for damaged_car_parts:

• damaged_car_ownership containing a table for owners of the damaged cars
• damaged_car_image_info containing a table with additional image information (such as owner ID and location/timestamp of the image)

2. Add some descriptive text to the overview for each dataset in Dataplex (just like you did previously for the Vertex AI dataset).

Expand the hints below for some helpful steps!

Option 1: Review Task 2

Option 2: Adapt steps from a related lab

Full solution (Expand to see all of the steps!)

Click Check my progress to verify the objective.

Apply your skills by adding the same aspect to other assets

Task 4. Search for assets using aspects and view lineage

For this task, log into Project 2 () as Username 2 ().

Expand the hint below for help with switching to a new project and user.

Full solution (Expand to see all of the steps!)

Now that the Data Engineer (you as Username 1!) has created an aspect type and added the aspect to assets, you can experience being the data scientist or machine learning engineer that benefits from these useful aspects to search for relevant data assets and view their lineage.

Search for assets using aspects

1. In the Dataplex menu, under Discover, click Search.

Make sure that the search platform is selected as Dataplex Universal Catalog on the top right.

2. For Filters > Aspects, enable the checkbox for VertexAI_Data.

3. Click on an asset name to see more details (such as the damaged_car_parts dataset).

Search for specific Vertex AI Assets using the same aspect

Now that you have the steps to search for assets using aspects, search for the three data assets that you previously added.

1. Use the aspect to find the following three assets and review the stored information for Overview and Related Model:

• damaged_car_parts (Vertex AI dataset for the images of damaged car parts)
• damaged_car_ownership (BigQuery dataset containing a table for owners of the damaged cars)
• damaged_car_image_info (BigQuery dataset containing a table with additional image information)

2. Notice that the BigQuery object dataset named damaged_car_image_metadata is not listed in your search results. Why not?

Expand to see the answer

View lineage for assets

Last, review the lineage of some of these assets to learn more about when and how they were created.

1. In the Dataplex search options, click damaged_car_parts > Lineage.

2. Then, click the asset name for damaged_car_parts on the Graph tab.

In addition to the date and time that the dataset was created, you can also see that it was created in a different project () than the current project (), and the region in which it was created ().

3. Now, return to the Search page, and search for damaged_car_image_metadata using its full name.

4. Click damaged_car_image_metadata > Entry list.

5. Click the table name for bumper_images, and then click Lineage.

6. Click the asset name for bumper_images on the Graph tab.

Similar to damaged_car_parts, you can see the created date, time, project ID, and region in which the asset was created.

With more elasped time, Dataplex lineage can also populate more information, in which case you would see the full lineage of the bumper_images table in the object dataset: .jpg files from a Cloud Storage directory named bumper are used in simple data pipeline to create the resulting bumper_images metadata table.

Task 5. Apply your skills by adding an aspect to protected data

For this task, stay logged into Project 2 () as Username 2 ().

While working with the data assets, you notice that there are additional aspect types that need to be created. Specifically, the BigQuery dataset with ownership information for the damaged cars contains personally identifiable information (PII) that should be handled carefully.

Apply your skills to create an aspect type for PII that you can apply to specific columns in the BigQuery table named owner_info in the damaged_car_ownership dataset.

You already created an aspect type in Task 2. You can use those steps as a starting point and modify them to the specifications provided below.

Remember that you get help by expanding the hint boxes as you need them!

Apply your skills by creating an aspect type for protected data

Begin by creating a new public aspect type called Protected Data with one enumerated field called Protected Data Flag with YES and NO values. Be sure to create the aspect in the assigned region for Project 2 ().

Option 1: Review the docs

Option 2: Adapt steps from a related lab

Full solution (Expand to see all of the steps!)

Note: If the progress check is not successful after implementing the full solution, check that you are logged into Project 2

as Username 2: .

Click Check my progress to verify the objective.

Apply your skills by creating an aspect type for protected data

Apply your skills by adding the aspect for protected data to assets

Add the aspect for Protected Data to the BigQuery table named owner_info in the damaged_car_ownership dataset.

You decide not to identify the columns for owner_id, state, and age as PII (as they cannot be used to identify specific individuals), and instead focus on adding the aspect to the following columns:

1. first_name
2. last_name
3. email
4. city
5. zip

Option 1: Review the docs

Option 2: Adapt steps from a related lab

Full solution (Expand to see all of the steps!)

Click Check my progress to verify the objective.

Apply your skills by adding the aspect for protected data to assets

Search for protected data using aspects

Last, use the aspect to search for data identified as containing PII.

Remember that you searched for assets using aspects in Task 4. Repeat those steps to find data assets identified as containing PII, or expand a hint below for some helpful steps.

Option 1: Ask Gemini

Option 2: Adapt steps from a related lab

Full solution (Expand to see all of the steps!)

Congratulations!

In this lab, you learned how to create a Vertex AI managed dataset, create aspect types in Dataplex, add aspects to Vertex AI assets, search for Vertex AI assets using filters and aspects, and view lineage of Vertex AI assets. Now you have what it takes to start cataloging and discovering models and datasets in your organization's projects using Dataplex!

Next steps / Learn more

• Learn more about data governance, quality, and security using Dataplex by completing the Build a Data Mesh with Dataplex skill badge.

Google Cloud training and certification

...helps you make the most of Google Cloud technologies. Our classes include technical skills and best practices to help you get up to speed quickly and continue your learning journey. We offer fundamental to advanced level training, with on-demand, live, and virtual options to suit your busy schedule. Certifications help you validate and prove your skill and expertise in Google Cloud technologies.

Manual Last Updated August 28, 2025

Lab Last Tested August 28, 2025

Copyright 2025 Google LLC. All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.