GSP138

Overview

This lab will show you how to deploy a set of Cloud Functions in order to process images and videos with the Cloud Vision API and Cloud Video Intelligence API.

Social marketing campaigns often invite consumers to submit user-generated images and videos. Campaigns that solicit videos and images often use them for contest submissions, product testimonials, or as user-generated content for public campaign websites. Processing these submissions at scale requires considerable resources.

The Cloud Video Intelligence and Cloud Vision APIs offer you a scalable and serverless way to implement intelligent image and video filtering, accelerating submission processing. If you use the safe-search feature in the Vision API solution and the explicit content detection feature in the Video Intelligence API, you can eliminate images and videos that are identified as unsafe or undesirable content before further processing.

Objectives

• Deploy four Cloud Functions.
• Create the supporting Cloud Storage buckets, Cloud Pub/Sub topics, and Cloud Storage Pub/Sub Notifications.
• Create the supporting BigQuery dataset and table.

Setup and requirements

You'll need image and video files that you can upload into the lab for analysis. Ideally they would be of different types - with people whose faces can be seen, no people, landscape, close-ups - so you can see how the image analysis treats the image. You can also just use a single image or video.

Before you click the Start Lab button

Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources will be made available to you.

This hands-on lab lets you do the lab activities yourself in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab.

To complete this lab, you need:

• Access to a standard internet browser (Chrome browser recommended).

Note: Use an Incognito or private browser window to run this lab. This prevents any conflicts between your personal account and the Student account, which may cause extra charges incurred to your personal account.

• Time to complete the lab---remember, once you start, you cannot pause a lab.

Note: If you already have your own personal Google Cloud account or project, do not use it for this lab to avoid extra charges to your account.

How to start your lab and sign in to the Google Cloud Console

1. Click the Start Lab button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is the Lab Details panel with the following:

   • The Open Google Console button
   • Time remaining
   • The temporary credentials that you must use for this lab
   • Other information, if needed, to step through this lab

2. Click Open Google Console. The lab spins up resources, and then opens another tab that shows the Sign in page.

   Tip: Arrange the tabs in separate windows, side-by-side.

   Note: If you see the Choose an account dialog, click Use Another Account.

3. If necessary, copy the Username from the Lab Details panel and paste it into the Sign in dialog. Click Next.

4. Copy the Password from the Lab Details panel and paste it into the Welcome dialog. Click Next.

   Important: You must use the credentials from the left panel. Do not use your Google Cloud Skills Boost credentials. Note: Using your own Google Cloud account for this lab may incur extra charges.

5. Click through the subsequent pages:

   • Accept the terms and conditions.
   • Do not add recovery options or two-factor authentication (because this is a temporary account).
   • Do not sign up for free trials.

After a few moments, the Cloud Console opens in this tab.

Note: You can view the menu with a list of Google Cloud Products and Services by clicking the Navigation menu at the top-left.

Activate Cloud Shell

Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.

1. Click Activate Cloud Shell at the top of the Google Cloud console.

When you are connected, you are already authenticated, and the project is set to your PROJECT_ID. The output contains a line that declares the PROJECT_ID for this session:

Your Cloud Platform project in this session is set to YOUR_PROJECT_ID

gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.

2. (Optional) You can list the active account name with this command:

gcloud auth list

3. Click Authorize.

4. Your output should now look like this:

Output:

ACTIVE: * ACCOUNT: student-01-xxxxxxxxxxxx@qwiklabs.net To set the active account, run: $ gcloud config set account `ACCOUNT`

5. (Optional) You can list the project ID with this command:

gcloud config list project

Output:

[core] project = <project_ID>

Example output:

[core] project = qwiklabs-gcp-44776a13dea667a6 Note: For full documentation of gcloud, in Google Cloud, refer to the gcloud CLI overview guide.

Architecture

The following diagram outlines the high-level architecture:

Task 1. Initializing your environment

Prepare for the lab by setting up some environment variables that you'll need in the lab.

• Enter the following command in the Cloud Shell to the variables that are used later in the lab:

export PROJECT_ID=$(gcloud info --format='value(config.project)') export IV_BUCKET_NAME=${PROJECT_ID}-upload export FILTERED_BUCKET_NAME=${PROJECT_ID}-filtered export FLAGGED_BUCKET_NAME=${PROJECT_ID}-flagged export STAGING_BUCKET_NAME=${PROJECT_ID}-staging Note: The lab's projectID is used and the four storage bucket names are created for you by appending the suffixes -upload, -filtered, -flagged, and -staging to the project ID in order to create a set of globally unique and valid Cloud Storage bucket names that will be used by the lab to process and store the uploaded image and video files. You can override these values with any valid storage bucket names if you prefer.

Task 2. Creating Cloud Storage buckets

Cloud Storage buckets provide a storage location for uploading your images and videos. Now you will create four different Cloud Storage buckets.

1. Create a bucket for storing your uploaded images and video files using the IV_BUCKET_NAME environment variable:

gsutil mb gs://${IV_BUCKET_NAME}

2. Create a bucket for storing your filtered image and video files using the FILTERED_BUCKET_NAME environment variable:

gsutil mb gs://${FILTERED_BUCKET_NAME}

3. Create a bucket for storing your flagged image and video files using the FLAGGED_BUCKET_NAME environment variable:

gsutil mb gs://${FLAGGED_BUCKET_NAME}

4. Create a bucket for your Cloud Functions to use as a staging location using the STAGING_BUCKET_NAME environment variable:

gsutil mb gs://${STAGING_BUCKET_NAME}

5. Check that the four storage buckets have been created:

gsutil ls

You should see the names of the four storage buckets listed in the output. These will be be in the format [PROJECT-ID]-upload, -filtered, -flagged, and -staging.

Click Check my progress to verify the objective. Creating Cloud Storage buckets

Task 3. Creating Cloud Pub/Sub topics

Cloud Pub/Sub topics is used for Cloud Storage notification messages and for messages between your Cloud Functions. This lab has some of the topic names preset to specific defaults which are used in this section for the topic names.

Note: You can change the values in this section to any other valid Cloud Pub/Sub topic name, but if you do you must also make those changes in the config.json file that is downloaded later as part of the solution.

1. Create a topic to receive Cloud Storage notifications whenever one of your files is uploaded to Cloud Storage. You set the default value to upload_notification and save it in an environment variable since it will be used later:

export UPLOAD_NOTIFICATION_TOPIC=upload_notification gcloud pubsub topics create ${UPLOAD_NOTIFICATION_TOPIC}

2. Create a topic to receive your messages from the Vision API. The default value in the config.json file is visionapiservice:

gcloud pubsub topics create visionapiservice

3. Next, create a topic to receive your messages from the Video Intelligence API. The default value in the config.json file is videointelligenceservice:

gcloud pubsub topics create videointelligenceservice

4. Create a topic to receive your messages to store in BigQuery. The default value in the config.json file is bqinsert:

gcloud pubsub topics create bqinsert

5. Check that the four pubsub topics have been created:

gcloud pubsub topics list

You should see the names of the four topics listed in the output: upload_notification, visionapiservice, videointelligenceservice and bqinsert.

Click Check my progress to verify the objective. Creating Cloud Pub/Sub topics

Task 4. Creating Cloud Storage notifications

1. Create a notification that is triggered only when one of your new objects is placed in the Cloud Storage file upload bucket:

gsutil notification create -t upload_notification -f json -e OBJECT_FINALIZE gs://${IV_BUCKET_NAME}

2. Confirm that your notification has been created for the bucket:

gsutil notification list gs://${IV_BUCKET_NAME}

You'll see this output, if the function succeeds:

Filters: Event Types: OBJECT_FINALIZE

Click Check my progress to verify the objective. Creating Cloud Storage notifications

Task 5. Preparing the Cloud functions for deployment

The code for the cloud functions used in this lab are available in a public Cloud Storage bucket, defined in the index.js file.

1. Download the code from the bucket using the following command:

gsutil -m cp -r gs://spls/gsp138/cloud-functions-intelligentcontent-nodejs .

2. Change directory to the application directory:

cd cloud-functions-intelligentcontent-nodejs

You can examine the source in detail by opening index.js with the editor of your choice to see how each of the functions is implemented.

Create the BigQuery dataset and table

The results of the Vision and Video Intelligence APIs are stored in BigQuery. The demo solution used in this Qwiklab has default dataset and table names set to intelligentcontentfilter and filtered_content. You can change these values, but if you do you must also make those changes in the config.json file that is downloaded later as part of the solution.

1. Create your BigQuery dataset:

export DATASET_ID=intelligentcontentfilter export TABLE_NAME=filtered_content bq --project_id ${PROJECT_ID} mk ${DATASET_ID}

The dataset name is set to intelligentcontentfilter to match the default value in the config.json file.

Now you'll create your BigQuery table from the schema file that is included with the lab. The dataset and table name is set to filtered_content to match the default values in the config.json file and the schema is defined in the file intelligent_content_bq_schema.json.

2. Run the following to create the BigQuery table:

bq --project_id ${PROJECT_ID} mk --schema intelligent_content_bq_schema.json -t ${DATASET_ID}.${TABLE_NAME}

3. Verify that your BigQuery table has been created by running:

bq --project_id ${PROJECT_ID} show ${DATASET_ID}.${TABLE_NAME}

Resulting output should contain the following:

Last modified Schema ----------------- --------------------------------------- 08 Feb 19:22:43 |- gcsUrl: string (required) |- contentUrl: string (required) |- contentType: string (required) |- insertTimestamp: timestamp (required) +- labels: record (repeated) | |- name: string +- safeSearch: record (repeated) | |- flaggedType: string | |- likelihood: string

Click Check my progress to verify the objective. Create the BigQuery dataset and table

Edit your JSON configuration file

Before you can deploy the cloud functions defined in the source code, you must modify the config.json file to use your specific Cloud Storage buckets, Cloud Pub/Sub topic names, and BigQuery dataset ID and table name.

• Enter these sed commands in the Google Cloud shell to make the changes for you:

sed -i "s/\[PROJECT-ID\]/$PROJECT_ID/g" config.json sed -i "s/\[FLAGGED_BUCKET_NAME\]/$FLAGGED_BUCKET_NAME/g" config.json sed -i "s/\[FILTERED_BUCKET_NAME\]/$FILTERED_BUCKET_NAME/g" config.json sed -i "s/\[DATASET_ID\]/$DATASET_ID/g" config.json sed -i "s/\[TABLE_NAME\]/$TABLE_NAME/g" config.json Note: Alternative method: You can manually edit the config.json file to replace the placeholders for [PROJECT-ID], [FLAGGED_BUCKET_NAME], [FILTERED_BUCKET_NAME], [DATASET_ID] and [TABLE_NAME] that you can see here with the appropriate values. { "VISION_TOPIC": "projects/[PROJECT-ID]/topics/visionapiservice", "VIDEOINTELLIGENCE_TOPIC": "projects/[PROJECT-ID]/topics/videointelligenceservice", "BIGQUERY_TOPIC": "projects/[PROJECT-ID]/topics/bqinsert", "REJECTED_BUCKET": "[FLAGGED_BUCKET_NAME]", "RESULT_BUCKET": "[FILTERED_BUCKET_NAME]", "DATASET_ID": "[DATASET_ID]", "TABLE_NAME": "[TABLE_NAME]", "GCS_AUTH_BROWSER_URL_BASE": "https://storage.cloud.google.com/" , "API_Constants": { "ADULT" : "adult", "VIOLENCE" : "violence", "SPOOF" : "spoof", "MEDICAL" : "medical" } } Note: [FLAGGED_BUCKET_NAME] and [FILTERED_BUCKET_NAME] here must not include the leading gs:// prefix.

Task 6. Deploying the Cloud functions

The code for the cloud functions used in this lab are available on GitHub, defined in the index.js file. You can examine the source in detail on Github to see how each of the functions is implemented. The deployments can each take a few minutes to complete.

Deploy the GCStoPubsub function

Next you will deploy the GCStoPubsub Cloud Function, which contains the logic to receive a Cloud Storage notification message from Cloud Pub/Sub and forward the message to the appropriate function with another Cloud Pub/Sub message.

1. Run the following:

gcloud functions deploy GCStoPubsub --runtime nodejs10 --stage-bucket gs://${STAGING_BUCKET_NAME} --trigger-topic ${UPLOAD_NOTIFICATION_TOPIC} --entry-point GCStoPubsub

2. Type "Y" if asked Allow unauthenticated invocations of new function [GCStoPubsub]?

The command-line output is similar to the following for each of the four Cloud Functions:

Deploying function (may take a while - up to 2 minutes)...done. availableMemoryMb: 256 entryPoint: GCStoPubsub eventTrigger: eventType: providers/cloud.pubsub/eventTypes/topic.publish failurePolicy: {} resource: projects/qwiklabs-gcp-8bbedd7d3dd97468/topics/qwiklabs-gcp-8bbedd7d3dd97468-upload service: pubsub.googleapis.com labels: deployment-tool: cli-gcloud name: projects/qwiklabs-gcp-8bbedd7d3dd97468/locations/us-central1/functions/GCStoPubsub serviceAccountEmail: qwiklabs-gcp-8bbedd7d3dd97468@appspot.gserviceaccount.com sourceArchiveUrl: gs://qwiklabs-gcp-8bbedd7d3dd97468-staging/us-central1-projects/qwiklabs-gcp-8bbedd7d3dd97468/locations/us-central1/functions/GCStoPubsub-xeejkketibhf.zip status: ACTIVE timeout: 60s updateTime: '2018-02-08T21:39:42Z' versionId: '1'

Deploy the visionAPI function

Deploy your visionAPI Cloud Function, which contains the logic to receive a message with Cloud Pub/Sub, call the Vision API, and forward the message to the insertIntoBigQuery Cloud Function with another Cloud Pub/Sub message. If you chose to use a different Vision API topic name then change that name here as well.

1. Run the following:

gcloud functions deploy visionAPI --runtime nodejs10 --stage-bucket gs://${STAGING_BUCKET_NAME} --trigger-topic visionapiservice --entry-point visionAPI

2. Type "Y" if asked to Allow unauthenticated invocations of new function [GCStoPubsub]?.

Deploy the videoIntelligenceAPI function

Deploy your videoIntelligenceAPI Cloud Function, which contains the logic to receive a message with Cloud Pub/Sub, call the Video Intelligence API, and forward the message to the insertIntoBigQuery Cloud Function with another Cloud Pub/Sub message. If you chose to use a different Video Intelligence API topic name then change that name here as well.

1. Run the following:

gcloud functions deploy videoIntelligenceAPI --runtime nodejs10 --stage-bucket gs://${STAGING_BUCKET_NAME} --trigger-topic videointelligenceservice --entry-point videoIntelligenceAPI --timeout 540

2. Type "Y" when asked to Allow unauthenticated invocations of new function [videoIntelligenceAPI]?

Deploy the insertIntoBigQuery function

Deploy your insertIntoBigQuery Cloud Function, which contains the logic to receive a message with Cloud Pub/Sub and call the BigQuery API to insert the data into your BigQuery table. If you chose to use a different BigQuery topic name then change that name here as well.

1. Run the following:

gcloud functions deploy insertIntoBigQuery --runtime nodejs10 --stage-bucket gs://${STAGING_BUCKET_NAME} --trigger-topic bqinsert --entry-point insertIntoBigQuery

2. Type "Y" when asked Allow unauthenticated invocations of new function [insertIntoBigQuery]?

Confirm that the Cloud Functions have been deployed

• Run the following:

gcloud beta functions list

You should see the names of the four, cloud functions listed in the output: GCStoPubsub, visionAPI, videoIntelligenceAPI and insertintobigquery.

Click Check my progress to verify the objective. Deploying the Cloud Functions

Task 7. Testing the flow

The following diagram outlines the processing flow:

You test the process by uploading your files to Cloud Storage, checking your logs, and viewing your results in BigQuery.

Upload an image and a video file to the upload storage bucket

1. On the Navigation menu (), click Cloud Storage > Buckets.

2. Click the name of the bucket with the -upload suffix and then click Upload Files.

3. Upload some image files and/or video files from your local machine.

Monitor log activity

Switch back to the Google Cloud Shell to verify that your Cloud Functions were triggered and ran successfully by viewing the Cloud Functions logs captured in Cloud Logging:

1. Run the following to test GCStoPubsub:

gcloud beta functions logs read --filter "finished with status" "GCStoPubsub" --limit 100

This command may take a minute or two to complete.

Resulting output:

LEVEL NAME EXECUTION_ID TIME_UTC LOG D GCStoPubsub 585471060278981 2019-06-12 00:12:32.695 Function execution took 2422 ms, finished with status: 'ok'

You will also notice that your uploaded image has been moved to the next bucket as well.

Note: If you get "Listed 0 items." in the output, wait a second and try running the command again.

2. Run the following to test insertIntoBigQuery:

gcloud beta functions logs read --filter "finished with status" "insertIntoBigQuery" --limit 100

Resulting output:

LEVEL NAME EXECUTION_ID TIME_UTC LOG D insertIntoBigQuery 578335797826253 2019-06-12 00:12:45.103 Function execution took 641 ms, finished with status: 'ok'

View results in BigQuery

To see your results in BigQuery, you'll create SQL commands to query BigQuery.

1. Run the following, replacing [PROJECT_ID], [DATASET_ID], and [TABLE_NAME] with your project ID, dataset ID, and BigQuery table name if you found out that variables created for above doesn't contain correct value:

echo " #standardSql SELECT insertTimestamp, contentUrl, flattenedSafeSearch.flaggedType, flattenedSafeSearch.likelihood FROM \`$PROJECT_ID.$DATASET_ID.$TABLE_NAME\` CROSS JOIN UNNEST(safeSearch) AS flattenedSafeSearch ORDER BY insertTimestamp DESC, contentUrl, flattenedSafeSearch.flaggedType LIMIT 1000 " > sql.txt

2. View your BigQuery results with the following command. Replace [PROJECT_ID]with your project ID:

bq --project_id ${PROJECT_ID} query < sql.txt

Resulting output:

Note: Alternate method

1. Sign in to BigQuery and run your queries.

2. Click on BigQuery in the Console left menu.

3. Enter the following SQL into the New Query Editor. Replace [PROJECT_ID], [DATASET_ID], and [TABLE_NAME] with your project ID, dataset ID, and BigQuery table name.

#standardSql
SELECT insertTimestamp,
contentUrl,
flattenedSafeSearch.flaggedType,
flattenedSafeSearch.likelihood
FROM
`[PROJECT_ID].[DATASET_ID].[TABLE_NAME]`
CROSS JOIN UNNEST(safeSearch) AS flattenedSafeSearch
ORDER BY insertTimestamp DESC,
contentUrl,
flattenedSafeSearch.flaggedType
LIMIT
1000

The following example shows what this SQL looks like in the UI:

Resulting output:

Click Check my progress to verify the objective. Testing the flow

Congratulations!

You have now successfully completed the Scanning User-generated Content Using the Cloud Video Intelligence and Cloud Vision APIs Qwiklab.

Finish your quest

Continue your quest with Google Cloud Solutions II: Data and Machine Learning. A quest is a series of related labs that form a learning path. Completing this quest earns you a badge to recognize your achievement. You can make your badge or badges public and link to them in your online resume or social media account. Enroll in this quest and get immediate completion credit. Refer to the Google Cloud Skills Boost catalog for all available quests.

Next steps / Learn more

• Learn more about the Video Intelligence API.
• Learn more about the Vision API.
• Learn more about Cloud Functions.

Google Cloud training and certification

...helps you make the most of Google Cloud technologies. Our classes include technical skills and best practices to help you get up to speed quickly and continue your learning journey. We offer fundamental to advanced level training, with on-demand, live, and virtual options to suit your busy schedule. Certifications help you validate and prove your skill and expertise in Google Cloud technologies.

Manual Last Updated September 15, 2023

Lab Last Tested September 15, 2023

Copyright 2023 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.