arrow_back

Creating an Object Detection Application Using TensorFlow

Join Sign in

Creating an Object Detection Application Using TensorFlow

45 minutes 5 Credits

GSP141

Google Cloud Self-Paced Labs

This lab will show you how to install and run an object detection application. The application uses TensorFlow and other public API libraries to detect multiple objects in an uploaded image.

Objectives

This is a basic lab designed to familiarize you with TensorFlow applications. When you are finished, you should be able to:

  • Create a virtual machine (VM) using Compute Engine.

  • Install the Object Detection API library.

  • Install and launch an object detection web application.

  • Test the web application with uploaded images.

Setup and Requirements

Requirements

You will need a image file (JPEG, JPG, or PNG) to upload to the lab to analyze.

Qwiklabs setup

Before you click the Start Lab button

Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources will be made available to you.

This hands-on lab lets you do the lab activities yourself in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab.

What you need

To complete this lab, you need:

  • Access to a standard internet browser (Chrome browser recommended).
  • Time to complete the lab.

Note: If you already have your own personal Google Cloud account or project, do not use it for this lab.

Note: If you are using a Chrome OS device, open an Incognito window to run this lab.

How to start your lab and sign in to the Google Cloud Console

  1. Click the Start Lab button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is a panel populated with the temporary credentials that you must use for this lab.

    Open Google Console

  2. Copy the username, and then click Open Google Console. The lab spins up resources, and then opens another tab that shows the Sign in page.

    Sign in

    Tip: Open the tabs in separate windows, side-by-side.

  3. In the Sign in page, paste the username that you copied from the left panel. Then copy and paste the password.

    Important: You must use the credentials from the left panel. Do not use your Google Cloud Training credentials. If you have your own Google Cloud account, do not use it for this lab (avoids incurring charges).

  4. Click through the subsequent pages:

    • Accept the terms and conditions.
    • Do not add recovery options or two-factor authentication (because this is a temporary account).
    • Do not sign up for free trials.

After a few moments, the Cloud Console opens in this tab.

Activate Cloud Shell

Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.

In the Cloud Console, in the top right toolbar, click the Activate Cloud Shell button.

Cloud Shell icon

Click Continue.

cloudshell_continue.png

It takes a few moments to provision and connect to the environment. When you are connected, you are already authenticated, and the project is set to your PROJECT_ID. For example:

Cloud Shell Terminal

gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.

You can list the active account name with this command:

gcloud auth list

(Output)

Credentialed accounts: - <myaccount>@<mydomain>.com (active)

(Example output)

Credentialed accounts: - google1623327_student@qwiklabs.net

You can list the project ID with this command:

gcloud config list project

(Output)

[core] project = <project_ID>

(Example output)

[core] project = qwiklabs-gcp-44776a13dea667a6

TensorFlow architecture overview

The object detection application uses the following components:

  • TensorFlow 2.x: An open source machine learning library developed by researchers and engineers within Google's Machine Intelligence research organization. TensorFlow 2.0 runs on multiple computers to distribute the training workloads.

  • Object Detection API: An open source framework built on top of TensorFlow 2.x that makes it easy to construct, train, and deploy object detection models.

  • Pre-trained object detection models: The Object Detection API provides pre-trained object detection models for users running inference jobs. Users are not required to train models from scratch.

Local implementation

The following diagram shows how this Qwiklab is implemented. The web application is deployed to a VM instance running on Compute Engine.

app_flow.png

When the client uploads an image to the application, the application runs the inference job locally. The pre-trained model returns the labels of detected objects, and the image coordinates of the corresponding objects. Using these values, the application generates new images populated with rectangles around the detected objects. Separate images are generated for each object category, allowing the client to discriminate between selected objects.

Remote implementation

You can deploy the pre-trained model on Google Cloud Machine Learning Engine to provide an API service for inference. If you do, the web application sends an API request to detect objects in the uploaded image, instead of running the inference job locally.

TensorFlow 2.x allows you to choose which platform to execute inference jobs on depending on your business needs. This flexibility shows the advantage of Google Cloud and TensorFlow 2.x as an open platform for machine learning.

Pre-trained models

You can use five pre-trained models with the Object Detection API. They are trained with the COCO dataset and are capable of detecting general objects in 80 categories.

The COCO mAP column shows the model's accuracy index. Higher numbers indicate better accuracy. As speed increases, accuracy decreases.

Model name

Speed

COCO mAP

ssd_mobilenet_v1_coco

fast

21

ssd_inception_v2_coco

fast

24

rfcn_resnet101_coco

medium

30

faster_rcnn_resnet101_coco

medium

32

faster_rcnn_inception_resnet_v2_atrous_coco

slow

37

Launch a VM instance

Open the navigation menu and select Compute Engine > VM Instances to go to the VM Instances page.

Click the Create button.

Create your VM with these settings:

  • In the Machine type section, click Custom from the dropdown.
  • Cores - replace 2 with 4.
  • Memory - replace 16 with 8.
  • In the Firewall section, select Allow HTTP traffic.

Click the Management, disks, networking, SSH keys dropdown, then click the Networking tab.

  • Click the pencil icon next to the default row in the Network interfaces section.
  • In the External IP dropdown, select Create IP address from to assign a static IP address.
  • Type staticip in the Name field, then click Reserve.

Click the Create button to create the instance and wait for the instance to be created.

Click Check my progress to verify the objective. Launch a VM instance

SSH into the instance

Next, click the SSH button for your instance to open an SSH terminal.

In the SSH shell, enter the following command to switch to the root user:

sudo -i

Install the Object Detection API library

Install the prerequisite packages:

apt-get update apt-get install -y protobuf-compiler python3-pil python3-lxml python3-pip python3-dev git pip3 install --upgrade pip pip3 install Flask==1.1.1 WTForms==2.2.1 Flask_WTF==0.14.2 Werkzeug==0.16.0

Install TensorFlow:

pip3 install tensorflow

Install the Object Detection API library:

cd /opt git clone https://github.com/tensorflow/models cd models/research protoc object_detection/protos/*.proto --python_out=.

Download the pre-trained model binaries by running the following commands:

mkdir -p /opt/graph_def cd /tmp for model in \ ssd_mobilenet_v1_coco_11_06_2017 \ ssd_inception_v2_coco_11_06_2017 \ rfcn_resnet101_coco_11_06_2017 \ faster_rcnn_resnet101_coco_11_06_2017 \ faster_rcnn_inception_resnet_v2_atrous_coco_11_06_2017 do \ curl -OL http://download.tensorflow.org/models/object_detection/$model.tar.gz tar -xzf $model.tar.gz $model/frozen_inference_graph.pb cp -a $model /opt/graph_def/ done

Now you will choose a model for the web application to use. For this lab, enter the following command faster_rcnn_resnet101_coco_11_06_2017:

ln -sf /opt/graph_def/faster_rcnn_resnet101_coco_11_06_2017/frozen_inference_graph.pb /opt/graph_def/frozen_inference_graph.pb

Install and launch the web application

Change to the root home directory:

cd $HOME

Clone the example application from Github:

git clone https://github.com/GoogleCloudPlatform/tensorflow-object-detection-example

Install the application:

cp -a tensorflow-object-detection-example/object_detection_app_p3 /opt/ chmod u+x /opt/object_detection_app_p3/app.py

Create the object detection service:

cp /opt/object_detection_app_p3/object-detection.service /etc/systemd/system/

Reload systemd to reload the systemd manager configuration:

systemctl daemon-reload

Launch the application.

systemctl enable object-detection systemctl start object-detection systemctl status object-detection

The last command outputs the application status, as in the following example:

object-detection.service - Object Detection API Demo Loaded: loaded (/opt/object_detection_app/object-detection.service; linked) Active: active (running) since Wed 2017-06-21 05:34:10 UTC; 22s ago Process: 7122 ExecStop=/bin/kill -TERM $MAINPID (code=exited, status=0/SUCCESS) Main PID: 7125 (app.py) CGroup: /system.slice/object-detection.service └─7125 /usr/bin/python /opt/object_detection_app/app.py Jun 21 05:34:10 object-detection systemd[1]: Started Object Detection API Demo. Jun 21 05:34:26 object-detection app.py[7125]: 2017-06-2105:34:26.518736: W tensorflow/core/platform/cpu_fe...ons. Jun 21 05:34:26 object-detection app.py[7125]: 2017-06-2105:34:26.518790: W tensorflow/core/platform/cpu_fe...ons. Jun 21 05:34:26 object-detection app.py[7125]: 2017-06-2105:34:26.518795: W tensorflow/core/platform/cpu_fe...ons. Jun 21 05:34:26 object-detection app.py[7125]: * Running on http://0.0.0.0:80/ (Press CTRL+C to quit) Hint: Some lines were ellipsized, use -l to show in full

If you are not getting the output or the command hangs on a line like this:

instance-1 app.py[10937]: [2.0K blob data]

Hold CTRL + C and then re re-run the command.

The application loads the model binary immediately after launch. It will take a minute to start serving requests from clients. You'll see the message Running on http://0.0.0.0:80/ (Press CTRL+C to quit) when it's ready.

Click Check my progress to verify the objective. Install and launch the web application

Test the web application

Return to the Google Cloud console tab.

Click the external ip-address link next to your VM instance to open a browser window to the example application.

Log in with the following credentials:

  • Username - username
  • Password - passw0rd

Click Choose File and select an image file from your computer to analyse.

Click Upload to test.

When you upload an image file with a JPEG, JPG, or PNG extension, the application shows the result of the object detection inference, as shown in the following image. The inference might take up to 30 seconds, depending on the image.

result_img.png

The object names detected by the model are shown to the right of the image, in the application window. Click an object name to display rectangles surrounding the corresponding objects in the image. The rectangle thickness increases with object identification confidence.

In the above image, "fork", "cup", "dining table", "person", and "knife", are detected. After clicking "cup", rectangles display around all detected cups in the image. Click "original" to see the original image.

Optional: If you have time, test this model's accuracy by uploading images that contain different types of objects.

Change the inference model

Go back to the SSH console and run the following to change the inference model by running the following:

systemctl stop object-detection

Replace [MODEL NAME] with one of the following options.

  • ssd_mobilenet_v1_coco_11_06_2017

  • ssd_inception_v2_coco_11_06_2017

  • rfcn_resnet101_coco_11_06_2017

  • faster_rcnn_resnet101_coco_11_06_2017

  • faster_rcnn_inception_resnet_v2_atrous_coco_11_06_2017

ln -sf /opt/graph_def/[MODEL NAME]/frozen_inference_graph.pb /opt/graph_def/frozen_inference_graph.pb

Restart the application:

systemctl start object-detection

Check that the application has restarted successfully:

systemctl status object-detection

Once the application has successfully re-started with the updated model, retest some of your images to see how change affects object detection.

Congratulations!

You have now completed the Creating an Object Detection Application Using TensorFlow.

Finish Your Quest

solution2_quest_icon.png ML-TensorFlow-on-GCP-badge.png

This self-paced lab is part of the Qwiklabs Google Cloud Solutions II: Data and Machine Learning and Intermediate ML: TensorFlow on Google Cloud Quests. A Quest is a series of related labs that form a learning path. Completing a Quest earns you a badge to recognize your achievement. You can make your badge (or badges) public and link to them in your online resume or social media account. Enroll in the above Quests and get immediate completion credit if you've taken this lab. See other available Qwiklabs Quests.

Take your next lab

Continue your quest with Using OpenTSDB to monitor Time-Series Data on Cloud Platform, or check out these suggestions:

Next steps / learn more

Google Cloud Training & Certification

...helps you make the most of Google Cloud technologies. Our classes include technical skills and best practices to help you get up to speed quickly and continue your learning journey. We offer fundamental to advanced level training, with on-demand, live, and virtual options to suit your busy schedule. Certifications help you validate and prove your skill and expertise in Google Cloud technologies.

Manual Last Updated June 25, 2021
Lab Last Tested June 25, 2021

Copyright 2022 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.