arrow_back

Creating an Object Detection Application Using TensorFlow

Join Sign in

Creating an Object Detection Application Using TensorFlow

45 minutes 1 Credit

GSP141

Google Cloud self-paced labs logo

This lab will show you how to install and run an object detection application. The application uses TensorFlow and other public API libraries to detect multiple objects in an uploaded image.

Objectives

This is a basic lab designed to familiarize you with TensorFlow applications. When you are finished, you should be able to:

  • Create a virtual machine (VM) using Compute Engine.

  • Install the Object Detection API library.

  • Install and launch an object detection web application.

  • Test the web application with uploaded images.

Setup and requirements

You will need a image file (JPEG, JPG, or PNG) to upload to the lab to analyze.

Before you click the Start Lab button

Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources will be made available to you.

This hands-on lab lets you do the lab activities yourself in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab.

To complete this lab, you need:

  • Access to a standard internet browser (Chrome browser recommended).
Note: Use an Incognito or private browser window to run this lab. This prevents any conflicts between your personal account and the Student account, which may cause extra charges incurred to your personal account.
  • Time to complete the lab---remember, once you start, you cannot pause a lab.
Note: If you already have your own personal Google Cloud account or project, do not use it for this lab to avoid extra charges to your account.

How to start your lab and sign in to the Google Cloud Console

  1. Click the Start Lab button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is the Lab Details panel with the following:

    • The Open Google Console button
    • Time remaining
    • The temporary credentials that you must use for this lab
    • Other information, if needed, to step through this lab
  2. Click Open Google Console. The lab spins up resources, and then opens another tab that shows the Sign in page.

    Tip: Arrange the tabs in separate windows, side-by-side.

    Note: If you see the Choose an account dialog, click Use Another Account.
  3. If necessary, copy the Username from the Lab Details panel and paste it into the Sign in dialog. Click Next.

  4. Copy the Password from the Lab Details panel and paste it into the Welcome dialog. Click Next.

    Important: You must use the credentials from the left panel. Do not use your Google Cloud Skills Boost credentials. Note: Using your own Google Cloud account for this lab may incur extra charges.
  5. Click through the subsequent pages:

    • Accept the terms and conditions.
    • Do not add recovery options or two-factor authentication (because this is a temporary account).
    • Do not sign up for free trials.

After a few moments, the Cloud Console opens in this tab.

Note: You can view the menu with a list of Google Cloud Products and Services by clicking the Navigation menu at the top-left. Navigation menu icon

Activate Cloud Shell

Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.

  1. Click Activate Cloud Shell Activate Cloud Shell icon at the top of the Google Cloud console.

  2. Click Continue.

It takes a few moments to provision and connect to the environment. When you are connected, you are already authenticated, and the project is set to your PROJECT_ID. The output contains a line that declares the PROJECT_ID for this session:

Your Cloud Platform project in this session is set to YOUR_PROJECT_ID

gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.

  1. (Optional) You can list the active account name with this command:

gcloud auth list

Output:

ACTIVE: * ACCOUNT: student-01-xxxxxxxxxxxx@qwiklabs.net To set the active account, run: $ gcloud config set account `ACCOUNT`
  1. (Optional) You can list the project ID with this command:

gcloud config list project

Output:

[core] project = <project_ID>

Example output:

[core] project = qwiklabs-gcp-44776a13dea667a6 Note: For full documentation of gcloud, in Google Cloud, refer to the gcloud CLI overview guide.

TensorFlow architecture overview

The object detection application uses the following components:

  • TensorFlow 2.x: An open source machine learning library developed by researchers and engineers within Google's Machine Intelligence research organization. TensorFlow 2.0 runs on multiple computers to distribute the training workloads.

  • Object Detection API: An open source framework built on top of TensorFlow 2.x that makes it easy to construct, train, and deploy object detection models.

  • Pre-trained object detection models: The Object Detection API provides pre-trained object detection models for users running inference jobs. Users are not required to train models from scratch.

Local implementation

The following diagram shows how this Qwiklab is implemented. The web application is deployed to a VM instance running on Compute Engine.

The upload image and detection result flow between a laptop and web application

When the client uploads an image to the application, the application runs the inference job locally. The pre-trained model returns the labels of detected objects, and the image coordinates of the corresponding objects. Using these values, the application generates new images populated with rectangles around the detected objects. Separate images are generated for each object category, allowing the client to discriminate between selected objects.

Remote implementation

You can deploy the pre-trained model on Google Cloud Machine Learning Engine to provide an API service for inference. If you do, the web application sends an API request to detect objects in the uploaded image, instead of running the inference job locally.

TensorFlow 2.x allows you to choose which platform to execute inference jobs on depending on your business needs. This flexibility shows the advantage of Google Cloud and TensorFlow 2.x as an open platform for machine learning.

Pre-trained models

You can use five pre-trained models with the Object Detection API. They are trained with the COCO dataset and are capable of detecting general objects in 80 categories.

The COCO mAP column shows the model's accuracy index. Higher numbers indicate better accuracy. As speed increases, accuracy decreases.

Model name

Speed

COCO mAP

ssd_mobilenet_v1_coco

fast

21

ssd_inception_v2_coco

fast

24

rfcn_resnet101_coco

medium

30

faster_rcnn_resnet101_coco

medium

32

faster_rcnn_inception_resnet_v2_atrous_coco

slow

37

Task 1. Launch a VM instance

  1. Open the navigation menu and select Compute Engine > VM Instances to go to the VM Instances page.

  2. Click the Create Instance button.

  3. Create your VM with these settings:

  • In the Machine type section, click Custom from the dropdown.

  • Cores - replace 2 with 4.

  • Memory - replace 16 with 8.

  • In the Firewall section, select Allow HTTP traffic.

  1. Click the Networking, disks, security, management, sole-tenancy dropdown, then click the Networking tab.

  2. On the Networking tabbed page:

  • Click on the default row in the Network interfaces section.

  • In the External IP dropdown, select Create IP address from to assign a static IP address.

  • Type staticip in the Name field, then click Reserve.

  1. Click the Create button to create the instance and wait for the instance to be created.

Click Check my progress to verify the objective. Launch a VM instance

Task 2. SSH into the instance

  1. Next, click the SSH button for your instance to open an SSH terminal.

  2. In the SSH shell, enter the following command to switch to the root user:

sudo -i

Task 3. Install the Object Detection API library

  1. Install the prerequisite packages:

apt-get update apt-get install -y protobuf-compiler python3-pil python3-lxml python3-pip python3-dev git pip3 install --upgrade pip pip3 install Flask==2.1.2 WTForms==3.0.1 Flask_WTF==1.0.1 Werkzeug==2.0.3 itsdangerous==2.1.2 jinja2==3.1.2 protobuf~=3.19.0
  1. Install TensorFlow:

pip3 install tensorflow==2.9.0
  1. Install the Object Detection API library:

cd /opt git clone https://github.com/tensorflow/models cd models/research protoc object_detection/protos/*.proto --python_out=.
  1. Download the pre-trained model binaries by running the following commands:

mkdir -p /opt/graph_def cd /tmp for model in \ ssd_mobilenet_v1_coco_11_06_2017 \ ssd_inception_v2_coco_11_06_2017 \ rfcn_resnet101_coco_11_06_2017 \ faster_rcnn_resnet101_coco_11_06_2017 \ faster_rcnn_inception_resnet_v2_atrous_coco_11_06_2017 do \ curl -OL http://download.tensorflow.org/models/object_detection/$model.tar.gz tar -xzf $model.tar.gz $model/frozen_inference_graph.pb cp -a $model /opt/graph_def/ done

Now you will choose a model for the web application to use.

  1. For this lab, enter the following command faster_rcnn_resnet101_coco_11_06_2017:

ln -sf /opt/graph_def/faster_rcnn_resnet101_coco_11_06_2017/frozen_inference_graph.pb /opt/graph_def/frozen_inference_graph.pb

Task 4. Install and launch the web application

1.Change to the root home directory:

cd $HOME
  1. Clone the example application from Github:

git clone https://github.com/GoogleCloudPlatform/tensorflow-object-detection-example
  1. Install the application:

cp -a tensorflow-object-detection-example/object_detection_app_p3 /opt/ chmod u+x /opt/object_detection_app_p3/app.py
  1. Create the object detection service:

cp /opt/object_detection_app_p3/object-detection.service /etc/systemd/system/
  1. Reload systemd to reload the systemd manager configuration:

systemctl daemon-reload Note: The application provides a simple user authentication mechanism. You can change the default username and password by modifying the /opt/object_detection_app/decorator.py file and changing the following lines: USERNAME = 'username' PASSWORD = 'passw0rd'
  1. Launch the application:

systemctl enable object-detection systemctl start object-detection systemctl status object-detection

The last command outputs the application status, as in the following example:

object-detection.service - Object Detection API Demo Loaded: loaded (/opt/object_detection_app/object-detection.service; linked) Active: active (running) since Wed 2017-06-21 05:34:10 UTC; 22s ago Process: 7122 ExecStop=/bin/kill -TERM $MAINPID (code=exited, status=0/SUCCESS) Main PID: 7125 (app.py) CGroup: /system.slice/object-detection.service └─7125 /usr/bin/python /opt/object_detection_app/app.py Jun 21 05:34:10 object-detection systemd[1]: Started Object Detection API Demo. Jun 21 05:34:26 object-detection app.py[7125]: 2017-06-2105:34:26.518736: W tensorflow/core/platform/cpu_fe...ons. Jun 21 05:34:26 object-detection app.py[7125]: 2017-06-2105:34:26.518790: W tensorflow/core/platform/cpu_fe...ons. Jun 21 05:34:26 object-detection app.py[7125]: 2017-06-2105:34:26.518795: W tensorflow/core/platform/cpu_fe...ons. Jun 21 05:34:26 object-detection app.py[7125]: * Running on http://0.0.0.0:80/ (Press CTRL+C to quit) Hint: Some lines were ellipsized, use -l to show in full
  1. If you are not getting the output or the command hangs on a line like this:
instance-1 app.py[10937]: [2.0K blob data]

Hold CTRL + C and then re re-run the command.

The application loads the model binary immediately after launch. It will take a minute to start serving requests from clients. You'll see the message Running on http://0.0.0.0:80/ (Press CTRL+C to quit) when it's ready.

Note: Do not open the URL http://0.0.0.0:80/! You will learn how to test the application in the next section.

Click Check my progress to verify the objective. Install and launch the web application

Task 5. Test the web application

  1. Return to the Google Cloud console tab.

  2. Click the external ip-address link next to your VM instance to open a browser window to the example application.

  3. Log in with the following credentials:

  • Username - username

  • Password - passw0rd

  1. Click Choose File and select an image file from your computer to analyse.

  2. Click Upload to test.

When you upload an image file with a JPEG, JPG, or PNG extension, the application shows the result of the object detection inference, as shown in the following image. The inference might take up to 30 seconds, depending on the image.

An image of someone at a dining table, wherein the coffee cups are highlighted on the table

The object names detected by the model are shown to the right of the image, in the application window.

  1. Click an object name to display rectangles surrounding the corresponding objects in the image. The rectangle thickness increases with object identification confidence.

In the above image, "fork", "cup", "dining table", "person", and "knife", are detected.

After clicking "cup", rectangles display around all detected cups in the image.

  1. Click "original" to see the original image.

  2. Optional: If you have time, test this model's accuracy by uploading images that contain different types of objects.

Task 6. Change the inference model

  1. Go back to the SSH console and run the following to change the inference model by running the following:

systemctl stop object-detection
  1. Replace [MODEL NAME] with one of the following options:

  • ssd_mobilenet_v1_coco_11_06_2017

  • ssd_inception_v2_coco_11_06_2017

  • rfcn_resnet101_coco_11_06_2017

  • faster_rcnn_resnet101_coco_11_06_2017

  • faster_rcnn_inception_resnet_v2_atrous_coco_11_06_2017

ln -sf /opt/graph_def/[MODEL NAME]/frozen_inference_graph.pb /opt/graph_def/frozen_inference_graph.pb
  1. Restart the application:

systemctl start object-detection
  1. Check that the application has restarted successfully:

systemctl status object-detection
  1. Once the application has successfully re-started with the updated model, retest some of your images to see how change affects object detection.

Congratulations!

You have now completed the Creating an Object Detection Application Using TensorFlow.

Finish your quest

This self-paced lab is part of the Google Cloud Solutions II: Data and Machine Learning and Intermediate ML: TensorFlow on Google Cloud quests. A quest is a series of related labs that form a learning path. Completing a quest earns you a badge to recognize your achievement. You can make your badge or badges public and link to them in your online resume or social media account. Enroll in any quest that contains this lab and get immediate completion credit. Refer to the Google Cloud Skills Boost catalog for all available quests.

Take your next lab

Continue your quest with Using OpenTSDB to monitor Time-Series Data on Cloud Platform, or check out these suggestions:

Next steps / Learn more

Google Cloud training and certification

...helps you make the most of Google Cloud technologies. Our classes include technical skills and best practices to help you get up to speed quickly and continue your learning journey. We offer fundamental to advanced level training, with on-demand, live, and virtual options to suit your busy schedule. Certifications help you validate and prove your skill and expertise in Google Cloud technologies.

Manual Last Updated September 20, 2022

Lab Last Tested May 18, 2022

Copyright 2022 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.