arrow_back

Google Assistant: Build an Application with Dialogflow and Cloud Functions

Join Sign in

Google Assistant: Build an Application with Dialogflow and Cloud Functions

1 hour 15 minutes 1 Credit

GSP174

Google Cloud self-paced labs logo

Overview

Google Assistant is a personal voice assistant that offers a host of actions and integrations. From making appointments and setting reminders, to ordering coffee and playing music, the 1 million+ actions available suit a wide range of voice command tasks. Google Assistant is offered on Android and iOS, but it can even be integrated with other devices like smartwatches, Google Homes, and Android TVs.

Actions is the central platform for developing Google Assistant applications. The Actions platform integrates with human-computer interaction suites, which simplifies conversational app development. The most widely used suite is Dialogflow, which uses an underlying machine learning (ML) and natural language understanding (NLU) schema to build rich Assistant applications. The Actions platform also integrates with Cloud Functions, which lets you run backend fulfillment code in response to events triggered by Dialogflow requests.

In this lab, you will get hands-on practice with the Actions platform, the Dialogflow suite, and Cloud Functions by building a "Silly Name Maker" application, which returns a user with a silly name after they have entered in a lucky number and favorite color. You will build a Dialogflow agent that intelligently parses user input for specific information. The agent will be supplemented with a webhook, which will trigger a Cloud Function that handles fulfillment logic and returns your user with their silly name.

What you will learn

In this lab, you will learn how to:

  • Create an Actions project and build an Action.

  • Create a Dialogflow agent and configure the default welcome intent.

  • Build a custom intent with entities.

  • Initialize a Cloud Function.

  • Add fulfillment logic and packages to your Cloud Function.

  • Add a webhook to your Action.

  • Test your Assistant application with the Actions simulator on expected and unexpected conversational paths.

  • Optional: test your Assistant application on a Google Home device.

Prerequisites

This is a fundamental level lab. Familiarity with the Actions Console and the Qwiklabs platform is expected. If you need to get up to speed with the lab's requirements, please complete one of the following Qwiklabs:

Since this lab works with the Actions simulator, having a pair of headphones or turning the volume up on your computer is recommended. If you want to test your Assistant application on a Google Home, keep your device handy.

Once you're ready, scroll down and follow the steps below to set up your lab environment.

Setup

Cloud Console

How to start your lab and sign in to the Google Cloud Console

  1. Click the Start Lab button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is the Lab Details panel with the following:

    • The Open Google Console button
    • Time remaining
    • The temporary credentials that you must use for this lab
    • Other information, if needed, to step through this lab
  2. Click Open Google Console. The lab spins up resources, and then opens another tab that shows the Sign in page.

    Tip: Arrange the tabs in separate windows, side-by-side.

    Note: If you see the Choose an account dialog, click Use Another Account.
  3. If necessary, copy the Username from the Lab Details panel and paste it into the Sign in dialog. Click Next.

  4. Copy the Password from the Lab Details panel and paste it into the Welcome dialog. Click Next.

    Important: You must use the credentials from the left panel. Do not use your Google Cloud Skills Boost credentials. Note: Using your own Google Cloud account for this lab may incur extra charges.
  5. Click through the subsequent pages:

    • Accept the terms and conditions.
    • Do not add recovery options or two-factor authentication (because this is a temporary account).
    • Do not sign up for free trials.

After a few moments, the Cloud Console opens in this tab.

Note: You can view the menu with a list of Google Cloud Products and Services by clicking the Navigation menu at the top-left. Navigation menu icon

Create an Actions project

Regardless of the Assistant application you're building, you will always have to create an Actions project so your app has an underlying organizational unit.

Open the Actions on Google Developer Console in a new tab. Sign in with your Qwiklabs credentials if prompted. You should be looking at a clean Actions console that resembles the following:

ActionsConsole.png

Click New Project and agree to Actions on Google's terms of service when prompted by clicking Agree and continue.

Click into the Project Name field and select your Qwiklabs Google Cloud project ID from the dropdown. Then click Import project:

2d3fbaed16930b02.png

Soon after you will be presented with a welcome page that resembles the following:

actions-homepage.png

Now click Actions Console in the top left corner to return to the homepage. Then click on the project you just created (title has your Project ID as the name.)

Build an Action

An action is an interaction you build for the Google Assistant. An action supports a specific intent (a goal or task that users want to accomplish), which is carried out by a corresponding fulfillment (logic that handles an intent and carries out the corresponding Action.) You will now build an Action that supports silly name generation.

Click on your project name. Then from the center menu click Build your Action > Add Action(s) > Get Started. Then select Custom Intent > BUILD:

297e7815f62b4452.png

This will take you to the Dialogflow console. Select your Qwiklabs account and click Allow when Dialogflow prompts you for permission to access your Google Account.

When you land on the Dialogflow account settings page, check the box next to Yes, I have read and accept the agreement and click ACCEPT.

If you are brought to the following Dialogflow agent creation page, click CREATE:

agent-creation.png

If you are brought to this page instead:

agent-creation-2.png

Close the Dialogflow agent creation tab. You will return to the Actions Console.

Click Get Started > Custom Intent > BUILD.

Select your Qwiklabs account and click Allow when Dialogflow prompts you for permission to access your Google Account.

Now click CREATE:

agent-creation.png

An agent is an organizational unit that collects information needed to complete a user's request, which it then forwards to a service that provides fulfillment logic.

You will now build the basic framework for fulfillment logic. This will be handled (later) by a Cloud Function, which will return a response with a user's silly name.

Test Completed Task

Click Check my progress to verify your performed task.

Create a Dialogflow Agent traverse from Action Project

Click Fulfillment from the left-hand menu. Move the slider for Webhook to the right, setting it to Enabled.

Now enter the temporary URL https://google.com in for the URL field. You will update this URL when you build your Cloud Function. Your page should resemble the following:

fulfillment.png

Scroll down and click Save in the bottom right corner. Then click Intents from the left hand menu and select Default Welcome Intent:

375730d34f1f3cc0.png

You will now build the main entry point into your application by configuring the default welcome intent.

Configure the default welcome intent

When you created the Dialogflow agent, a Default Welcome intent is automatically created. This intent represents the main entry point into your conversation and the main action of your app. Your app must have a Default Welcome Intent defined, so that Actions on Google knows how to invoke your app. See Invocation and Discovery for more information on how these invocation models work.

Make sure that you are in the Default Welcome Intent. Here are some things to notice about the default settings:

  • The Events section of the intent specifies a Welcome event, which signifies that this intent is the default entry point into your app. The Google Assistant uses this event to trigger your app when users invoke your app by name, such as with "Ok Google, talk to Silly Name Maker".
  • In the Responses section, there are default, static text responses. The Default Welcome Intent contains pre-populated responses that you should remove to add your own.

Now that you know the Default Welcome Intent's purpose, you will modify some fields. Scroll down to the Responses section and click the trash icon to scrap all of the default text responses:

584f65bfc02cac9.png

Now click on ADD RESPONSES > Text response, and type in the following: Hi! Welcome to Silly Name Maker! Let's get started. What is your lucky number?

default-response.png

Now scroll up and click Save in the top-right corner. Now when users invoke your app, they know that they are entering your app's experience and what they should say next.

Test Completed Task

Click Check my progress to verify your performed task.

Configure the Default Welcome Intent

In general, the app's responses should guide users on what to say next and should stay within your conversations grammar. This will be mapped out further in the following section.

Design the conversation

Before you begin building out your Assistant application's language path, it's always a good idea to map out a conversation by writing sample dialogs. For the Silly Name Maker app, we've provided a couple of simple dialogs, which are described below:

Expected path

  • App: Hi! Welcome to Silly Name Maker! Let's get started. What is your lucky number?

  • User: 5

  • App: What is your favorite color?

  • User: Yellow

  • App: Alright, your silly name is Yellow 5! I hope you like it. See you next time.

Unexpected path

  • App: Hi! Welcome to Silly Name Maker! Let's get started. What is your lucky number?
  • User: Dog
  • App: Hmm that doesn't sound like a number to me. What's your lucky number again?
  • User: 22
  • App: What is your favorite color?
  • User: Blue
  • App: Alright, your silly name is Blue 22! I hope you like it. See you next time.
Conversational design tips: This example shows only a couple of the dialogs that you could write for your own apps. See this Design Walkthrough for more examples of dialogs.

Now that you have a better understanding of the type of application you'll be building, you will develop your custom intent to parse and extract colors and numbers from user input.

Configure a custom intent

Dialogflow intents define your conversation's grammar and what tasks to carry out when users make specific requests. Since Dialogflow includes a natural language understanding (NLU) engine, you don't need to be exhaustive when defining phrases. Dialogflow automatically expands on the phrases you provide and understands many more variations of them.

You will now create an intent that defines how users need to provide their lucky number and favorite color to generate a silly name.

From the left hand menu, click on the + icon by the Intents menu item. For the Intent name field, enter make_name:

make_name.png

In the Training phrases field click Add Training Phrases. Then in the Add user expression field enter in My lucky number is 23. Highlight the number 23 and assign it the @sys.number "entity" and hit Enter.

Your actions and parameters section should resemble the following:

58525c8326cb596f.png

Entities extract specific data from user expressions and store them in accessible variables. By assigning this portion of the user says to be an entity, Dialogflow can extract parameters from the user input, validate the parameter and provide it to your fulfillment as a variable.

Without assigning entities, you would have to parse the input yourself and find the parameters you need.

You will now add some more user expressions in the Training phrases section, and assign any number within those phrases the @sys.number entity (if it isn't automatically set.) Add the following training phrases by entering them in the Add user expression field:

  • 23
  • The luckiest number I have is 12
  • My lucky number is 18

Your Training phrases section should now resemble the following — note the highlighted numbers, which align with the @sys.number entity:

training-phrases.png

Now that your training phrases have been added, scroll down and expand the Actions and parameters > MANAGE PARAMETERS AND ACTION section.

Check the required checkbox for the number parameter. This tells Dialogflow to not trigger the intent until the parameter is properly provided by the user.

Now click on Define prompts for the number parameter (right-hand side) and provide a re-prompt phrase. Type in What's your lucky number? in the prompt field and then click Close:

lucky-number.png

This phrase will be spoken to the user repeatedly until Dialogflow detects a number in the user input.

In the Action and parameters section, add a parameter with the following information:

  • Required - Select the checkbox
  • Parameter name - color
  • Entity - @sys.color
  • Value - $color
  • Prompts - What's your favorite color?

This additional parameter uses Dialogflow's built-in "slot-filling" feature, which allows you to obtain additional input parameters from the user without having to create an intent for each one and without having to make the user speak all the required input in one phrase.

Slot filling also lets you set parameters as being required so that your agent doesn't process the input until the user provides all required parameters. Your actions and parameters should now resemble the following:

actions-and-parameters.png

Now scroll to the top of the page and click Save.

Now scroll down and expand the Fulfillment section and click Enable fulfillment. Then click the Enable webhook call for this intent slider:

ba365cb2730dd15b.png

This tells Dialogflow to call your fulfillment to generate a response to the user instead of using Dialogflow's response feature.

In the Responses section right above Fulfillment, click on + icon and select Google Assistant.

Move the toggle for Set this intent as end of conversation. This tells Dialogflow to relinquish control back to the Google Assistant after your fulfillment returns a response to the user.

g-a.png

Then scroll to the top of the intent and click SAVE once more to save the entire intent.

Test Completed Task

Click Check my progress to verify your performed task.

Configure a Custom Intent (name: make_name)

You have now successfully declared your conversation's grammar with Dialogflow intents. You have also used Dialogflow's built-in response feature to return a static response to the user when they invoke your app. You also created an intent that uses fulfillment to return a response, which you will now implement with a Cloud Function.

Initialize and configure a Cloud Function

You will now build a Cloud Function to handle your fulfillment logic. Your fulfillment takes the extracted user input that Dialogflow parsed and responds with the user's silly name. Return to the Cloud Console for this step.

Open the Navigation menu and select Cloud Functions, which is located under the serverless header. Then click CREATE FUNCTION.

This will open a template to create a new Cloud Function. Your page will resemble the following:

cloud-func-template.png

For the Cloud Function Name field, enter in silly-name-maker.

Then scroll down to the authentication section and check the box next to Allow unauthenticated invocations:

authentication.png

Click Save.

Forgetting to do the above will cause your simulation test to fail at the end!

Click Next.

Now scroll down and find the inline editor for index.js and package.json. Make sure that the index.js tab is open. This file defines your fulfillment logic and is used to create and deploy a Cloud Function. Here are some specifics on its basic functioning:

  • When Dialogflow intents are triggered, the intent's action name (declared in the action area of the intent) is provided to you in the request to your fulfillment. You use this action name to determine what logic to carry out.
  • Within every request to your fulfillment, if Dialogflow parsed parameters from the user input, you can access the parameter by name. Here, you declare the names of the parameters so you can access them later.
  • This function fulfills the action by generating a silly name. It's called whenever the make_name intent is triggered. It uses the parameters from the user input to generate the name.

Now that you have a better understanding of index.js, you will build out the function's fulfillment logic. Remove the boilerplate code from the file. Then copy and paste the following code into index.js:

'use strict'; const {dialogflow} = require('actions-on-google'); const functions = require('firebase-functions'); const app = dialogflow({debug: true}); app.intent('make_name', (conv, {color, number}) => { conv.close(`Alright, your silly name is ${color} ${number}! ` + `I hope you like it. See you next time.`); }); exports.sillyNameMaker = functions.https.onRequest(app);

Now open the package.json tab. This file declares package dependencies for your fulfillment, including the Actions client library. Replace the contents of the file with the following:

{ "name": "silly-name-maker", "description": "Find out your silly name!", "version": "0.0.1", "author": "Google Inc.", "dependencies": { "actions-on-google": "^3.0.0", "firebase-admin": "^10.0.1", "firebase-functions": "3.16.0" } }

Once you have those files configured, find the Entry point field. Enter sillyNameMaker for the value.

Now click the Deploy button below. It will take about a minute for your function to be built. When the creation completes, your overview page will resemble the following:

cloud-func.png

Now click on the silly-name-maker function to get more details about it's configuration. Then click on the Trigger tab. You will see a trigger URL that resembles the following:

func-url.png

Copy the trigger URL. You will use it as the URL for the Dialogflow webhook, which is configured in the next section.

Test Completed Task

Click Check my progress to verify your performed task.

Initialize and Configure a Cloud Function

Configure the webhook

Return to the Dialogflow console and click on the Fulfillment menu item from the left hand navigation menu.

In the URL field, replace https://google.com with the trigger URL you generated in the previous step.

Your webhook should now resemble the following:

da2de2577e20c6d7.png

Scroll down to the bottom of the page and click Save in the lower right corner.

Test your Assistant application with the Actions simulator

Now that you your Cloud Function has been deployed and your webhook has been properly set up, you can preview the app in the Actions simulator.

Check your Google permission settings

In order to test the Silly Name Maker, you need to enable the necessary permissions.

Open a new tab in your browser and visit the Activity Controls page. Sign in with your Qwiklabs credentials if requested.

Ensure that the following permissions are enabled by sliding the toggles to TURN ON the following cards (confirm by clicking on Turn on > Got it):

  • Web & App Activity

Now close the Activity Controls page.

Test the application with the simulator

Return to the Dialogflow console. Then from the left-hand menu, click Integrations. Then click on integration link inside the Try Actions Builder section.

Once you land on the following page, click TEST:

10b7d49ec40d1157.png

To invoke the Action, hit the enter key in the Talk to my test app box of the simulator console. You should be presented with a similar response:

288fd9ee31b6f1c8.png

Now enter in a number. When prompted to enter in a favorite color, enter in a color. You should be returned with a silly name built from the number and color you entered in:

3fff7ac7722b5dca.png

Now try entering in some input that follows the unexpected path. Enter Talk to my test app in the input area once more. Then provide a number when asked to enter in your lucky number.

When asked for your favorite color, try entering in a word that isn't a color. The reprompt phrase should kick in, asking you for your favorite color again:

bad-input.png

Once you provide the correct input, your silly name will be generated and the conversation will end.

Optional: test your application on a Google Home device

In this optional section, you will learn how to test an application on a Google Home device. At the risk of stating the obvious, you must have a Google Home device to complete this step. You will also need a smart phone (Android or iOS) to configure your Google Home. The following walkthrough uses an iPhone. If you use Android the steps may be slightly different.

Make sure that you have a new device or one that is factory reset before you begin this step. This guide will teach you how to reset all Google Home devices.

Configure your Google Home

Ensure that your device is plugged into an outlet and download the Google Home application for android or iOS. Ensure that your smartphone's Wi-Fi and Bluetooth are turned on.

Once the application has downloaded, open it and click GET STARTED. Then for the Choose account section, select Use another account > OK:

choose-account.png

Then sign in using your Qwiklabs username and password. You will then be prompted to Choose a Google Home device. Select Create another home > Next:

choose-home.png

Now give your Google Home a nickname (lab instructions use Qwiklabs.) Your page should now resemble the following:

qwiklabs-home.png

Now, open your Wi-Fi settings on your smart phone. You should see a connection like GoogleHomeXXXX.x. Connect to this Wi-Fi network:

wifi.png

Note: the above screenshot comes from an iPhone. Android will differ.

Now return to the Google Home application. Your application should automatically detect the Google Home device and will attempt to connect to it. Once connected, you will be asked if you heard the test sound:

test-sound.png

If so, select YES. Decide whether or not you want to help improve your Google Home by sending crash reports. When prompted, select where your device is located (e.g. the kitchen) and click NEXT.

Now you will be asked to select a Wi-Fi network. Choose your local network. You will then see the following "Connecting to Wi-Fi" screen:

connecting.png

Once the device is connected, you will be asked to "Set up Google Assistant". Click NEXT.

Note: if this step fails and you get an "operation couldn't be completed" error, try clicking NEXT once more. If this doesn't work, you may have to run through the steps in this section once more.

Decide whether or not you want Google Assistant to recognize your voice. Decide whether you want to get personal results. Specify whether or not you want to use Voice 1 or Voice 2 for your Assistant. Enter your address.

Say NOT NOW when prompted to add media services. Say NO THANKS when prompted to receive email updates. When you land on the "Almost done!" page, click NEXT. You should now receive a prompt that says your device is ready:

speaker-ready.png

Click CONTINUE. Follow any extra set up prompts.

Now that you have set up your Google Home with your Qwiklabs account, you're all ready to test your application.

Test the application on your Google Home

Now that your device is configured with your Qwiklabs account, you can test your application using voice commands.

Say the following to your Google Home:

"Okay Google, talk to my test app".

You will then hear "Sure, here's the version of of my test app." This will be followed by the welcome intent: "Hi, welcome to Silly Name Maker! Let's get started. What is your favorite number?"

You can now follow the prompts to generate a silly name.

Congratulations!

In this lab, you built a robust Google Assistant application with Dialogflow and Cloud Functions. After creating an Actions project, you configured and built two intents. From there you learned how to initialize a Cloud Function, where you added your own fulfillment logic and packages. After deploying the function, you added a webhook and tested your Assistant application with the Actions simulator. You are now ready to take more advanced Google Assistant labs.

Finish Your Quest

assistant-badge.png Firebase_125x125

This self-paced lab is part of the Qwiklabs OK Google: Build Interactive Apps with Google Assistant and Build Apps & Websites with Firebase Quests. A Quest is a series of related labs that form a learning path. Completing this Quest earns you the badge above, to recognize your achievement. You can make your badge (or badges) public and link to them in your online resume or social media account. Enroll in a Quest and get immediate completion credit if you've taken this lab. See other available Qwiklabs Quests.

Take Your Next Lab

Continue your Quest with Google Assistant: Build a Youtube Entertainment App, or check out these other suggestions:

Google Cloud training and certification

...helps you make the most of Google Cloud technologies. Our classes include technical skills and best practices to help you get up to speed quickly and continue your learning journey. We offer fundamental to advanced level training, with on-demand, live, and virtual options to suit your busy schedule. Certifications help you validate and prove your skill and expertise in Google Cloud technologies.

Manual Last Updated January 11, 2022
Lab Last Tested January 11, 2022

Copyright 2022 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.