GSP853

Overview

Workflows is used to create serverless workflows that link a series of serverless tasks together in an order you define. You can combine the power of Google Cloud's APIs, serverless products like Cloud Functions and Cloud Run, and calls to external APIs to create flexible serverless applications.

A workflow is made up of a series of steps described using the Workflows YAML-based syntax. This is the workflow's definition. For a detailed explanation of the Workflows YAML syntax, see the Syntax reference page.

When a workflow is created, it is deployed, which makes the workflow ready for execution. An execution is a single run of the logic contained in a workflow's definition. All workflow executions are independent and the product supports a high number of concurrent executions.

Workflows requires no infrastructure management and scales seamlessly with demand, including scaling down to zero. With its pay-per-use pricing model, you only pay for execution time.

In this lab you will connect Cloud Functions and Cloud Run services with Workflows and connect two public Cloud Functions services, one private Cloud Run service, and an external public HTTP API into a workflow.

What you'll learn

• Basics of Workflows.

• How to connect public Cloud Functions with Workflows.

• How to connect private Cloud Run services with Workflows.

• How to connect external HTTP APIs with Workflows.

Prerequisites

Based on the content, it is recommended to have some familiarity with:

• Cloud Run

• Cloud Functions

• Cloud IAM

Setup and requirements

Before you click the Start Lab button

Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources will be made available to you.

This hands-on lab lets you do the lab activities yourself in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab.

To complete this lab, you need:

• Access to a standard internet browser (Chrome browser recommended).

Note: Use an Incognito or private browser window to run this lab. This prevents any conflicts between your personal account and the Student account, which may cause extra charges incurred to your personal account.

• Time to complete the lab---remember, once you start, you cannot pause a lab.

Note: If you already have your own personal Google Cloud account or project, do not use it for this lab to avoid extra charges to your account.

How to start your lab and sign in to the Google Cloud Console

1. Click the Start Lab button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is the Lab Details panel with the following:

   • The Open Google Console button
   • Time remaining
   • The temporary credentials that you must use for this lab
   • Other information, if needed, to step through this lab

2. Click Open Google Console. The lab spins up resources, and then opens another tab that shows the Sign in page.

   Tip: Arrange the tabs in separate windows, side-by-side.

   Note: If you see the Choose an account dialog, click Use Another Account.

3. If necessary, copy the Username from the Lab Details panel and paste it into the Sign in dialog. Click Next.

4. Copy the Password from the Lab Details panel and paste it into the Welcome dialog. Click Next.

   Important: You must use the credentials from the left panel. Do not use your Google Cloud Skills Boost credentials. Note: Using your own Google Cloud account for this lab may incur extra charges.

5. Click through the subsequent pages:

   • Accept the terms and conditions.
   • Do not add recovery options or two-factor authentication (because this is a temporary account).
   • Do not sign up for free trials.

After a few moments, the Cloud Console opens in this tab.

Note: You can view the menu with a list of Google Cloud Products and Services by clicking the Navigation menu at the top-left.

Activate Cloud Shell

Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.

1. Click Activate Cloud Shell at the top of the Google Cloud console.

When you are connected, you are already authenticated, and the project is set to your PROJECT_ID. The output contains a line that declares the PROJECT_ID for this session:

Your Cloud Platform project in this session is set to YOUR_PROJECT_ID

gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.

2. (Optional) You can list the active account name with this command:

gcloud auth list

3. Click Authorize.

4. Your output should now look like this:

Output:

ACTIVE: * ACCOUNT: student-01-xxxxxxxxxxxx@qwiklabs.net To set the active account, run: $ gcloud config set account `ACCOUNT`

5. (Optional) You can list the project ID with this command:

gcloud config list project

Output:

[core] project = <project_ID>

Example output:

[core] project = qwiklabs-gcp-44776a13dea667a6 Note: For full documentation of gcloud, in Google Cloud, refer to the gcloud CLI overview guide.

Task 1. Enable services

1. Connect Cloud Shell to the lab's project:

gcloud config set project $(gcloud projects list \ --format='value(PROJECT_ID)' \ --filter='qwiklabs-gcp')

2. Enable all necessary services:

gcloud services enable \ cloudfunctions.googleapis.com \ run.googleapis.com \ workflows.googleapis.com \ cloudbuild.googleapis.com \ storage.googleapis.com

Click Check my progress to verify that you've performed the above task.

Services enabled for the lab

In the next step, you will connect two Cloud Functions together in a workflow.

Task 2. Deploy a random number generator Cloud Function

The first function is a random number generator in Python.

1. Create and navigate to a directory for the function code:

mkdir ~/randomgen && cd $_

2. Create a main.py file in the directory then add the following contents:

import random, json from flask import jsonify def randomgen(request): randomNum = random.randint(1,100) output = {"random":randomNum} return jsonify(output) Note: When the application receives an HTTP request, the function generates a random number between 1 and 100 and returns in JSON format back to the caller.

3. Dependencies in Python are managed with pip and expressed in a metadata file called requirements.txt. Create a requirements.txt file in the same directory then add the following contents:

flask>=1.0.2

4. Deploy the function with an HTTP trigger and with unauthenticated requests allowed with this command:

gcloud functions deploy randomgen \ --runtime python37 \ --trigger-http \ --allow-unauthenticated Note: If you receive The service has encountered an error during container import, please wait for a minute and run the command again.

5. The deployed Cloud Function relies on Flask for HTTP processing. Cloud Function information can be viewed using the gcloud commands:

gcloud functions describe randomgen \ --format="(httpsTrigger.url)" gcloud functions describe randomgen

6. View the deployed Cloud Function URL:

   • Using the httpsTrigger.url property
   • Using the gcloud functions describe command

7. The function is ready for the workflow.

Click Check my progress to verify that you've performed the above task. Random number generator has been successfully deployed

Task 3. Deploy a number multiplier Cloud Function

The second Cloud Function is a multiplier. It multiplies the received input by 2.

1. Create and navigate to a directory for the function code:

mkdir ~/multiply && cd $_

2. Create a main.py file in the directory then add the following contents:

import random, json from flask import jsonify def multiply(request): request_json = request.get_json() output = {"multiplied":2*request_json['input']} return jsonify(output) Note: The Python application takes a numeric input and multiples by a factor of 2.

3. When the application receives an HTTP request, this function extracts the input from the JSON body, multiplies it by 2 and returns in JSON format back to the caller.

4. Create a new requirements.txt file in the same directory with the following contents:

flask>=1.0.2

5. Deploy the function with an HTTP trigger and with unauthenticated requests allowed with this command:

gcloud functions deploy multiply \ --runtime python37 \ --trigger-http \ --allow-unauthenticated

6. Once the Cloud Function is deployed, visit the URL of the function with the following curl command:

curl $(gcloud functions describe multiply \ --format='value(httpsTrigger.url)') \ -X POST \ -H "content-type: application/json" \ -d '{"input": 5}'

7. The function is ready for the workflow.

Click Check my progress to verify that you've performed the above task.

Number multiplier has been successfully deployed

Task 4. Connect the two Cloud Functions

1. In the first workflow, connect the two functions together. You will need the URLs for the functions you deployed earlier. To get the URLs, run the following commands:

gcloud functions describe multiply \ --format='value(httpsTrigger.url)' gcloud functions describe randomgen \ --format='value(httpsTrigger.url)'

2. Create a workflow.yaml file then add the following contents:

- randomgenFunction: call: http.get args: url: https://<region>-<project-id>.cloudfunctions.net/randomgen result: randomgenResult - multiplyFunction: call: http.post args: url: https://<region>-<project-id>.cloudfunctions.net/multiply body: input: ${randomgenResult.body.random} result: multiplyResult - returnResult: return: ${multiplyResult}

3. Now update the workflow.yaml file. Add the URLs for the functions deployed earlier in the lab, the <region> and <project-id>.

In your first workflow, you used the first Cloud Function (Randomgen) to generate a random number. The result of this function is passed to the second Cloud Function (Multiply).

The second Cloud Function will then return a multiplier of the random number passed to it.

Note: The order of the workflow declarations is important. In the example, randomFunction is called by multipleFunction so randomFunction needs to be declared first.

4. Deploy the first workflow:

gcloud beta workflows deploy workflow --source=workflow.yaml

5. Execute the first workflow:

gcloud beta workflows execute workflow

The command will return the workflow execution id.

6. Once the workflow is executed, you can see the result by passing in the execution ID given in the previous step:

gcloud beta workflows executions describe-last

The output will include result and state:

state: SUCCEEDED

Task 5. Explore the Workflow interface

Use Cloud Console to update your workflow.

1. Find Workflows in Google Cloud Console under the Application Integration category:

   

2. Click on workflow.

3. Find your workflow and click on the SOURCE tab:

   

In this section you can see both the source code and a visualisation of the functions associated with the workflow.

Next, you will connect the external web service math.js to this workflow.

Task 6. Connect to an external HTTP API

Math.js is an extensive math library for JavaScript and Node.js. In this section use the API to evaluate mathematical expressions derived from the Workflow.

1. The math.js API can be used to evaluate mathematical expressions like this:

curl https://api.mathjs.org/v4/?'expr=log(56)' -w "\n"

2. Run the following to delete the existing workflow:

gcloud beta workflows delete workflow

3. Replace the original workflow definition in workflow.yaml with the below code:

- randomgenFunction: call: http.get args: url: https://<region>-<project-id>.cloudfunctions.net/randomgen result: randomgenResult - multiplyFunction: call: http.post args: url: https://<region>-<project-id>.cloudfunctions.net/multiply body: input: ${randomgenResult.body.random} result: multiplyResult - logFunction: call: http.get args: url: https://api.mathjs.org/v4/ query: expr: ${"log(" + string(multiplyResult.body.multiplied) + ")"} result: logResult - returnResult: return: ${logResult} Note: Make sure to replace the URL values with the URLs of your functions, the correct Region and Project Id.

4. Deploy the updated workflow:

gcloud beta workflows deploy workflow \ --source=workflow.yaml

5. Return to the Workflows Console UI.

6. Find your workflow and click on SOURCE tab:

   

7. The workflow now includes the logFunction .

8. Click on Execute to execute the workflow. You'll see the details of the execution:

   

9. Notice the status code 200 and a body with the output of log function.

Task 7. Deploy a Cloud Run service

Finalize the workflow with a call to a private Cloud Run service. This means that the workflow needs to be authenticated to call the Cloud Run service. The Cloud Run service returns the math.floor of the passed in number.

1. Create and navigate to a directory for the service code:

mkdir ~/floor && cd $_

2. Create a app.py file in the directory then add the following contents:

import json import logging import os import math from flask import Flask, request app = Flask(__name__) @app.route('/', methods=['POST']) def handle_post(): content = json.loads(request.data) input = float(content['input']) return f"{math.floor(input)}", 200 if __name__ != '__main__': # Redirect Flask logs to Gunicorn logs gunicorn_logger = logging.getLogger('gunicorn.error') app.logger.handlers = gunicorn_logger.handlers app.logger.setLevel(gunicorn_logger.level) app.logger.info('Service started...') else: app.run(debug=True, host='0.0.0.0', port=int(os.environ.get('PORT', 8080)))

Cloud Run deploys containers, so you need a Dockerfile and your container needs to bind to 0.0.0.0 and PORT env variable, which was done in the code above.

When it receives an HTTP request, this function extracts the input from the JSON body, calls math.floor and returns the result back to the caller.

3. In the same directory, create the a Dockerfile and add the following contents:

# Use an official lightweight Python image. # https://hub.docker.com/_/python FROM python:3.7-slim # Install production dependencies. RUN pip install Flask gunicorn # Copy local code to the container image. WORKDIR /app COPY . . # Run the web service on container startup. Here we use the gunicorn # web server, with one worker process and 8 threads. # For environments with multiple CPU cores, increase the number of workers # to be equal to the cores available. CMD exec gunicorn --bind 0.0.0.0:8080 --workers 1 --threads 8 app:app

4. Run the following command to build the container:

export SERVICE_NAME=floor gcloud builds submit \ --tag gcr.io/${GOOGLE_CLOUD_PROJECT}/${SERVICE_NAME}

5. Once the container is built, deploy to Cloud Run. Notice the no-allow-unauthenticated flag. This makes sure the service only accepts authenticated calls:

gcloud run deploy ${SERVICE_NAME} \ --image gcr.io/${GOOGLE_CLOUD_PROJECT}/${SERVICE_NAME} \ --platform managed \ --no-allow-unauthenticated \ --region us-west1 \ --max-instances=3

Once deployed, the service is ready for the workflow.

Click Check my progress to verify that you've performed the above task. Cloud Run service successfully deployed

Task 8. Connect the Cloud Run service

Before you can configure Workflows to call the private Cloud Run service, you need to create a service account for Workflows to use.

1. Define an environment variable to hold the service account name:

export SERVICE_ACCOUNT=workflows-sa

2. Create a service account:

gcloud iam service-accounts create ${SERVICE_ACCOUNT}

3. Grant run.invoker role to the service account; this will allow the service account to call authenticated Cloud Run services:

gcloud projects add-iam-policy-binding ${GOOGLE_CLOUD_PROJECT} \ --member "serviceAccount:${SERVICE_ACCOUNT}@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com" \ --role "roles/run.invoker"

4. Create a workflow definition file as workflow.yaml to include the Cloud Run service. Notice that now you are including auth field to make sure Workflows passes in the authentication token in its calls to the Cloud Run service:

Note: Make sure you replace the URL values with the actual URLs of your functions, region and project ID. - randomgenFunction: call: http.get args: url: https://<region>-<project-id>.cloudfunctions.net/randomgen result: randomgenResult - multiplyFunction: call: http.post args: url: https://<region>-<project-id>.cloudfunctions.net/multiply body: input: ${randomgenResult.body.random} result: multiplyResult - logFunction: call: http.get args: url: https://api.mathjs.org/v4/ query: expr: ${"log(" + string(multiplyResult.body.multiplied) + ")"} result: logResult - floorFunction: call: http.post args: url: https://floor-<random-hash>.run.app auth: type: OIDC body: input: ${logResult.body} result: floorResult - returnResult: return: ${floorResult}

5. Update the workflow, this time passing in the service-account:

gcloud beta workflows deploy workflow --source=workflow.yaml \ --service-account=${SERVICE_ACCOUNT}@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com

6. Execute the workflow:

gcloud beta workflows execute workflow

7. In a few seconds, run the following to take a look at the workflow execution to see the result:

gcloud beta workflows executions describe-last

The output will include an integer result and state:

result: '{"body":"5","code":200 ... } ... state: SUCCEEDED

Congratulations!

You have successfully completed the lab and demonstrated your knowledge of Workflows on Google Cloud infrastructure.

Google Cloud training and certification

...helps you make the most of Google Cloud technologies. Our classes include technical skills and best practices to help you get up to speed quickly and continue your learning journey. We offer fundamental to advanced level training, with on-demand, live, and virtual options to suit your busy schedule. Certifications help you validate and prove your skill and expertise in Google Cloud technologies.

Manual Last Updated February 13, 2023

Lab Last Tested February 13, 2023

Copyright 2023 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.