One of the best tools for reducing Gen AI hallucinations is to use Retrieval Augmented Generation (RAG). RAG is the concept of retrieving some data or information, then augmenting your prompt used with your Large Language Model (LLM), which allows it to generate more accurate responses based on the data included in the prompt.
You'll also leverage the capabilities of AlloyDB AI, Google Cloud's database for AI-powered applications, and LangChain, a framework for developing applications, to connect the LLM to external data sources. By the end of this lab, you'll have a functional chat application that can intelligently answer questions by retrieving relevant information from your database.
What you'll learn
How LLMs process language and how RAG enhances their capabilities by retrieving relevant information from a knowledge base.
Set up and interact with AlloyDB, a scalable and performant PostgreSQL database designed for demanding workloads.
Explore LangChain's tools and components for building LLM-powered applications, including document loaders, prompt templates, and chains.
Connect AlloyDB as a vector store with LangChain, allowing the chat application to access and retrieve relevant information for generating responses.
Develop a user-friendly interface for interacting with the chat application, allowing users to ask questions and receive informative answers.
Setup and Requirements
Before you click the Start Lab button
Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources are made available to you.
This hands-on lab lets you do the lab activities in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials you use to sign in and access Google Cloud for the duration of the lab.
To complete this lab, you need:
Access to a standard internet browser (Chrome browser recommended).
Note: Use an Incognito (recommended) or private browser window to run this lab. This prevents conflicts between your personal account and the student account, which may cause extra charges incurred to your personal account.
Time to complete the lab—remember, once you start, you cannot pause a lab.
Note: Use only the student account for this lab. If you use a different Google Cloud account, you may incur charges to that account.
How to start your lab and sign in to the Google Cloud console
Click the Start Lab button. If you need to pay for the lab, a dialog opens for you to select your payment method.
On the left is the Lab Details pane with the following:
The Open Google Cloud console button
Time remaining
The temporary credentials that you must use for this lab
Other information, if needed, to step through this lab
Click Open Google Cloud console (or right-click and select Open Link in Incognito Window if you are running the Chrome browser).
The lab spins up resources, and then opens another tab that shows the Sign in page.
Tip: Arrange the tabs in separate windows, side-by-side.
Note: If you see the Choose an account dialog, click Use Another Account.
If necessary, copy the Username below and paste it into the Sign in dialog.
{{{user_0.username | "Username"}}}
You can also find the Username in the Lab Details pane.
Click Next.
Copy the Password below and paste it into the Welcome dialog.
{{{user_0.password | "Password"}}}
You can also find the Password in the Lab Details pane.
Click Next.
Important: You must use the credentials the lab provides you. Do not use your Google Cloud account credentials.
Note: Using your own Google Cloud account for this lab may incur extra charges.
Click through the subsequent pages:
Accept the terms and conditions.
Do not add recovery options or two-factor authentication (because this is a temporary account).
Do not sign up for free trials.
After a few moments, the Google Cloud console opens in this tab.
Note: To access Google Cloud products and services, click the Navigation menu or type the service or product name in the Search field.
Activate Cloud Shell
Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.
Click Activate Cloud Shell at the top of the Google Cloud console.
Click through the following windows:
Continue through the Cloud Shell information window.
Authorize Cloud Shell to use your credentials to make Google Cloud API calls.
When you are connected, you are already authenticated, and the project is set to your Project_ID, . The output contains a line that declares the Project_ID for this session:
Your Cloud Platform project in this session is set to {{{project_0.project_id | "PROJECT_ID"}}}
gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.
(Optional) You can list the active account name with this command:
gcloud auth list
Click Authorize.
Output:
ACTIVE: *
ACCOUNT: {{{user_0.username | "ACCOUNT"}}}
To set the active account, run:
$ gcloud config set account `ACCOUNT`
(Optional) You can list the project ID with this command:
gcloud config list project
Output:
[core]
project = {{{project_0.project_id | "PROJECT_ID"}}}
Note: For full documentation of gcloud, in Google Cloud, refer to the gcloud CLI overview guide.
Configure Cloud Shell
Inside Cloud Shell, make sure that your project ID is setup:
Usually the project ID is shown in parentheses in the command prompt in the cloud shell as it is shown in the picture:
Configure your default region to us-central1 to use the Vertex AI models. Read more about regional restrictions.
gcloud config set compute/region {{{project_0.default_region | REGION }}}
Task 1. Initialize the Environment
Install Postgres client
Install the PostgreSQL client software on the deployed VM
Connect to the VM:
gcloud compute ssh instance-1 --zone={{{project_0.default_zone | ZONE }}}
Enter Y and then when asked for a passphrase press ENTER twice.
Note: First time the SSH connection to the VM can take longer since the process includes creation of RSA key for secure connection and propagating the public part of the key to the project
Expected console output:
student@cloudshell:~ (test-project-402417)$ gcloud compute ssh instance-1 --zone=us-central1-a
Updating project ssh metadata...working..Updated [https://www.googleapis.com/compute/v1/projects/test-project-402417].
Updating project ssh metadata...done.
Waiting for SSH key to propagate.
Warning: Permanently added 'compute.5110295539541121102' (ECDSA) to the list of known hosts.
Linux instance-1 5.10.0-26-cloud-amd64 #1 SMP Debian 5.10.197-1 (2023-09-29) x86_64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
student@instance-1:~$
Install the software running command inside the VM:
student@instance-1:~$ export PGPASSWORD=P9...
student@instance-1:~$ export REGION=us-central1
student@instance-1:~$ export ADBCLUSTER=alloydb-aip-01
student@instance-1:~$ export INSTANCE_IP=export INSTANCE_IP=$(gcloud alloydb instances describe $ADBCLUSTER-pr --cluster=$ADBCLUSTER --region=$REGION --format="value(ipAddress)")
student@instance-1:~$ psql "host=$INSTANCE_IP user=postgres sslmode=require"
psql (13.11 (Debian 13.11-0+deb11u1), server 14.7)
WARNING: psql major version 13, server major version 14.
Some psql features might not work.
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.
postgres=>
Exit from the psql session keeping the SSH connection up:
exit
Expected console output:
postgres=> exit
student@instance-1:~$
Task 2. Initialize the database
You are going to use your client VM as a platform to populate the database with data and host your application. The first step is to create a database and populate it with data.
Create database
Create a database named assistantdemo.
Note: If your SSH session was terminated you need to reset your environment variables such as:
export PGPASSWORD=
export INSTANCE_IP=
cd genai-databases-retrieval-app/retrieval_service
cp example-config.yml config.yml
sed -i s/127.0.0.1/$INSTANCE_IP/g config.yml
sed -i s/my-password/$PGPASSWORD/g config.yml
sed -i s/my_database/assistantdemo/g config.yml
sed -i s/my-user/postgres/g config.yml
cat config.yml
Note: If your SSH session was terminated you need to set your environment variables such as:
export PGPASSWORD=
REGION=us-central1
INSTANCE_IP=$(gcloud alloydb instances describe $ADBCLUSTER-pr --cluster=$ADBCLUSTER --region=$REGION --format="value(ipAddress)")
Expected console output:
student@instance-1:~$ cd genai-databases-retrieval-app/retrieval_service
cp example-config.yml config.yml
sed -i s/127.0.0.1/$INSTANCE_IP/g config.yml
sed -i s/my-password/$PGPASSWORD/g config.yml
sed -i s/my_database/assistantdemo/g config.yml
sed -i s/my-user/postgres/g config.yml
cat config.yml
host: 0.0.0.0
# port: 8080
datastore:
# Example for AlloyDB
kind: "postgres"
host: 10.65.0.2
# port: 5432
database: "assistantdemo"
user: "postgres"
password: "P9..."
Now populate database with the sample dataset. In the VM execute:
The first command is adding all required packages to the Python virtual environment and the second command is populating the database with the data.
Expected console output(redacted):
student@instance-1:~/genai-databases-retrieval-app/retrieval_service$ pip install -r requirements.txt
python run_database_init.py
Collecting asyncpg==0.28.0 (from -r requirements.txt (line 1))
Obtaining dependency information for asyncpg==0.28.0 from https://files.pythonhosted.org/packages/77/a4/88069f7935b14c58534442a57be3299179eb46aace2d3c8716be199ff6a6/asyncpg-0.28.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Downloading asyncpg-0.28.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.3 kB)
Collecting fastapi==0.101.1 (from -r requirements.txt (line 2))
...
database init done.
student@instance-1:~/genai-databases-retrieval-app/retrieval_service$
Populate database with the sample dataset.
Task 5. Deploy the Retrieval Service to Cloud Run
Now you can deploy the retrieval service to Cloud Run. The service is responsible for working with the database. It extracts the necessary information from the database based on the request from an AI application.
Create Service Account
Create a service account for the extension service and grant necessary privileges.
Open another Cloud Shell tab using the "+" at the top.
student@cloudshell:~ (gleb-test-short-003)$ gcloud iam service-accounts create retrieval-identity
Created service account [retrieval-identity].
Close the tab by either execution command "exit" in the tab:
exit
Create the service account retrieval-identity.
Task 6. Deploy the Retrieval Service
Continue in the first tab where you are connected to the VM through SSH by deploying the service.
In the VM SSH session execute:
cd ~/genai-databases-retrieval-app
gcloud alpha run deploy retrieval-service \
--source=./retrieval_service/\
--no-allow-unauthenticated \
--service-account retrieval-identity \
--region us-central1 \
--network=default \
--quiet
Expected console output:
student@instance-1:~/genai-databases-retrieval-app$ gcloud alpha run deploy retrieval-service \
--source=./retrieval_service/\
--no-allow-unauthenticated \
--service-account retrieval-identity \
--region us-central1 \
--network=default
This command is equivalent to running `gcloud builds submit --tag [IMAGE] ./retrieval_service/` and `gcloud run deploy retrieval-service --image [IMAGE]`
Building using Dockerfile and deploying container to Cloud Run service [retrieval-service] in project [gleb-test-short-003] region [us-central1]
X Building and deploying... Done.
✓ Uploading sources...
✓ Building Container... Logs are available at [https://console.cloud.google.com/cloud-build/builds/6ebe74bf-3039-4221-b2e9-7ca8fa8dad8e?project=1012713954588].
✓ Creating Revision...
✓ Routing traffic...
Setting IAM Policy...
Completed with warnings:
Setting IAM policy failed, try "gcloud beta run services remove-iam-policy-binding --region=us-central1 --member=allUsers --role=roles/run.invoker retrieval-service"
Service [retrieval-service] revision [retrieval-service-00002-4pl] has been deployed and is serving 100 percent of traffic.
Service URL: https://retrieval-service-onme64eorq-uc.a.run.app
student@instance-1:~/genai-databases-retrieval-app$
Verify The Service
Now we can check if the service runs correctly and the VM has access to the endpoint. We use gcloud utility to get the retrieval service endpoint. Alternatively you can check it in the cloud console and replace in the curl command the "$(gcloud run services list –filter="(retrieval-service)" by the value from there.
In the VM SSH session execute:
curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" $(gcloud run services list --filter="(retrieval-service)" --format="value(URL)")
Expected console output:
student@instance-1:~/genai-databases-retrieval-app$ curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" $(gcloud run services list --filter="(retrieval-service)" --format="value(URL)")
{"message":"Hello World"}student@instance-1:~/genai-databases-retrieval-app$
If you see the "Hello World" message it means your service is up and serving the requests.
Deploy the Retrieval Service.
Task 7. Deploy sample application
When you have the retrieval service up and running, you can deploy a sample application which is going to use the service. The application can be deployed on the VM or any other service like Cloud Run, Kubernetes or even locally on a laptop.
Next you are going to deploy a sample application on the VM.
Prepare the environment
First, add the necessary modules to Python.
In the VM SSH session execute:
cd ~/genai-databases-retrieval-app/llm_demo
pip install -r requirements.txt
Expected output (redacted):
student@instance-1:~$ cd ~/genai-databases-retrieval-app/llm_demo
pip install -r requirements.txt
Collecting fastapi==0.104.0 (from -r requirements.txt (line 1))
Obtaining dependency information for fastapi==0.104.0 from https://files.pythonhosted.org/packages/db/30/b8d323119c37e15b7fa639e65e0eb7d81eb675ba166ac83e695aad3bd321/fastapi-0.104.0-py3-none-any.whl.metadata
Downloading fastapi-0.104.0-py3-none-any.whl.metadata (24 kB)
...
Run Assistant application
Before starting the application you need to set up some environment variables. The basic functionality of the application, such as query flights and airport amenities, requires BASE_URL which points application to the retrieval service. We can get it using the gcloud command .
In the VM SSH session execute:
export BASE_URL=$(gcloud run services list --filter="(retrieval-service)" --format="value(URL)")
Expected output (redacted):
student@instance-1:~/genai-databases-retrieval-app/llm_demo$ export BASE_URL=$(gcloud run services list --filter="(retrieval-service)" --format="value(URL)")
Prepare Client ID
To use more advanced capabilities of the application like booking and changing flights we need to sign-in to the application using our Google account and for that purpose we need to provide CLIENT_ID environment variable using the OAuth client ID from the Prepare Client ID chapter:
To use booking functionality of the application we need to prepare OAuth 2.0 Client ID using Cloud Console. It will be when we sign into the application since booking is using clients credentials to record the booking data in the database.
In the Cloud Console go to the APIs and Services and click on "OAuth consent screen" and click Get Started.
Then follow on the next screen.
You need to fill out required fields such as "App name" and "User support email". Select Internal for "Audience" and finally the "Contact information".
Agree to the user data policy. Click Continue and then click Create at the bottom of the page and it will lead you to the next page.
The next step is to create the client ID. On the left panel you click "Clients" which lead you to the credentials for OAuth2.
Here you click "Create client" at the top. Then it will open another screen.
Pick up "Web application" from the dropdown list for application type and put your application URI (and port - optionally) as the "Authorized JavaScript origins". And you need to add to the "Authorized redirect URIs" your application host with "/login/google" at the end to be able to use the authorization popup screen.
After pushing the "Create" button.
We will need the Client ID (and optionally Client secret) later to use with your application.
export CLIENT_ID=450....apps.googleusercontent.com
Note: Replace the CLIENT_ID value with your Client Id that you just created.
Expected output (redacted):
student@instance-1:~/genai-databases-retrieval-app/llm_demo$ export CLIENT_ID=215....apps.googleusercontent.com
Create a Client Id.
Now you can run the application:
python run_app.py
Expected output:
student@instance-1:~/genai-databases-retrieval-app/llm_demo$ python main.py
INFO: Started server process [28565]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8081 (Press CTRL+C to quit)
Connect to the Application
You have several ways to connect to the application running on the VM. For example you can open port 8081 on the VM using firewall rules in the VPC or create a load balancer with public IP. Here we are going to use a SSH tunnel to the VM translating the local port 8080 to the VM port 8081.
Open another Cloud Shell tab using the sign "+" at the top.
In the new cloud shell tab start the tunnel to your VM by executing the gcloud command:
It will show an error "Cannot assign requested address" - please ignore it.
Here is the expected output:
student@cloudshell:~ gcloud compute ssh instance-1 --zone=us-central1-a -- -L 8080:localhost:8081
bind [::1]:8081: Cannot assign requested address
linux instance-1.us-central1-a.c.gleb-codelive-01.internal 6.1.0-21-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.90-1 (2024-05-03) x86_64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Sat May 25 19:15:46 2024 from 35.243.235.73
student@instance-1:~$
It opens port 8080 in your Cloud Shell which can be used for the Web preview.
Click on the Web preview button on the right top of your Cloud Shell then select Preview on port 8080.
This opens a new tab in your web browser with the application interface. You should see the Cymbal Air Customer Service Assistant page. In the address bar of the page you'll see the URI to the preview page.
Remove the part /?authuser=0&redirectedPreviously=true at the end.
Use the first part of URI like "https://8080-cs-35704030349-default.cs-us-east1-vpcf.cloudshell.dev/" is to be left in the browser window and to provide as the "Authorized JavaScript origins" and "Authorized redirect URIs" for our credentials created in the "Prepare Client ID" chapter replacing or adding to the originally provided http://localhost:8080 values. The upper value would look like "https://8080-cs-35704030349-default.cs-us-east1-vpcf.cloudshell.dev" and the lower one would be "https://8080-cs-35704030349-default.cs-us-east1-vpcf.cloudshell.dev/login/google"
(Optional) Task 100 Sign into the application
Click the__Sign in__ button at the top right of the application screen to provide our credentials. That is optional and required only if you want to try booking functionality of the application.
This demo showcases the Cymbal Air customer service assistant. Cymbal Air is a fictional passenger airline. The assistant is an AI chatbot that helps travellers to manage flights and look up information about Cymbal Air's hub at San Francisco International Airport (SFO).
It can help answer users questions like:
When is the next flight to Denver?
Are there any luxury shops around gate D50?
Where can I get coffee near gate A6?
Where can I buy a gift?
When you are signed in to the application you can try other capabilities like booking flights or check if the seat assigned to you is a window or aisle seat.
The application uses the latest Google foundation models to generate responses and augment it by information about flights and amenities from the operational AlloyDB database. You can read more about this demo application on the GitHub page of the project.
Congratulations
You've successfully built a cutting-edge chat application that leverages Large Language Models (LLMs) and Retrieval Augmented Generation (RAG) for engaging and informative conversations. By combining the power of AlloyDB's vector database with LangChain's flexible framework, you've created an intelligent agent capable of accessing and utilizing vast amounts of information to generate contextually relevant responses.
...helps you make the most of Google Cloud technologies. Our classes include technical skills and best practices to help you get up to speed quickly and continue your learning journey. We offer fundamental to advanced level training, with on-demand, live, and virtual options to suit your busy schedule. Certifications help you validate and prove your skill and expertise in Google Cloud technologies.
Manual Last Updated March 07, 2025
Lab Last Tested March 07, 2025
Copyright 2025 Google LLC. All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.
Labs create a Google Cloud project and resources for a fixed time
Labs have a time limit and no pause feature. If you end the lab, you'll have to restart from the beginning.
On the top left of your screen, click Start lab to begin
Use private browsing
Copy the provided Username and Password for the lab
Click Open console in private mode
Sign in to the Console
Sign in using your lab credentials. Using other credentials might cause errors or incur charges.
Accept the terms, and skip the recovery resource page
Don't click End lab unless you've finished the lab or want to restart it, as it will clear your work and remove the project
This content is not currently available
We will notify you via email when it becomes available
Great!
We will contact you via email if it becomes available
One lab at a time
Confirm to end all existing labs and start this one
Use private browsing to run the lab
Use an Incognito or private browser window to run this lab. This
prevents any conflicts between your personal account and the Student
account, which may cause extra charges incurred to your personal account.
Learn how to create an interactive application within a deployed environment.