GSP1165

Overview

Prompt design is the process of creating prompts that are effective in generating the desired output from a large language model (LLM) like PaLM. Prompts can be used to generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way.

To get a good results from your prompt, the user needs to be specific, use keywords that are relevant to the output you want to generate, and provide PaLM with examples of the output you want to help it to better understand what you are looking for.

In this lab you'll have the role of a marketing analyst for a real estate firm. You need to create prompts that can extract summaries from very long home descriptions on your company's website to help customers find homes that match their specific requirements.

What you'll learn

In this lab, you review Vertex AI as an AI/ML Platform and explore how to:

• Design good quality prompts.
• Interact with PaLM to get the desired responses.
• Be aware of hallucinations in responses.

Use case

Your company is interested in using large language models (LLMs) to return brief text descriptions of homes to buyers after they provide the home features they are looking for.

You have been tasked with creating prompts that will summarize text from very long home descriptions on your real estate site. The home descriptions are stored in a file in a Google Cloud Storage bucket.

You begin by using Model Garden to explore available pre-built models to save time, and implement a solution as quickly as possible.

Vertex AI features used in this lab

Generative AI models

The generative AI models available in Vertex AI, also called foundation models, are categorized by the type of content that it's designed to generate. This content includes text and chat, image, code, and text embeddings. Each model is exposed through a publisher endpoint that's specific to your Google Cloud project, so there's no need to deploy the foundation model unless you need to tune them for a specific use case.

Pathways and Pathways Language Model (PaLM)

Pathways is a single model that could generalize across domains and tasks while being highly efficient. The Pathways Language Model (PaLM), a dense decoder-only Transformer model trained with the Pathways system, which enabled Google to efficiently train a single model across multiple TPU v4 Pods. PaLM is evaluated on hundreds of language understanding and generation tasks, it has strong capabilities in multilingual tasks and source code generation.

PaLM 2 is the underlying model that is driving the PaLM API. PaLM 2 is a state-of-the-art language model with improved multilingual, reasoning, and coding capabilities. To learn more about PaLM 2, see Introducing PaLM 2. This notebook shows you how to design prompts to create different kinds of summaries.

Vertex AI Studio

Vertex AI Studio is a Google Cloud console tool for rapidly prototyping and testing generative AI models. You can test sample prompts, design your own prompts, and customize foundation models to handle tasks that meet your application's needs. You can perform the following:

• Test models using prompt samples.
• Design and save your own prompts.
• Tune a foundation model.
• Convert between speech and text.

Learn more about AI Studio in the official documentation.

Vertex AI Workbench

Vertex AI Workbench is a fully managed, scalable, enterprise-ready compute infrastructure that provides a single environment for data scientists to complete all of their ML work, from experimentation, to deployment, to managing and monitoring models. It is a Jupyter-based environment that includes a wide range of tools and services, such as Jupyter notebooks or managed compute resources.

Setup and requirements

Before you click the Start Lab button

Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources will be made available to you.

This hands-on lab lets you do the lab activities yourself in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab.

To complete this lab, you need:

• Access to a standard internet browser (Chrome browser recommended).

Note: Use an Incognito or private browser window to run this lab. This prevents any conflicts between your personal account and the Student account, which may cause extra charges incurred to your personal account.

• Time to complete the lab---remember, once you start, you cannot pause a lab.

Note: If you already have your own personal Google Cloud account or project, do not use it for this lab to avoid extra charges to your account.

How to start your lab and sign in to the Google Cloud console

1. Click the Start Lab button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is the Lab Details panel with the following:

   • The Open Google Cloud console button
   • Time remaining
   • The temporary credentials that you must use for this lab
   • Other information, if needed, to step through this lab

2. Click Open Google Cloud console (or right-click and select Open Link in Incognito Window if you are running the Chrome browser).

   The lab spins up resources, and then opens another tab that shows the Sign in page.

   Tip: Arrange the tabs in separate windows, side-by-side.

   Note: If you see the Choose an account dialog, click Use Another Account.

3. If necessary, copy the Username below and paste it into the Sign in dialog.

   {{{user_0.username | "Username"}}}

   You can also find the Username in the Lab Details panel.

4. Click Next.

5. Copy the Password below and paste it into the Welcome dialog.

   {{{user_0.password | "Password"}}}

   You can also find the Password in the Lab Details panel.

6. Click Next.

   Important: You must use the credentials the lab provides you. Do not use your Google Cloud account credentials. Note: Using your own Google Cloud account for this lab may incur extra charges.

7. Click through the subsequent pages:

   • Accept the terms and conditions.
   • Do not add recovery options or two-factor authentication (because this is a temporary account).
   • Do not sign up for free trials.

After a few moments, the Google Cloud console opens in this tab.

Note: To view a menu with a list of Google Cloud products and services, click the Navigation menu at the top-left.

Enable APIs

1. In the Google Cloud console, from the Navigation menu (), click More products, and then from the Artificial Intelligence section select Vertex AI .
2. From the Vertex AI Dashboard, click Enable all Recommended APIs.

Open the Vertex AI Workbench

1. Still in the Vertex AI Dashboard, click Workbench in the left pane.

   Note: You may have to increase the width of your browser window to see the left pane.

2. Click the User-managed notebooks tab.

Task 1. Question answering with Generative Models on Vertex AI

You've been asked to build a UI to allow prospective customers to filter real estate offerings based on their answers.

1. In the User-managed notebooks tab, click Open JupyterLab next to your notebook name.

The Vertex AI Workbench opens, notice the files in the left pane.

2. Navigate to and then double-click to open training-data-analyst/self-paced-labs/genai/housing_question_answering.ipynb.

housing_question_answering.ipynbis open and visible in the right pane.

3. Click Edit > Clear All Outputs

4. Carefully read through the notebook instructions and run the code in each cell.

To run the current cell, click the cell and press SHIFT + ENTER. Other cell commands are listed in the notebook UI under Run.

Leave your Jupyter notebook environment open, and continue reading this task. You return to the notebook environment in Task 3.

Click Check my progress to verify the objective.

Run the cells in the housing_question_answering.ipynb file.

Parameter values

Each call that you send to a model includes parameter values that control how the model generates a response. The model generates different results for different parameter values. For this task, you experiment with different parameter values to get the best results for the task.

The parameters available for different models may differ, but the most common are:

• Temperature
• Token limit
• Top-K
• Top-P

Temperature

Temperature controls the degree of randomness in token selection, when Top-K and Top-P are applied. Lower temperatures are good for prompts that require a more deterministic and less open-ended or creative response, while higher temperatures can lead to more diverse or creative results. A temperature of 0 is deterministic, meaning that the highest probability response is always selected.

For most use cases, try starting with a temperature of 0.2. If the model returns a response that's too generic, too short, or the model gives a fallback response, try increasing the temperature.

Token limit

Token limit determines the maximum amount of text output from one prompt. A token is approximately four characters. The default value is 256.

Specify a lower value for shorter responses and a higher value for longer responses.

Top-K

Top-K changes how the model selects tokens for output. A Top-K of 1 means the next selected token is the most probable among all tokens in the model's vocabulary (also called greedy decoding), while a Top-K of 3 means that the next token is selected from among the three most probable tokens by using temperature.

For each token selection step, the Top-K tokens with the highest probabilities are sampled. Then tokens are further filtered based on Top-P with the final token selected using temperature sampling.

Specify a lower value for less random responses and a higher value for more random responses. The default Top-K is 40.

Top-P

Top-P changes how the model selects tokens for output. Tokens are selected from the most (see Top-K) to least probable until the sum of their probabilities equals the Top-P value. For example, if tokens A, B, and C have a probability of 0.3, 0.2, and 0.1 and the Top-P value is 0.5, then the model will select either A or B as the next token by using temperature and excludes C as a candidate.

Specify a lower value for less random responses and a higher value for more random responses. The default Top-P is 0.80.

Task 2. Different parameter values

You're ready to experiment with different parameter values.

1. In the Google Cloud console, from the Navigation menu (), select Vertex AI from the Artificial Intelligence section.

2. In the Tools pane on the left, click Language from the Vertex AI Studio section. Then, click Text Prompt on the Get Started page.

3. Enter the following prompt:

Summarize the beginning of Harry Potter for me.

4. In the Parameters section box, change Temperature from 0.9 to 1. Set region to us-central1. Click Submit. Give your prompt a name and Save it. Read the response to see how different temperature values impact the response.

5. In the Parameters section change Token limit from 2048 to 1024. Click Submit and Save. Read the response to see how different token limits impact the response.

6. In the Parameters section, click Advanced toggle options button to change Top-K value to 1. Click Submit and Save. Read the response to see how different Top-K values impact the response.

7. In the Parameters section change Top-P value to 1 click Submit and Save. Read the response to see how different Top-P values impact the response.

Click Check my progress to verify the objective.

Change the parameters and save your response

Task 3. Text summarization with generative models on Vertex AI

Extract house summaries of the most important information to streamline the search process.

1. In your JupyterLab Notebook, double-click to open housing_text_summarization.ipynb in the left panel.

2. Click Edit > Clear All Outputs.

3. Carefully read through the notebook instructions and run the code in each cell.

Run the cells in the housing_text_summarization.ipynb file.

Congratulations!

You have used PaLM API and AI Studio to create and experiment with prompts for various generative AI use cases. You also explored the AI Studio UI, and created text and code prompts with different parameters.

Next steps / Learn more

• To learn more about prompt design read the official documentation.

• View the notebooks used in this lab.

Google Cloud training and certification

...helps you make the most of Google Cloud technologies. Our classes include technical skills and best practices to help you get up to speed quickly and continue your learning journey. We offer fundamental to advanced level training, with on-demand, live, and virtual options to suit your busy schedule. Certifications help you validate and prove your skill and expertise in Google Cloud technologies.

Manual Last Updated March 05, 2024

Lab Last Tested March 05, 2024

Copyright 2024 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.