Overview

Google Cloud has a number of compute-related resources including Compute Engine, Kubernetes, and Cloud Run. In this exercise, you install the logging and monitoring agents into a VM running a Nginx server to maximize the metrics to easily view. You also set up a Kubernetes cluster and monitor an application deployed into it.

Objectives

In this lab, you learn how to perform the following tasks:

• Set up a VM and a GKE cluster.
• Install and use the logging and monitoring agents for Compute Engine.
• Add a service to the GKE cluster and examine its logs and metrics.

Setup

Access the lab

For each lab, you get a new Google Cloud project and set of resources for a fixed time at no cost.

1. Sign in to Qwiklabs using an incognito window.

2. Note the lab's access time (for example, 1:15:00), and make sure you can finish within that time.
   There is no pause feature. You can restart if needed, but you have to start at the beginning.

3. When ready, click Start lab.

4. Note your lab credentials (Username and Password). You will use them to sign in to the Google Cloud Console.

5. Click Open Google Console.

6. Click Use another account and copy/paste credentials for this lab into the prompts.
   If you use other credentials, you'll receive errors or incur charges.

7. Accept the terms and skip the recovery resource page.

After you complete the initial sign-in steps, the project dashboard appears.

Activate Google Cloud Shell

Google Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud.

Google Cloud Shell provides command-line access to your Google Cloud resources.

1. In Cloud console, on the top right toolbar, click the Open Cloud Shell button.

   

2. Click Continue.

It takes a few moments to provision and connect to the environment. When you are connected, you are already authenticated, and the project is set to your PROJECT_ID. For example:

gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.

• You can list the active account name with this command:

gcloud auth list

Output:

Credentialed accounts: - @.com (active)

Example output:

Credentialed accounts: - google1623327_student@qwiklabs.net

• You can list the project ID with this command:

gcloud config list project

Output:

[core] project =

Example output:

[core] project = qwiklabs-gcp-44776a13dea667a6 Note: Full documentation of gcloud is available in the gcloud CLI overview guide .

Task 1. Set up a VM and a GKE cluster

In this task, you:

• Set up a VM running Nginx.
• Create a GKE cluster.

Set up a VM to host Nginx

1. In the Cloud Console, select Navigation menu > Compute Engine > VM instances, then click Create Instance.

2. Set the following fields:

• Name: web-server-vm
• Region:
• Zone:

3. Click OS and storage.

4. If the Image shown is not Debian GNU/Linux 11 (bullseye), click Change and select Debian GNU/Linux 11 (bullseye), and then click Select.

5. Click Networking.

6. For Firewall, click Allow HTTP traffic.

7. Leave the rest of the fields at their default values and click Create.

Create a GKE cluster

To explore a little of what Google Kubernetes Engine offers in the way of logging and monitoring, let's add a cluster, which will come with logging and monitoring enabled by default.

1. Use the Navigation menu to navigate to the Kubernetes Engine > Clusters page.

2. Click Create and then click on the Switch to Standard Cluster. Confirm Switch to Standard Cluster.

3. Set the Name to gke-cluster and Zone to , then under the Cluster settings on the left, select Features.

4. Verify that Enable Logging is checked and dropbox is set to System and Workloads also verify Enable Cloud Monitoring is checked and System is selected by default in the dropdown, and then press Create.

5. Once you see the cluster creation start, move on to the next step.

Install Nginx

1. Switch back to Compute Engine and SSH into the web-server-vm.

2. Use APT to install Nginx:

sudo apt-get update sudo apt-get install nginx

3. Enter Y when prompted.

4. Verify that Nginx installed and is running:

sudo nginx -v

5. Switch back to Compute Engine and if you enabled the HTTP access firewall rule, then you should be able to click the external IP. Do so and make sure the Nginx server is up and running.

6. Copy the URL to the server.

7. Change to the Cloud Shell terminal window.

8. Create a URL environmental variable and set it equal to the URL to your server:

URL=URL_to_your_server

9. Use a bash while loop to place some load on the server. Make sure you are seeing the Welcome to nginx responses:

while true; do curl -s $URL | grep -oP "<title>.*</title>"; \ sleep .1s;done

Task 2. Install and use the logging and monitoring agents for Compute Engine

In this task, you:

• Examine the logs and see some data that isn't included.
• Install the logging and monitoring agents in the Compute Engine instance.
• Re-examine the logs to see the new data.

Examine the logs and see some data that isn't included

A standard Debian Linux image from Google will not have the logging or monitoring agents installed, so you won't have access to metrics from googleapis.com/nginx/ such as:

• connections/accepted_count
• connections/current
• connections/handled_count
• request_count

1. Switch to the Google Console window and navigate to Monitoring > Overview.

2. Wait for the monitoring workspace to create, and then navigate to Metrics explorer.

3. Click on Select a metric dropdown and select VM Instance > Instance > CPU utilization.

4. Click Apply, and set the Filter to instance_name = web-server-vm.

5. You may not see a spike when you apply the load, since it's not a big load, but you are able to access the metric.

6. Delete the Filter and the Metric.

7. Click on Select a metric dropdown and deselect Active.

8. Select VM Instance > nginx > Requests. Click Apply.

Why is there no data?

Install the logging and monitoring agents in the Compute Engine instance

The data for Nginx requests is missing because without the logging and monitoring agents being installed, the best Google Cloud can do is black-box monitoring. If you want to see more details, then you need to enable white-box monitoring by installing the agents.

1. Use the Navigation menu to navigate to Compute Engine > VM instances.

2. Click the SSH link for the web-server-vm.

3. Check to see if the logging agent is installed/running:

sudo service google-fluentd status sudo service stackdriver-agent status

Not surprisingly, the services could not be found.

4. Check to make sure you have the requisite scopes to perform logging and monitoring.

curl --silent --connect-timeout 1 -f -H "Metadata-Flavor: Google" \ http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/scopes

5. Note in the response the logging.write and monitoring.write scopes.

6. Download the script, add the monitoring agent repo to apt, and install the agent.

curl -sSO https://dl.google.com/cloudagents/add-monitoring-agent-repo.sh sudo bash add-monitoring-agent-repo.sh --also-install

7. Start the monitoring agent:

sudo service stackdriver-agent start

8. Install the logging agent:

curl -sSO https://dl.google.com/cloudagents/add-logging-agent-repo.sh sudo bash add-logging-agent-repo.sh --also-install

9. Retest the two agents again and verify they are both active. If you see any "can not take infinite value" warnings, ignore them and if the status is active (running), press CTRL+C and move on to the next step:

sudo service google-fluentd status sudo service stackdriver-agent status

10. To fully integrate the server, you enable the status information handler in Nginx by adding a configuration file to the Nginx configuration directory:

(cd /etc/nginx/conf.d/ && sudo curl -O https://raw.githubusercontent.com/Stackdriver/stackdriver-agent-service-configs/master/etc/nginx/conf.d/status.conf)

11. Reload the Nginx service:

sudo service nginx reload

12. Enable the Nginx monitoring plugin:

(cd /opt/stackdriver/collectd/etc/collectd.d/ && sudo curl -O https://raw.githubusercontent.com/Stackdriver/stackdriver-agent-service-configs/master/etc/collectd.d/nginx.conf)

13. Restart the monitoring agent:

sudo service stackdriver-agent restart

Re-examine the metrics to see the new data

Now that the agents are installed and the Nginx plugins have been added, retest the Nginx metric that we examined earlier.

1. Check the Cloud Shell window and verify that the test loop is still running, and that you are still receiving responses. If not, reset the URL property and restart the while.

2. Navigate to the Monitoring > Metrics explorer.

3. Click on Select a metric dropdown and select VM Instance > nginx > Requests, then click Apply.

You may have to wait a few minutes and refresh the page, but you should soon see metric data coming in from Nginx. The monitoring agent is working as it should.

Task 3. Add a service to the GKE cluster and examine its logs and metrics

You created a GKE cluster with logging and monitoring enabled earlier in this exercise. Now, load the HelloLoggingNodeJS application to it, put it behind a load balancer service, and view some metrics for your cluster.

1. Switch to or reopen the Cloud Shell console.

2. Break the running test loop with CTRL+C.

3. Enable the Cloud Build API as it is needed in a few steps:

gcloud services enable cloudbuild.googleapis.com

4. Clone the https://github.com/haggman/HelloLoggingNodeJS.git repo:

git clone https://github.com/haggman/HelloLoggingNodeJS.git

This repository contains a basic Node.js web application that is used for testing.

5. Change into the HelloLoggingNodeJS folder and open the index.js in the Cloud Shell editor:

cd HelloLoggingNodeJS edit index.js

6. Take a few minutes to peruse the code.

7. In the editor, also take a look at the buildContainer.sh file, package.json file which contains the dependencies, and the Dockerfile which plans the Docker container we generate and deploy to GKE.

8. Executing the file buildContainer.sh to submit the Dockerfile by Google's Cloud Build to generate a container and store it in your Artifact Registry:

chmod +x buildContainer.sh ./buildContainer.sh

9. In the k8sapp.yaml file, replace the $PROJECT with your actual project ID. Remember, your project ID is located on the Homepage of your Google Cloud Console and in Qwiklabs just below your temporary Google Cloud password.

10. Open the k8sapp.yaml file and explore the instructions to Kubernetes to create three hello-logging pods, and then expose them through a LoadBalancer service.

11. Use the Navigation menu to navigate to Kubernetes Engine, and click on the triple dot icon for your cluster and select Connect.

12. Copy the command line to configure kubectl.

13. Switch back to Cloud Shell and execute the command.

14. Use kubectl to apply your k8sapp.yaml.

kubectl apply -f k8sapp.yaml

15. Get a list of your services. It may take a minute or two for the new hello-logging-service to appear. Wait for it to get an External IP:

kubectl get services

16. Once the service appears with the external IP, copy the external IP value.

17. Open a tab in the browser and paste in the IP. Make sure you see the Hello World message.

Note: If prompted The connection is not secure click Continue to site

18. Copy the page URL from the browser and switch back to Cloud Shell. Update the URL environmental variable and restart the while loop. Make sure you are seeing the Hello World responses:

URL=url_to_k8s_app while true; do curl -s $URL -w "\n"; sleep .1s;done

19. Navigate to the Monitoring > Dashboards and open the GKE dashboard.

20. Click on Configuration icon next to VIEW ALL and enable Sparklines and click Apply.

21. On the GKE dashboard, for gke-cluster scroll horizontally and see the charts. If all of the small charts are reading No Data, refresh the page until they start showing readings for CPU and Memory Utilization. You might also want to toggle Off to On to enable auto-refresh of the data.

If you want to examine the different cluster nodes, you can scroll horizontally in Nodes tab and see that the three hello-logging-js-deployments are spread across the nodes. This tab focuses on what's running where, from Cluster, to Node, to Pod.

22. Switch to the Workloads tab. This is focused on the deployed workloads, grouped by namespace.

23. Finally, scroll to the Kubernetes Services tab and expand hello-logging-service. This view is more about how services relate to their pods.

In any of the views, if you drill all the way down to one of our hello-logging-js-containers, a right-hand window appears with details. Investigate Incidents, Metrics, and Logs.

Congratulations!

In this exercise, you installed and used logging and monitoring agents on Compute Engine instances, and explored logging and monitoring in a GKE cluster. Nice job.

End your lab

When you have completed your lab, click End Lab. Google Cloud Skills Boost removes the resources you’ve used and cleans the account for you.

You will be given an opportunity to rate the lab experience. Select the applicable number of stars, type a comment, and then click Submit.

The number of stars indicates the following:

• 1 star = Very dissatisfied
• 2 stars = Dissatisfied
• 3 stars = Neutral
• 4 stars = Satisfied
• 5 stars = Very satisfied

You can close the dialog box if you don't want to provide feedback.

For feedback, suggestions, or corrections, please use the Support tab.

Copyright 2022 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.