Verify prometheus has been deployed
Verify metric filter is applied
Verify metric filter is applied
Reduce Costs for the Managed Service for Prometheus
In this lab, you will explore cost control mechanisms when utilizing the Managed Service for Prometheus on Google Cloud.
The Google Cloud Managed Service for Prometheus charges for the number of samples ingested into Cloud Monitoring and for read requests to the Monitoring API. The number of samples ingested is the primary contributor to your cost.
In this lab, you will learn how to:
Understand how to reduce the costs associated with using the Managed Service for Prometheus
Deploy GMP in a GKE cluster as well as a python application
Reduce the number of time series metrics you send to the managed service by filtering the metric data generated
Reduce the number of samples collected by changing the scraping interval
Task 1. Deploy GKE cluster
Deploy a basic GKE cluster to setup lab
Task 2. Deploy managed collection
Configure a PodMonitoring resource
The following manifest defines a PodMonitoring resource,
prom-example, in the
gmp-test namespace. The resource uses a Kubernetes label selector to find all pods in the namespace that have the label
app with the value
prom-example. The matching pods are scraped on a port named
metrics, every 30 seconds, on the
/metrics HTTP path.
To apply this resource, run the following command:
Your managed collector is now scraping the matching pods.
Deploy the example application
The managed service provides a manifest for an example application that emits Prometheus metrics on its metrics port. The application uses three replicas.
To deploy the example application, run the following command:
Task 3. Cloud Console for Monitoring
To view your Managed Service for Prometheus data as Cloud Monitoring time series, you can use Metrics Explorer. To configure Metrics Explorer to display metrics, do the following:
In the Google Cloud Console, go to Monitoring.
In the Monitoring navigation pane, click Metrics Explorer.
Specify the data to appear on the chart. You can use the MQL tab or the Configuration tab.
To use the MQL tab, do the following:
a. Select the MQL tab.
b. Enter the following query:
c. Click Run Query.
To use the Configuration tab, do the following:
a. Select the Configuration tab.
b. In the Resource & Metric field, type "up/" to filter the list, then select Prometheus Target > Up > prometheus/up/gauge.
c. Then select Apply.
Task 4. Populates a graph
Go to Monitoring > Managed Prometheus > PromQL Query and run a query to see metrics: “go_memstats_heap_alloc_bytes” is a decent one that populates a graph nicely.
Task 5. Filter exported metrics
If you collect a lot of data, you might want to prevent some time series from being sent to Managed Service for Prometheus to keep costs down.
To filter exported metrics, you can configure a set of PromQL series selectors in the OperatorConfig resource. A time series is exported to Managed Service for Prometheus if it satisfies at least one of the selectors.
Open the OperatorConfig resource for editing:
After the apiVersion line. Press "i" to go into insert mode. Go to the final line and press enter to go to a newline. Ensure there are no indents at the beginning of the line. Then paste the following:
The file should look like the following
To save the file and exit press "Esc" then type ":wq" then enter
This addition causes only metrics for the "prometheus" job as well as metrics produced by recording rules that aggregate to the job level—when following naming best practices—to be exported. Samples for all other time series are filtered out. By default, no selectors are specified and all time series are exported.
filter.matchOneOf configuration section has the same semantics as the match parameters for Prometheus federation.
Create a config.yaml file
Copy the following into the config.yaml file
Upload the config file you created to verify.
Task 6. Run the query
Go back to the PromQL Query editor in the UI and run the “up” query. Notice that no other metrics except for prom-example show up in the results. This will be the case with other metrics as well.
Task 7. Monitor the app
Increase the scrape interval for the PodMonitoring resource created to monitor the example app:
Change the interval in the file to 60s from 30s.
Create a prom-example-config.yaml file you created to verify.
Copy the following into the file
Task 8. Complete the lab
This completes the lab. Should have a statement at the end of the lab saying that filtering metrics or increasing the scrape interval are two methods to reduce the amount of metrics stored and therefore will reduce billing for the service.
In this lab, you deployed prometheus and an example application. Then, you applied a metrics filter and changed the scraping interval.
Finish your quest
This self-paced lab is part of the Monitor Environments with Google Cloud managed Service for Prometheus skill badge quest. A quest is a series of related labs that form a learning path. Completing this quest will earn you a badge to recognize your achievement.
Next Steps / Learn More
You can read more about Google Cloud Managed Service for Prometheus.
Google Cloud training and certification
...helps you make the most of Google Cloud technologies. Our classes include technical skills and best practices to help you get up to speed quickly and continue your learning journey. We offer fundamental to advanced level training, with on-demand, live, and virtual options to suit your busy schedule. Certifications help you validate and prove your skill and expertise in Google Cloud technologies.
Manual Last Updated: June 21, 2022
Lab Last Tested: June 2, 2022
Copyright 2022 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.