---
title: Prometheus Configuration
category: product
entity_type: skill
price: $15
canonical: https://forgehouse.ai/skills/prometheus-configuration/
lang: en
hreflang_alt: https://forgehouse.ai/tr/skiller/prometheus-configuration/
last_updated: 2026-06-20
---

# Prometheus Configuration

> Set up Prometheus for comprehensive metric collection, storage, and monitoring of…

Set up Prometheus end to end for metric collection, scraping, recording rules, and alerting across infrastructure and applications. It delivers production-ready scrape configs, service discovery for Kubernetes, pre-computed recording rules, and severity-tuned alert rules backed by the USE method and cardinality control. The goal is meaningful alerts that engineers act on, not dashboards that drown teams in noise.

## Use cases
- Standing up Prometheus monitoring from scratch
- Kubernetes pod and service discovery scraping
- Recording rules for expensive queries
- USE-method resource alerting (CPU, memory, disk)
- Severity-tiered alert routing to PagerDuty and Slack
- Validating config and rules before deploy

## Benefits
- Faster diagnosis with USE-based, resource-specific alerts
- Lower memory and query cost through cardinality control
- Reduced on-call fatigue via actionable, tuned severities
- Config drift prevented with Git-managed single source of truth

## What’s included
- Complete prometheus.yml with global, alerting, and scrape sections
- Static, file-based, and Kubernetes service discovery configs
- Recording rules for request rate, error rate, and P95 latency
- Availability and resource alert rule sets with runbook annotations
- relabel_configs patterns to drop high-cardinality labels
- promtool validation commands and troubleshooting endpoints

## Who it’s for
DevOps and SRE teams building observability infrastructure that surfaces real problems without alert overload.

## How it runs
Monitoring fails in two ways: it misses the incident, or it pages you for nothing. This Prometheus setup controls cardinality at the door, pre-computes expensive queries, and writes alerts humans can live with.
1. Stand up the server with sizing decided up front: kube-prometheus-stack via Helm or Docker Compose, retention and storage volume set on day one, not after the disk fills.
2. Write prometheus.yml deliberately: 15s scrape and evaluation intervals, external labels for cluster and region, static targets plus Kubernetes service discovery gated by scrape annotations.
3. Control cardinality at the door: relabel_configs drop high-cardinality labels like user_id and request_id before scrape, and prometheus_tsdb_head_series is watched for series explosions.
4. Pre-compute the expensive queries as recording rules: per-job request rates, error percentage, p95 latency and per-node USE metrics, so dashboards never recompute them live.
5. Write alert rules that respect humans: minimum for: 5m to filter spikes, severity tiers (critical pages, warning goes to chat, info stays on the dashboard) and a runbook link in every annotation.
6. Gate every change in CI: promtool check on config and rules, all configuration versioned in git, hand-editing on the server forbidden, and a reload-failure metric alerting when config does not apply.

## FAQ
### We are not on Kubernetes, is the service discovery still relevant?
Yes, Kubernetes discovery is one option among several: the configs cover static targets and file-based discovery too. The scrape, recording-rule, and alerting patterns apply to any infrastructure Prometheus can reach.

### How does it keep alerts from becoming the usual on-call noise?
Alert rules follow the USE method (utilization, saturation, errors) per resource, carry severity tiers routed differently to PagerDuty and Slack, and include runbook annotations so each page is actionable. Recording rules pre-compute expensive queries so dashboards stay fast without firing on raw noise.

### Does it include Grafana dashboards or long-term metric storage?
No. The scope is Prometheus itself: prometheus.yml, scrape configs, recording rules, alert rules, cardinality control, and promtool validation. Visualization and remote long-term storage are separate concerns you add on top.

## Price
$15, one-time, no subscription. VAT included.

Related guide: [How to run a marketing agency with AI automation](https://forgehouse.ai/guides/ai-marketing-agency-automation/)