TallyScrapeJobMissing
absent(up{job="reporting-api"}) or absent(up{job="otel-collector"}) or absent(up{job="openstack-collector"}), for: 5m, in scrape-rules.yaml, which an overlay replaces together with its scrape config. The expression above is the base's.
Symptom
A discovered scrape job resolves to no target at all. up is a per-target series, so zero targets produce no up series rather than up == 0: the job disappears from /targets instead of turning red, and TallyScrapeTargetDown stays silent.
Impact on billing
The same as the job's targets being down, described in TallyScrapeTargetDown, without a red target to see it by.
First checks
- The Deployment's replica count. A Deployment scaled to zero takes its job off the page.
- The Service name and the port name the relabel rule keeps,
reporting-api;http,otel-collector;metricsandopenstack-collector;httpinscrape.yaml. A rename on either side drops every endpoint of the job. - The
victoriametrics-scrapeRoleBinding invictoriametrics.yaml. Without it the endpointslice discovery is refused and the job has nothing to keep. - The VictoriaMetrics log for discovery errors, an unreachable API server among them.
- When the firing job is one the deployment does not scrape, compare the overlay's
scrape-rules.yamlwith itsscrape.yaml. A clause left over from the base names a job the overlay never configured, and it fires for as long as the cluster runs. The overlay's file carries oneabsentclause per job of itsscrape.yamlthat haskubernetes_sd_configs, and nothing else.