Skip to content

Metrics for Approver Policy Enterprise

The Approver Policy Enterprise component exposes operational and usage telemetry metrics suitable for popular monitoring solutions, enabling alerts on important operational states. The graphic below shows an example of a Grafana dashboard for Approver Policy Enterprise.

Grafana dashboard for Approver Policy Enterprise

All metrics formatted in Prometheus format are available at the standard /metrics path. They can accessed using the configured HTTP port of the controller component.

Enabling metrics

You need Approver Policy Enterprise version 0.12.0 or later to use Prometheus metrics. The metrics aren't enabled by default. The example configuration below enables the metrics, and specifies the interval at which metrics should be scraped.

To enable the metrics, follow the deployment instructions at deploy Approver Policy Enterprise and add the following to the Helm values file (approver-policy-enterprise.values.yaml):

cert-manager-approver-policy:
  app:
    metrics:
      service:
        enabled: true
        servicemonitor:
          enabled: true
          interval: 10s
          scrapeTimeout: 5s
          labels:
            instance: primary

Certificate request count metrics

These two metrics provide information on "orphan" certificate requests (certificate requests that haven't been matched to any certificate request policies), and when there is an increase in the number of denials (indicating a possible mis-configuration).

  • approverpolicy_certificaterequest_orphan_count: An enum gauge metric for "orphan" certificate requests.

    Labels

    • namespace: The namespace of the certificate request resource.
  • approverpolicy_certificaterequest_approved_count: An gauge metric for approved/denied certificate requests.

    Labels

    • namespace: The namespace of the certificate request resource.
    • approved_status: The approval status of the certificate request. Possible values are True, False.

Health of the Venafi Connection metrics

The Venafi Connection resource venaficonnection_status indicates if connection to the Venafi API is successful or not with respect to which service is using the connection.

Labels

  • name: The name of the Venafi Connection resource.
  • namespace: The name of the service that is using the connection.
  • cond_type: This string represents the name of the component for which cond_status applies. The possible values are VenafiEnhancedIssuerReady and ApproverPolicyVenafiReady.
  • cond_status: The possible values are True (the connection to the Venafi service is OK) or False (the connection or authentication is failing).

For example, the following line means that the Venafi Connection conn is being used by the Approver Policy Enterprise component, and that the connection between Approver Policy Enterprise and Venafi is failing:

venaficonnection_status{name="conn",namespace="venafi",ready_status="False",cond_type="ApproverPolicyVenafiReady"} 1

Alerts

Venafi Connection resources with the condition type VenafiEnhancedIssuerReady or ApproverPolicyEnterpriseReady equal to False must be investigated. This will alert you when a component is unable to connect to the Venafi API.

If you have jq installed, you can identify the component that is using the connection. Use the following command to list the Venafi Connection resources:

kubectl get venaficonnection --all-namespaces -o json \
  | jq -r '.items[] | select(.status.conditions[].status == "True") | "\(.metadata.namespace) \(.metadata.name)"'

Once you have identified the Venafi Connection resource that contain a False condition, you can debug the problem by displaying its events:

kubectl describe venaficonnection -n <namespace> <name>

Latency and status codes of the HTTP requests metrics

These two metrics represent the latency and status codes of the HTTP requests to Venafi Control Plane and Venafi TLS Protect Datacenter.

  • vcert_http_request_duration_seconds_bucket: A histogram metric indicating the durations of requests to Venafi Control Plane and Venafi TLS Protect Datacenter.

    Labels

    • method: The HTTP method (for example, POST) of the request.
    • path: The HTTP path of the request (for example, /outagedetection/v1/certificaterequests).
    • host: The HTTP Host header of the request (for example, api.venafi.cloud).
    • le: The time between the invocation of the HTTP request and the corresponding response.
  • vcert_http_requests_total: A counter metric indicating the total number of HTTP requests to Venafi Control Plane and Venafi TLS Protect Datacenter.

    Labels

    • method: The HTTP method for example, POST) of the request.
    • path: The HTTP path of the request (for example, /outagedetection/v1/certificaterequests).
    • host: The HTTP Host header of the request (for example, api.venafi.cloud).
    • code: The HTTP status code (for example, 200).

Alerts

If you notice a significant increase in failed HTTP requests (vcert_http_requests_total with code equal to or higher than 400), it's crucial to investigate the root cause of this issue.

The metric vcert_http_request_duration_seconds_bucket, when averaged, measures how fast Venafi Control Plane and Venafi TLS Protect Datacenter respond to requests from Venafi Enhanced Issuer. If you notice a significant increase in this metric, this requires investigation.

CPU and memory metrics

The CPU and memory metrics are provided by Kubernetes. The kubelet processes running on each Kubernetes node expose the endpoint /metrics/resources which, when scraped, provide the two following metrics (among others):

  • container_memory_usage_bytes
  • container_cpu_usage_seconds_total

Tip

If you are using Prometheus and have installed it through the kube-prometheus operator, the metrics are already available to you. To install the kube-prometheus operator, see the Getting Started tutorial.

Visualizing metrics

Venafi offers a sample Grafana dashboard in JSON format for you to import into your Grafana instance.

approver-policy-enterprise.grafana.json
{
  "__inputs": [
    {
      "name": "DS_GRAFANACLOUD-MAELVLS-PROM",
      "label": "grafanacloud-maelvls-prom",
      "description": "",
      "type": "datasource",
      "pluginId": "prometheus",
      "pluginName": "Prometheus"
    }
  ],
  "__elements": {},
  "__requires": [
    {
      "type": "grafana",
      "id": "grafana",
      "name": "Grafana",
      "version": "10.3.0-63839"
    },
    {
      "type": "datasource",
      "id": "prometheus",
      "name": "Prometheus",
      "version": "1.0.0"
    },
    {
      "type": "panel",
      "id": "stat",
      "name": "Stat",
      "version": ""
    },
    {
      "type": "panel",
      "id": "timeseries",
      "name": "Time series",
      "version": ""
    }
  ],
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": {
          "type": "datasource",
          "uid": "grafana"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "target": {
          "limit": 100,
          "matchAny": false,
          "tags": [],
          "type": "dashboard"
        },
        "type": "dashboard"
      }
    ]
  },
  "description": "",
  "editable": true,
  "fiscalYearStartMonth": 0,
  "gnetId": 15920,
  "graphTooltip": 0,
  "id": null,
  "links": [],
  "liveNow": false,
  "panels": [
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
      },
      "description": "",
      "fieldConfig": {
        "defaults": {
          "color": {
            "fixedColor": "green",
            "mode": "thresholds",
            "seriesBy": "min"
          },
          "mappings": [],
          "noValue": "All the certificate requests have been approved, all good!",
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 1
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 6,
        "w": 9,
        "x": 0,
        "y": 0
      },
      "id": 25,
      "options": {
        "colorMode": "value",
        "graphMode": "none",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.3.0-63839",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
          },
          "disableTextWrap": false,
          "editorMode": "code",
          "exemplar": false,
          "expr": "sum by(exported_namespace) (approverpolicy_certificaterequest_denied_count{cluster=\"$cluster\"})",
          "format": "time_series",
          "fullMetaSearch": false,
          "hide": false,
          "includeNullMetadata": true,
          "instant": false,
          "interval": "",
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Denied CertificateRequests",
      "transformations": [],
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
      },
      "description": "Count of CertificateRequests that can't be matched to any CertificateRequestPolicy. A CertificateRequest is said to be orphan when the Approved and Denied conditions are not set.",
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "noValue": "All the certificate requests have been matched, all good!",
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "text",
                "value": null
              },
              {
                "color": "red",
                "value": 1
              }
            ]
          },
          "unit": "short"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 6,
        "w": 9,
        "x": 9,
        "y": 0
      },
      "id": 14,
      "options": {
        "colorMode": "value",
        "graphMode": "none",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.3.0-63839",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum(approverpolicy_certificaterequest_unmatched_count{cluster=\"$cluster\"})",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "instant": false,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "CertificateRequests With No Matching Policy",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
      },
      "description": "",
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "noValue": "0",
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 1
              }
            ]
          },
          "unit": "short"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 6,
        "w": 6,
        "x": 18,
        "y": 0
      },
      "id": 27,
      "options": {
        "colorMode": "value",
        "graphMode": "none",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.3.0-63839",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum(venaficonnection_status{cluster=\"$cluster\", cond_type=\"ApproverPolicyVenafiReady\", cond_status!=\"True\"})",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "instant": false,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Failing VenafiConnections",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
      },
      "description": "",
      "fieldConfig": {
        "defaults": {
          "color": {
            "fixedColor": "red",
            "mode": "fixed",
            "seriesBy": "min"
          },
          "custom": {
            "axisBorderShow": false,
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 20,
            "gradientMode": "scheme",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "insertNulls": false,
            "lineInterpolation": "smooth",
            "lineWidth": 3,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "spanNulls": false,
            "stacking": {
              "group": "A",
              "mode": "normal"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "noValue": "All the certificate requests have been approved, all good!",
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 9,
        "x": 0,
        "y": 6
      },
      "id": 20,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "pluginVersion": "10.3.0-62736",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
          },
          "disableTextWrap": false,
          "editorMode": "code",
          "exemplar": false,
          "expr": "sum by(exported_namespace) (approverpolicy_certificaterequest_denied_count{cluster=\"$cluster\"})",
          "format": "time_series",
          "fullMetaSearch": false,
          "hide": false,
          "includeNullMetadata": true,
          "instant": false,
          "interval": "",
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Denied CertificateRequests By Namespace",
      "transformations": [],
      "type": "timeseries"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
      },
      "description": "Count of CertificateRequests that can't be matched to any CertificateRequestPolicy. A CertificateRequest is said to be orphan when the Approved and Denied conditions are not set.",
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "custom": {
            "axisBorderShow": false,
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 25,
            "gradientMode": "none",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "insertNulls": false,
            "lineInterpolation": "linear",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "spanNulls": false,
            "stacking": {
              "group": "A",
              "mode": "normal"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "noValue": "All the certificate requests have been matched, all good!",
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 1
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 9,
        "x": 9,
        "y": 6
      },
      "id": 26,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "pluginVersion": "10.3.0-62736",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(exported_namespace) (approverpolicy_certificaterequest_unmatched_count{cluster=\"$cluster\"})",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "instant": false,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "CertificateRequests With No Matching Policy By Namespace",
      "type": "timeseries"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "#ccccdb1a",
                "value": null
              },
              {
                "color": "green",
                "value": 0.1
              },
              {
                "color": "yellow",
                "value": 1
              },
              {
                "color": "red",
                "value": 5
              }
            ]
          },
          "unit": "s"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 6,
        "x": 18,
        "y": 6
      },
      "id": 15,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "value_and_name",
        "wideLayout": true
      },
      "pluginVersion": "10.3.0-63839",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
          },
          "disableTextWrap": false,
          "editorMode": "code",
          "expr": "histogram_quantile(0.95, sum by(le, method) (rate(vcert_http_request_duration_seconds_bucket{cluster=\"$cluster\",job=\"$job\"}[$__rate_interval])))",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "instant": false,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "P95 latency of HTTP calls to Venafi",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
      },
      "description": "CPU Usage and limits, as percent of a vCPU core. ",
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisBorderShow": false,
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 0,
            "gradientMode": "none",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "insertNulls": false,
            "lineInterpolation": "linear",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "never",
            "spanNulls": false,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "links": [],
          "mappings": [],
          "min": 0,
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          },
          "unit": "percentunit"
        },
        "overrides": [
          {
            "matcher": {
              "id": "byName",
              "options": "max"
            },
            "properties": [
              {
                "id": "color",
                "value": {
                  "fixedColor": "dark-yellow",
                  "mode": "fixed"
                }
              }
            ]
          },
          {
            "matcher": {
              "id": "byName",
              "options": "CPU"
            },
            "properties": [
              {
                "id": "custom.fillOpacity",
                "value": 10
              },
              {
                "id": "custom.fillOpacity",
                "value": 50
              }
            ]
          },
          {
            "matcher": {
              "id": "byRegexp",
              "options": "/Request.*/"
            },
            "properties": [
              {
                "id": "color",
                "value": {
                  "fixedColor": "#FF9830",
                  "mode": "fixed"
                }
              },
              {
                "id": "custom.lineStyle",
                "value": {
                  "dash": [
                    10,
                    10
                  ],
                  "fill": "dash"
                }
              }
            ]
          },
          {
            "matcher": {
              "id": "byRegexp",
              "options": "/Limit.*/"
            },
            "properties": [
              {
                "id": "color",
                "value": {
                  "fixedColor": "#F2495C",
                  "mode": "fixed"
                }
              },
              {
                "id": "custom.lineStyle",
                "value": {
                  "dash": [
                    10,
                    10
                  ],
                  "fill": "dash"
                }
              }
            ]
          }
        ]
      },
      "gridPos": {
        "h": 8,
        "w": 8,
        "x": 0,
        "y": 14
      },
      "id": 24,
      "interval": "1m",
      "links": [],
      "maxDataPoints": 250,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "multi",
          "sort": "none"
        }
      },
      "pluginVersion": "10.3.0-62411",
      "targets": [
        {
          "datasource": {
            "uid": "$datasource"
          },
          "editorMode": "code",
          "expr": "avg by (pod) (rate(container_cpu_usage_seconds_total{container=\"cert-manager-approver-policy\", cluster=\"$cluster\"}[$__rate_interval]))",
          "format": "time_series",
          "hide": false,
          "interval": "",
          "intervalFactor": 2,
          "legendFormat": "CPU {{ pod }}",
          "range": true,
          "refId": "A"
        },
        {
          "datasource": {
            "uid": "$datasource"
          },
          "editorMode": "code",
          "expr": "avg by (pod) (kube_pod_container_resource_limits_cpu_cores{container=\"cert-manager-approver-policy\", cluster=\"$cluster\"})",
          "format": "time_series",
          "hide": true,
          "interval": "",
          "intervalFactor": 1,
          "legendFormat": "Limit {{ pod }}",
          "range": true,
          "refId": "B"
        },
        {
          "datasource": {
            "uid": "$datasource"
          },
          "editorMode": "code",
          "expr": "avg by (pod) (kube_pod_container_resource_requests_cpu_cores{container=\"cert-manager-approver-policy\", cluster=\"$cluster\"})",
          "format": "time_series",
          "hide": true,
          "interval": "",
          "intervalFactor": 1,
          "legendFormat": "Request {{ pod }}",
          "range": true,
          "refId": "C"
        }
      ],
      "title": "CPU",
      "type": "timeseries"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
      },
      "description": "Memory utilisation and limits.",
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisBorderShow": false,
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 0,
            "gradientMode": "none",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "insertNulls": false,
            "lineInterpolation": "linear",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "never",
            "spanNulls": false,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "links": [],
          "mappings": [],
          "min": 0,
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          },
          "unit": "bytes"
        },
        "overrides": [
          {
            "matcher": {
              "id": "byName",
              "options": "max"
            },
            "properties": [
              {
                "id": "color",
                "value": {
                  "fixedColor": "dark-yellow",
                  "mode": "fixed"
                }
              }
            ]
          },
          {
            "matcher": {
              "id": "byName",
              "options": "Memory"
            },
            "properties": [
              {
                "id": "custom.fillOpacity",
                "value": 10
              },
              {
                "id": "custom.fillOpacity",
                "value": 50
              }
            ]
          },
          {
            "matcher": {
              "id": "byName",
              "options": "Request"
            },
            "properties": [
              {
                "id": "color",
                "value": {
                  "fixedColor": "#FF9830",
                  "mode": "fixed"
                }
              },
              {
                "id": "custom.lineStyle",
                "value": {
                  "dash": [
                    10,
                    10
                  ],
                  "fill": "dash"
                }
              }
            ]
          },
          {
            "matcher": {
              "id": "byName",
              "options": "Limit"
            },
            "properties": [
              {
                "id": "color",
                "value": {
                  "fixedColor": "#F2495C",
                  "mode": "fixed"
                }
              },
              {
                "id": "custom.lineStyle",
                "value": {
                  "dash": [
                    10,
                    10
                  ],
                  "fill": "dash"
                }
              }
            ]
          }
        ]
      },
      "gridPos": {
        "h": 8,
        "w": 8,
        "x": 8,
        "y": 14
      },
      "id": 23,
      "interval": "1m",
      "links": [],
      "maxDataPoints": 250,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "multi",
          "sort": "none"
        }
      },
      "pluginVersion": "10.3.0-62411",
      "targets": [
        {
          "datasource": {
            "uid": "$datasource"
          },
          "editorMode": "code",
          "expr": "avg by (pod) (container_memory_usage_bytes{container=\"cert-manager-approver-policy\", cluster=\"$cluster\"})",
          "format": "time_series",
          "hide": false,
          "interval": "",
          "intervalFactor": 1,
          "legendFormat": "Memory {{ pod }}",
          "range": true,
          "refId": "A"
        },
        {
          "datasource": {
            "uid": "$datasource"
          },
          "editorMode": "code",
          "expr": "avg by (pod) (kube_pod_container_resource_limits_memory_bytes{container=\"cert-manager-approver-policy\", cluster=\"$cluster\"})",
          "format": "time_series",
          "interval": "",
          "intervalFactor": 1,
          "legendFormat": "Limit {{ pod }}",
          "range": true,
          "refId": "B"
        },
        {
          "datasource": {
            "uid": "$datasource"
          },
          "editorMode": "code",
          "expr": "avg by (pod) (kube_pod_container_resource_requests_memory_bytes{container=\"cert-manager-approver-policy\", cluster=\"$cluster\"})",
          "format": "time_series",
          "interval": "",
          "intervalFactor": 1,
          "legendFormat": "Request {{ pod }}",
          "range": true,
          "refId": "C"
        }
      ],
      "title": "Memory",
      "type": "timeseries"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
      },
      "description": "",
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisBorderShow": false,
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 0,
            "gradientMode": "none",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "insertNulls": false,
            "lineInterpolation": "smooth",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "spanNulls": false,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          },
          "unit": "reqps"
        },
        "overrides": [
          {
            "__systemRef": "hideSeriesFrom",
            "matcher": {
              "id": "byNames",
              "options": {
                "mode": "exclude",
                "names": [
                  "200"
                ],
                "prefix": "All except:",
                "readOnly": true
              }
            },
            "properties": [
              {
                "id": "custom.hideFrom",
                "value": {
                  "legend": false,
                  "tooltip": false,
                  "viz": true
                }
              }
            ]
          }
        ]
      },
      "gridPos": {
        "h": 8,
        "w": 8,
        "x": 16,
        "y": 14
      },
      "id": 16,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "asc"
        }
      },
      "pluginVersion": "10.2.0-61298",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
          },
          "disableTextWrap": false,
          "editorMode": "code",
          "expr": "sum by (code) (\n  rate(vcert_http_requests_total{cluster=\"$cluster\"}[$__rate_interval])\n)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "instant": false,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Number of HTTP calls to Venafi",
      "type": "timeseries"
    }
  ],
  "refresh": "5s",
  "schemaVersion": 39,
  "tags": [],
  "templating": {
    "list": [
      {
        "current": {},
        "datasource": {
          "type": "prometheus",
          "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
        },
        "definition": "label_values(venafienhancedissuer_info,cluster)",
        "hide": 0,
        "includeAll": false,
        "multi": false,
        "name": "cluster",
        "options": [],
        "query": {
          "qryType": 1,
          "query": "label_values(venafienhancedissuer_info,cluster)",
          "refId": "PrometheusVariableQueryEditor-VariableQuery"
        },
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      },
      {
        "current": {},
        "datasource": {
          "type": "prometheus",
          "uid": "${DS_GRAFANACLOUD-MAELVLS-PROM}"
        },
        "definition": "label_values(approverpolicy_certificaterequest_approved_count,job)",
        "hide": 0,
        "includeAll": false,
        "multi": false,
        "name": "job",
        "options": [],
        "query": {
          "qryType": 1,
          "query": "label_values(approverpolicy_certificaterequest_approved_count,job)",
          "refId": "PrometheusVariableQueryEditor-VariableQuery"
        },
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      },
      {
        "current": {
          "selected": false,
          "text": "default",
          "value": "default"
        },
        "hide": 0,
        "includeAll": false,
        "multi": false,
        "name": "datasource",
        "options": [],
        "query": "prometheus",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "type": "datasource"
      }
    ]
  },
  "time": {
    "from": "now-1h",
    "to": "now"
  },
  "timepicker": {
    "refresh_intervals": [
      "5s",
      "10s",
      "30s",
      "1m",
      "5m",
      "15m",
      "30m",
      "1h",
      "2h",
      "1d"
    ]
  },
  "timezone": "",
  "title": "Approver Policy Enterprise",
  "uid": "a1831fd0-63a4-40ff-a463-fcd63b56103a",
  "version": 12,
  "weekStart": ""
}