collector

package
v0.6.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 24, 2024 License: Apache-2.0 Imports: 41 Imported by: 0

Documentation

Overview

Package collector implements the metrics-prometheus-collector which exports metrics to prometheus when scraped.

Collector reads utilization metrics from the metrics-node-sampler, and other metrics directly from the Kubernetes objects.

Collector is highly configurable with respect to how metrics are labeled and aggregated at the time they are collected.

Package labels is responsible for parsing metric labels from Kubernetes objects

Index

Constants

View Source 
const (
	// MemoryRound defines infinite decimal precision for memory
	MemoryRound = 0
	// CPUUSecRound defines infinite decimal precision for cpu usage seconds
	CPUUSecRound = 6
	// CPUSecRound defines infinite decimal precision for cpu seconds
	CPUSecRound = 1
	// AveragePrecision defines infinite decimal precision for averages
	AveragePrecision = 6
)
View Source 
const MaxExtensionLabels = collectorcontrollerv1alpha1.MaxExtensionLabels
View Source 
const (
	Undefined = "undefined"
)

Variables

View Source 
var InvalidContainerUtilizationInitialLevelOperations = sets.NewString("sum", "")
View Source 
var (
	KeepAnnotations = []string{
		"workload-name", "workload-kind", "workload-api-group", "workload-api-version",
		"kubernetes.io/hostname", corev1.MirrorPodAnnotationKey,
	}
)
View Source 
var LabelOverrides []LabelOverrider

LabelOverrides may be appended to in order to explicitly override values To introduce a new label and override it, use an extension to create the label and then override it by appending to this list.

Functions

func CollectorFuncs added in v0.6.2 

func CollectorFuncs() map[string]CollectorFunc

CollectorFuncs returns a copy of the map holding all the collector functions. This function is safe to be used concurrently.

func GetNewCacheFunc 

func GetNewCacheFunc(c collectorcontrollerv1alpha1.MetricsPrometheusCollector) cache.NewCacheFunc

GetNewCacheFunc returns a function which sets informer cache options to reduce memory consumption. * Delete labels and annotations that aren't used by the collector * Don't deep copy objects when reading from the cache

func ProtoToJSON added in v0.2.0 

func ProtoToJSON(s *collectorapi.SampleList, b *bytes.Buffer) error

func SetCollectorFuncs added in v0.6.2 

func SetCollectorFuncs(fns map[string]CollectorFunc)

SetCollectorFuncs copies the given map and sets the current collector functions. This function is safe to be used concurrently.

Types

type CapacityObjects added in v0.6.2 

type CapacityObjects struct {
	Pods       corev1.PodList
	Nodes      corev1.NodeList
	Namespaces corev1.NamespaceList
	Quotas     corev1.ResourceQuotaList
	RQDs       quotamanagementv1alpha1.ResourceQuotaDescriptorList
	PVCs       corev1.PersistentVolumeClaimList
	PVs        corev1.PersistentVolumeList
	Samplers   corev1.PodList

	NamespacesByName map[string]*corev1.Namespace
	PodsByNodeName   map[string][]*corev1.Pod
	PodsByPVC        map[string][]*corev1.Pod
	PVsByName        map[string]*corev1.PersistentVolume
	PVCsByName       map[string]*corev1.PersistentVolumeClaim
	NodesByName      map[string]*corev1.Node
	RQDsByRQDKey     map[ResourceQuotaDescriptorKey]*quotamanagementv1alpha1.ResourceQuotaDescriptor
	SamplersByNode   map[string]*corev1.Pod

	ClusterScopedByName map[collectorcontrollerv1alpha1.Source]*metav1.PartialObjectMetadataList

	UtilizationByNode map[string]*samplerapi.ListMetricsResponse
}

CapacityObjects are the CapacityObjects used for computing metrics

type Collector 

type Collector struct {
	IsLeaderElected *atomic.Bool

	client.Client
	Labeler Labeler
	Reader  ValueReader

	UtilizationServer utilization.Server
	*collectorcontrollerv1alpha1.MetricsPrometheusCollector
	// contains filtered or unexported fields
}

func NewCollector 

func NewCollector(
	ctx context.Context,
	client client.Client,
	config *collectorcontrollerv1alpha1.MetricsPrometheusCollector) (*Collector, error)

NewCollector returns a collector which publishes capacity and utilisation metrics.

func (*Collector) AggregateAndCollect added in v0.6.2 

func (c *Collector) AggregateAndCollect(
	a *collectorcontrollerv1alpha1.Aggregation, m map[MetricName]*Metric,
	ch chan<- prometheus.Metric, sCh chan<- *collectorapi.SampleList)

AggregateAndCollect aggregates the metrics at each level and then collects them

func (*Collector) Collect 

func (c *Collector) Collect(ch chan<- prometheus.Metric)

Collect returns the current state of all metrics of the collector. This there are cached metrics, Collect will return the cached metrics.

func (*Collector) Describe 

func (c *Collector) Describe(ch chan<- *prometheus.Desc)

Describe returns all descriptions of the collector.

func (*Collector) InitInformers 

func (c *Collector) InitInformers(ctx context.Context) error

InitInformers initializes the informer caches by listing all the objects

func (*Collector) NewAggregatedSampleListBuilder 

func (c *Collector) NewAggregatedSampleListBuilder(src collectorcontrollerv1alpha1.SourceType) *SampleListBuilder

func (*Collector) NewSampleListBuilder 

func (c *Collector) NewSampleListBuilder(src collectorcontrollerv1alpha1.SourceType) *SampleListBuilder

func (*Collector) NormalizeForSave added in v0.2.0 

func (c *Collector) NormalizeForSave(sr *collectorapi.ScrapeResult) error

func (*Collector) Run 

func (c *Collector) Run(ctx context.Context)

Start runs the collector

func (*Collector) SaveScrapeResultToFile 

func (c *Collector) SaveScrapeResultToFile(sr *collectorapi.ScrapeResult) error

func (*Collector) SaveScrapeResultToJSONFile added in v0.2.0 

func (c *Collector) SaveScrapeResultToJSONFile(sr *collectorapi.ScrapeResult) error

type CollectorFunc added in v0.6.2 

type CollectorFunc func(c *Collector, o *CapacityObjects, ch chan<- prometheus.Metric, sCh chan<- *collectorapi.SampleList) error

CollectorFunc is the type of the metric collector functions accepted for collector overrides.

type JSONLine added in v0.2.0 

type JSONLine struct {
	MetricName string            `json:"metricName"`
	Timestamp  time.Time         `json:"timestamp"`
	Value      float64           `json:"value"`
	Labels     map[string]string `json:"labels"`
}

type LabelOverrider 

type LabelOverrider interface {
	// OverrideLabels takes as an arg a map of all labels for a sample and returns
	// a map of labels to change the values of.
	OverrideLabels(map[string]string) map[string]string
}

type LabelOverriderFn 

type LabelOverriderFn func(map[string]string) map[string]string

LabelOverriderFn simplifies writing override types

func (LabelOverriderFn) OverrideLabels 

func (l LabelOverriderFn) OverrideLabels(m map[string]string) map[string]string

type Labeler added in v0.6.2 

type Labeler struct {
	// BuiltIn contains compiled in metric labels
	BuiltIn builtInLabler
	// Extension contains extension metric labels
	Extension extensionLabler
}

Labeler parses labels from Kubernetes objects and sets them on Values

func (Labeler) SetLabelsFoCGroup added in v0.6.2 

func (l Labeler) SetLabelsFoCGroup(labels *LabelsValues, u *api.NodeAggregatedMetrics)

func (Labeler) SetLabelsForClusterScoped added in v0.6.2 

func (l Labeler) SetLabelsForClusterScoped(labels *LabelsValues, discoveredLabels map[string]string)

func (Labeler) SetLabelsForContainer added in v0.6.2 

func (l Labeler) SetLabelsForContainer(
	labels *LabelsValues, container *corev1.Container)

SetLabelsForContainer parses metric labels from a container and pod

func (Labeler) SetLabelsForContainerStatus added in v0.6.2 

func (l Labeler) SetLabelsForContainerStatus(
	labels *LabelsValues, container *corev1.ContainerStatus)

SetLabelsForContainerStatus parses metric labels from a container status

func (Labeler) SetLabelsForNamespaces added in v0.6.2 

func (l Labeler) SetLabelsForNamespaces(labels *LabelsValues, namespace *corev1.Namespace)

SetLabelsForNamespaces parses metric labels from a namespace

func (Labeler) SetLabelsForNode added in v0.6.2 

func (l Labeler) SetLabelsForNode(labels *LabelsValues, node *corev1.Node)

SetLabelsForNode parses metric labels from a node

func (Labeler) SetLabelsForPVCQuota added in v0.6.2 

func (l Labeler) SetLabelsForPVCQuota(labels *LabelsValues,
	quota *corev1.ResourceQuota, resource collectorcontrollerv1alpha1.ResourceName)

SetLabelsForPVCQuota set storage class label from resource and quota spec

func (Labeler) SetLabelsForPersistentVolume added in v0.6.2 

func (l Labeler) SetLabelsForPersistentVolume(labels *LabelsValues,
	pv *corev1.PersistentVolume, pvc *corev1.PersistentVolumeClaim, node *corev1.Node)

func (Labeler) SetLabelsForPersistentVolumeClaim added in v0.6.2 

func (l Labeler) SetLabelsForPersistentVolumeClaim(labels *LabelsValues,
	pvc *corev1.PersistentVolumeClaim,
	pv *corev1.PersistentVolume, namespace *corev1.Namespace,
	pod *corev1.Pod, w workload, node *corev1.Node)

func (Labeler) SetLabelsForPod added in v0.6.2 

func (l Labeler) SetLabelsForPod(
	labels *LabelsValues, pod *corev1.Pod, w workload,
	node *corev1.Node, namespace *corev1.Namespace)

func (Labeler) SetLabelsForQuota added in v0.6.2 

func (l Labeler) SetLabelsForQuota(labels *LabelsValues,
	quota *corev1.ResourceQuota, rqd *quotamanagementv1alpha1.ResourceQuotaDescriptor, namespace *corev1.Namespace)

SetLabelsForQuota parses metric labels from a namespace

type LabelsValues added in v0.6.2 

type LabelsValues struct {
	BuiltIn builtInLabelsValues

	Extension extensionLabelsValues
}

LabelsValues contains metric label values

type Metric 

type Metric struct {
	// Mask contains the set of labels that apply to this metric
	Mask collectorcontrollerv1alpha1.LabelsMask

	Name MetricName

	Buckets map[string][]float64

	// Values contains the metric values for each unique set of labels
	Values map[LabelsValues][]resource.Quantity
}

Metric contains all the values for a metric

type MetricName added in v0.6.2 

type MetricName struct {
	Prefix string

	Level collectorcontrollerv1alpha1.AggregationLevel // e.g. cluster

	Operation collectorcontrollerv1alpha1.AggregationOperation // e.g. sum

	Source collectorcontrollerv1alpha1.Source // e.g. requests_quota_hard

	SourceAlias string

	ResourceAlias collectorcontrollerv1alpha1.ResourceAlias // e.g. cpu_cores

	Resource collectorcontrollerv1alpha1.ResourceName // e.g. cpu

	SourceType collectorcontrollerv1alpha1.SourceType // e.g. quota
}

func (MetricName) String added in v0.6.2 

func (m MetricName) String() string

type MissingContainerID 

type MissingContainerID struct {
	PodNamespace  string
	PodName       string
	ContainerName string
	NodeName      string
	ID            sampler.ContainerKey
}

type ResourceQuotaDescriptorKey added in v0.6.2 

type ResourceQuotaDescriptorKey struct {
	Name      string
	Namespace string
}

ResourceQuotaDescriptorKey is used as a key for priority class - namespace ResourceQuotaDescriptor pair

type SampleListBuilder 

type SampleListBuilder struct {
	Mask          collectorcontrollerv1alpha1.LabelsMask
	SourceType    collectorcontrollerv1alpha1.SourceType
	DirectoryPath string
	Save          bool
	TimeFormat    string
	SampleList    *collectorapi.SampleList
	C             *Collector
}

func (*SampleListBuilder) AddHistogramValues added in v0.6.2 

func (sb *SampleListBuilder) AddHistogramValues(s *collectorapi.Sample, resourceType collectorcontrollerv1alpha1.ResourceName,
	source collectorcontrollerv1alpha1.Source, buckets *collectorcontrollerv1alpha1.ExponentialBuckets, v ...resource.Quantity)

AddHistogramValues adds a Metric to the sample with values in histogram format

func (*SampleListBuilder) AddIntValues 

func (sb *SampleListBuilder) AddIntValues(s *collectorapi.Sample, resourceType collectorcontrollerv1alpha1.ResourceName, source collectorcontrollerv1alpha1.Source, v ...int64)

AddIntValues adds a Metric to the sample with the provided values

func (*SampleListBuilder) AddQuantityValues 

func (sb *SampleListBuilder) AddQuantityValues(s *collectorapi.Sample, resourceType collectorcontrollerv1alpha1.ResourceName, source collectorcontrollerv1alpha1.Source, v ...resource.Quantity)

AddQuantityValues adds a Metric to the sample with the provided values

func (*SampleListBuilder) NewSample 

func (sb *SampleListBuilder) NewSample(labels LabelsValues) *collectorapi.Sample

NewSample creates a new Sample for an object

func (*SampleListBuilder) SaveSamplesToFile 

func (sb *SampleListBuilder) SaveSamplesToFile() error

SaveSamplesToFile writes the SampleList to a binary file with the timestamp

type ValueReader added in v0.6.2 

type ValueReader struct{}

ValueReader reads the requests value as a ResourceList

func (ValueReader) GetValuesForContainer added in v0.6.2 

func (r ValueReader) GetValuesForContainer(
	container *corev1.Container, pod *corev1.Pod, usage *api.ContainerMetrics) map[collectorcontrollerv1alpha1.Source]value

GetValuesForContainer returns the ResourceLists from a container for: - requests_allocated - limits_allocated - utilization - requests_allocated_minus_utilization - nr_periods - nr_throttled - oom_kill - container (items)

func (ValueReader) GetValuesForNode added in v0.6.2 

func (r ValueReader) GetValuesForNode(node *corev1.Node, pods []*corev1.Pod) map[collectorcontrollerv1alpha1.Source]value

GetValuesForNode returns the metric values for a Node. pods is the Pods scheduled to this Node.

func (ValueReader) GetValuesForPV added in v0.6.2 

func (r ValueReader) GetValuesForPV(pv *corev1.PersistentVolume) map[collectorcontrollerv1alpha1.Source]value

func (ValueReader) GetValuesForPVC added in v0.6.2 

func (r ValueReader) GetValuesForPVC(pvc *corev1.PersistentVolumeClaim) map[collectorcontrollerv1alpha1.Source]value

func (ValueReader) GetValuesForPod added in v0.6.2 

func (r ValueReader) GetValuesForPod(pod *corev1.Pod) map[collectorcontrollerv1alpha1.Source]value

func (ValueReader) GetValuesForQuota added in v0.6.2 

func (r ValueReader) GetValuesForQuota(quota *corev1.ResourceQuota, rqd *quotamanagementv1alpha1.ResourceQuotaDescriptor, enableRqd bool) map[collectorcontrollerv1alpha1.Source]value

GetValuesForQuota returns the ResourceLists from a namespace quota

func (ValueReader) GetValuesForSchedulerHealth added in v0.6.2 

func (r ValueReader) GetValuesForSchedulerHealth(pod *corev1.Pod, s *collectorcontrollerv1alpha1.SchedulerHealth) map[collectorcontrollerv1alpha1.Source]value

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.