Grafana Mimir is a product developed by Grafana Labs, built upon the Cortex project. It was announced and launched in 2022. There are few detailed breakdowns and guides available from users about it. In this article, we will explain its structure, its pros and cons, and also compare it with well-known tools like Thanos and VictoriaMetrics.

What is Mimir?

Mimir is designed for long-term storage of time series metrics commonly used in monitoring systems like Prometheus, which is a popular open-source monitoring system.

Mimir receives data from source systems using Prometheus remote-write feature and stores it using external object stores such as Amazon S3 or GCS. This reduces storage costs and increases reliability.

The main advantage of Grafana Mimir is its relatively high data processing speed achieved through load distribution, optimized storage, compression mechanisms, caching, and indexing. It has a new architecture that allows fast loading and processing of large volumes of data from various sources.

Using Mimir enables you to collect and aggregate data from multiple Prometheus installations, VictoriaMetrics, and Grafana Agent in one place. The product also provides advanced capabilities for complex queries and data processing. This effectively solves the problems of handling and analyzing large volumes of data over extended periods.

Deployment options

Mimir can be deployed in two ways: all-in-one (monolithic) where all components work within a single container or executable file, and microservices where components run in separate containers/pods, and each component is scalable. The microservices option is ideal for working in Kubernetes.

Mimir Components

• Distributor: This component doesn’t require its own persistent storage. It receives and validates data from sources like Prometheus or Grafana Agent and sends them to the Ingester component in parallel, according to the configured replication factor (default is 3).
• Ingester: It writes metrics received from the Distributor into long-term storage and also keeps them in its cache for some time, responding to queries from the Querier if the data blocks are still in the cache.
• Querier: It searches for data based on PromQL expressions for recent data in the Ingester and long-term storage through the Store-gateway.
• Query Frontend: It caches responses from the Querier component and immediately serves clients. The client interacts with Query Frontend when accessing Mimir.
• Store-gateway: It is responsible for retrieving data from long-term storage. The Store-gateway periodically queries the storage for an updated bucket index compacted by the Compactor component and saves it locally.
• Compactor: It improves storage performance and reduces resource utilization by merging, compressing, and deduplicating data blocks. It also removes data blocks according to the configured storage policies.
• Ruler: It allows storing data for metric aggregation and alerting rules based on PromQL expressions, for example, between different tenants. It utilizes the built-in Querier and Distributor components.
• Alertmanager: It adds multi-tenancy and scalable functionality to Prometheus Alertmanager.
• Query-scheduler: It handles the storage and distribution of query queues among available Queriers.
• Overrides-exporter: It allows configuring various limitations for tenants, such as the number of metrics. Individual metrics are available for analysis and monitoring of these limits.

Optional components of the architecture

Advantages and Comparison with Competitors

There is another product on the market with similar functionality and purpose – Thanos, which also implements long-term storage for Prometheus-like monitoring systems.

Although Thanos and Mimir have a similar architecture, there are some differences worth noting:

Mimir does not require connecting to the current Prometheus as a sidecar. Thanos has the Thanos receiver component, but the documentation suggests using it only for isolated environments.
The Store component in Thanos does not support native scalability.
The reliability of Thanos Rule leaves room for improvement; for example, alerts can be lost in the event of failures in other Thanos components.

However, at the moment, unlike Thanos, Mimir does not support downsampling and individual data block deletion, except through storage policies.

Comparing it with VictoriaMetrics would not be entirely accurate since these tools primarily serve different functions.

Mimir is primarily a storage system into which other metric sources write data using a push model.
VictoriaMetrics is a comprehensive monitoring system that pulls metrics from various exporters and aggregates them locally in its own storage format. Therefore, VictoriaMetrics and Mimir can be used together, and in some cases, it may be necessary to do so.

Additionally, based on testing results, we have identified some clear advantages of Mimir:

1. It is easy and quick to deploy, and it has clear documentation. New metric sources can be easily integrated without major difficulties.
2. It can be horizontally scaled by launching the required number of replicas for its components. Autoscaling configuration is available.
3. It integrates with Grafana’s alerting mechanism.
4. It enables multi-tenancy, based on additional HTTP request headers in Prometheus remote-write.

Practical Use Cases

Mimir has several use cases that vary depending on the tasks at hand. The main ones include:

• Collecting and storing metrics from remote Prometheus-like monitoring systems with minimal changes to their configuration.
• Standardizing and aggregating data collected by different monitoring systems within an organization (enabling a unified entry point for all users).
• Providing MaaS (Monitoring as a Service) to both internal and external users.

Integrating the Cloud with Grafana Mimir

To use Mimir for long-term data storage, it needs to be configured to work with object storage using the S3 protocol. You can deploy it locally, but it’s also convenient to use ready-made object storage from cloud providers.

Deployment

By default, Mimir uses MinIO as the storage. Let’s consider the standard deployment in Kubernetes using the official helm chart (https://grafana.com/docs/helm-charts/mimir-distributed/latest/) and connecting it to an external S3 storage.

Create a new namespace for Mimir:

kubectl create namespace mimir-test

Let’s add the Grafana Helm repository:

helm repo add grafana https://grafana.github.io/helm-charts
helm repo update

Create a file with values to configure the S3 storage. We will use the Cloud KROK object storage for this purpose.

nginx:
ingress:
enabled: true
ingressClassName: nginx
annotations: {}
hosts:
– host: mimir-example.croc.ru
paths:
– path: /
pathType: Prefix
minio:
enabled: false

global:
extraEnvFrom:
– secretRef:
name: mimir-bucket-secret
podAnnotations:
bucketSecretVersion: “0”

mimir:
structuredConfig:
common:
storage:
backend: s3
s3:
endpoint: storage.cloud.croc.ru
region: croc
secret_access_key: ${AWS_SECRET_ACCESS_KEY}
access_key_id: ${AWS_ACCESS_KEY_ID}
alertmanager_storage:
s3:
bucket_name: mimir-croc-alertmanager
access_key_id: ${AWS_ACCESS_KEY_ID}
endpoint: storage.cloud.croc.ru
secret_access_key: ${AWS_SECRET_ACCESS_KEY}
blocks_storage:
backend: s3
s3:
bucket_name: mimir-croc-blocks
access_key_id: ${AWS_ACCESS_KEY_ID}
endpoint: storage.cloud.croc.ru
secret_access_key: ${AWS_SECRET_ACCESS_KEY}
ruler_storage:
s3:
bucket_name: mimir-croc-ruler
access_key_id: ${AWS_ACCESS_KEY_ID}
endpoint: storage.cloud.croc.ru
secret_access_key: ${AWS_SECRET_ACCESS_KEY}

Create a Secret to store the access keys for S3:

apiVersion: v1
kind: Secret
metadata:
name: mimir-bucket-secret
data:
AWS_ACCESS_KEY_ID: example:example@croc.ru
AWS_SECRET_ACCESS_KEY: secretkey

kubectl -n mimir apply -f secret.yml

Deploy the Helm chart in the Kubernetes cluster:

helm install grafana/mimir-distributed -f values.yml

In Prometheus/VictoriaMetrics, specify the Ingress endpoint for RemoteWrite:

remote_write:
– url: http://mimir-example.croc.ru/api/v1/push

With Prometheus/VictoriaMetrics configured this way, they will send data based on the specified sources in the scrape_configs section.

Connect Mimir to Grafana as a Data Source:

• Go to Configuration -> Data Sources -> Add new data source.

• In the URL field, enter the host that was specified in the values.yml file.

• Save the data source.

Now, with the added dashboards, we can visualize the metrics stored in Mimir.

 

Stay updated with new posts on your favorite topics!

Subscribe to the tags that interest you to keep track of new posts and stay informed.