{"id":279720,"date":"2024-01-15T17:08:46","date_gmt":"2024-01-15T16:08:46","guid":{"rendered":"https:\/\/www.ci4rail.com\/?p=279720"},"modified":"2024-01-16T14:41:32","modified_gmt":"2024-01-16T13:41:32","slug":"how-to-monitor-your-microcontroller-devices-with-prometheus-and-grafana","status":"publish","type":"post","link":"https:\/\/www.ci4rail.com\/de\/how-to-monitor-your-microcontroller-devices-with-prometheus-and-grafana\/","title":{"rendered":"Wie du deine Microcontroller Devices mit Prometheus und Grafana monitoren kannst."},"content":{"rendered":"<p>[et_pb_section fb_built=&#8220;1&#8243; admin_label=&#8220;section&#8220; _builder_version=&#8220;4.16&#8243; background_color=&#8220;RGBA(0,0,0,0)&#8220; min_height=&#8220;265px&#8220; global_colors_info=&#8220;{}&#8220; theme_builder_area=&#8220;post_content&#8220;][et_pb_row admin_label=&#8220;row&#8220; _builder_version=&#8220;4.16&#8243; background_size=&#8220;initial&#8220; background_position=&#8220;top_left&#8220; background_repeat=&#8220;repeat&#8220; min_height=&#8220;185px&#8220; global_colors_info=&#8220;{}&#8220; theme_builder_area=&#8220;post_content&#8220;][et_pb_column type=&#8220;4_4&#8243; _builder_version=&#8220;4.16&#8243; custom_padding=&#8220;|||&#8220; global_colors_info=&#8220;{}&#8220; custom_padding__hover=&#8220;|||&#8220; theme_builder_area=&#8220;post_content&#8220;][et_pb_text admin_label=&#8220;Text&#8220; _builder_version=&#8220;4.23.4&#8243; background_color=&#8220;RGBA(0,0,0,0)&#8220; background_size=&#8220;initial&#8220; background_position=&#8220;top_left&#8220; background_repeat=&#8220;repeat&#8220; hover_enabled=&#8220;0&#8243; global_colors_info=&#8220;{}&#8220; theme_builder_area=&#8220;post_content&#8220; sticky_enabled=&#8220;0&#8243;]<\/p>\n<h4><span><span class=\"ui-provider a b c d e f g h i j k l m n o p q r s t u v w x y z ab ac ae af ag ah ai aj ak\" dir=\"ltr\"><strong>Why monitoring?<\/strong><\/span><\/span><\/h4>\n<p>You have just completed your firmware for your microcontroller and have a bunch of them installed at your customer&#8217;s site. They seem to work, but you are unsure how they behave in the long run? You want to know if they are online, how much memory they uses and how many times they have restarted?<\/p>\n<p>Well, you could implement a custom solution, but why reinvent the wheel? There are already many tools out there that can help you with this task.<\/p>\n<p>In this tutorial, I will show you how to monitor your microcontroller devices with Prometheus and Grafana. I will use the <a aria-label=\"Link ESP32\" href=\"https:\/\/www.espressif.com\/en\/products\/socs\/esp32\" rel=\"noreferrer noopener\" target=\"_blank\" class=\"fui-Link ___1rxvrpe f2hkw1w f3rmtva f1ewtqcl fyind8e f1k6fduh f1w7gpdv fk6fouc fjoy568 figsok6 f1hu3pq6 f11qmguv f19f4twv f1tyq0we f1g0x7ka fhxju0i f1qch9an f1cnd47f fqv5qza f1vmzxwi f1o700av f13mvf36 f1cmlufx f9n3di6 f1ids18y f1tx3yz7 f1deo86v f1eh06m1 f1iescvh fhgqx19 f1olyrje f1p93eir f1nev41a f1h8hb77 f1lqvz6u f10aw75t fsle3fq f17ae5zn\" title=\"https:\/\/www.espressif.com\/en\/products\/socs\/esp32\">ESP32<\/a> as an example. The approach shown is best suited when you have a bunch of devices you want to monitor and the monitoring is done on a server in the same network, for example in an industrial plant.<\/p>\n<h4><strong>What is Prometheus?<\/strong><\/h4>\n<p><a aria-label=\"Link Prometheus\" href=\"https:\/\/prometheus.io\/\" rel=\"noopener noreferrer\" target=\"_blank\" class=\"fui-Link ___1rxvrpe f2hkw1w f3rmtva f1ewtqcl fyind8e f1k6fduh f1w7gpdv fk6fouc fjoy568 figsok6 f1hu3pq6 f11qmguv f19f4twv f1tyq0we f1g0x7ka fhxju0i f1qch9an f1cnd47f fqv5qza f1vmzxwi f1o700av f13mvf36 f1cmlufx f9n3di6 f1ids18y f1tx3yz7 f1deo86v f1eh06m1 f1iescvh fhgqx19 f1olyrje f1p93eir f1nev41a f1h8hb77 f1lqvz6u f10aw75t fsle3fq f17ae5zn\" title=\"https:\/\/prometheus.io\/\">Prometheus<\/a> is an open-source monitoring system that was originally built by SoundCloud. It&#8217;s main purpose is to collect metrics from servers and applications and store them in a time-series database. It also provides a web interface where you can view the collected metrics and create alerts, but it is not very user-friendly. That&#8217;s why we will use Grafana to visualize the data.<\/p>\n<p>So, how does Prometheus work? It uses a pull-based model, which means that it periodically scrapes metrics from the configured targets. The targets can be an application that exposes metrics, a server, or it can be your microcontroller device.<\/p>\n<p>Prometheus has a node-exporter that can be used to collect metrics from the host system. It can be used to monitor the CPU, memory, disk space, network traffic, etc. The node-exporter is made for typical linux servers, but since we are using a microcontroller based device which has very limited resources, we will implement our own solution.<\/p>\n<h4><strong>How to expose the metrics?<\/strong><\/h4>\n<p>Prometheus expects the metrics in the <a aria-label=\"Link Prometheus exposition format\" href=\"https:\/\/prometheus.io\/docs\/instrumenting\/exposition_formats\/\" rel=\"noreferrer noopener\" target=\"_blank\" class=\"fui-Link ___1rxvrpe f2hkw1w f3rmtva f1ewtqcl fyind8e f1k6fduh f1w7gpdv fk6fouc fjoy568 figsok6 f1hu3pq6 f11qmguv f19f4twv f1tyq0we f1g0x7ka fhxju0i f1qch9an f1cnd47f fqv5qza f1vmzxwi f1o700av f13mvf36 f1cmlufx f9n3di6 f1ids18y f1tx3yz7 f1deo86v f1eh06m1 f1iescvh fhgqx19 f1olyrje f1p93eir f1nev41a f1h8hb77 f1lqvz6u f10aw75t fsle3fq f17ae5zn\" title=\"https:\/\/prometheus.io\/docs\/instrumenting\/exposition_formats\/\">Prometheus exposition format<\/a> via a http endpoint. A typical endpoint is <code>\/metrics<\/code>, on port 9100, but you can use other endpoints and ports. We&#8217;ll see later how to configure Prometheus. This means our microcontroller device needs to implement a http server that exposes the metrics in the correct format.<\/p>\n<h4><strong>Metrics<\/strong><\/h4>\n<p>The most important types of metrics are counters and gauges.<\/p>\n<ul>\n<li>Counters are used to count the number of events that occur. They can only increase or be reset to zero. Examples for counters are the number of requests to a web server or the number of times a device has restarted.<\/li>\n<li>Gauges are used to measure the current state of something. Examples for gauges are the current temperature or the amount of free memory.<\/li>\n<\/ul>\n<p>Here is an example of the data that Prometheus expects:<\/p>\n<pre class=\"language-plaintext\"><code># TYPE free_heap_bytes gauge<br \/>free_heap_bytes{} 122884<br \/># TYPE restarts_total counter<br \/>restarts_total{} 43<\/code><\/pre>\n<h4><strong>Metric Labels<\/strong><\/h4>\n<p>In the previous example, you have seen the metric name <code>free_heap_bytes<\/code> and <code>restarts_total<\/code>. These are the names of the metrics. But what are the curly braces <code>{}<\/code>? This is the place to put in labels.<\/p>\n<p>Labels are used to distinguish between different instances of the same metric. For example, if you have two temperature sensors in your device, you can use labels to distinguish between them.<\/p>\n<pre class=\"language-plaintext\"><code># TYPE temperature_celsius gauge<br \/>temperature_celsius{sensor=\"cpu_die\"} 48.5<br \/>temperature_celsius{sensor=\"pcb\"} 32.0<\/code><\/pre>\n<h4><strong>How to implement the metrics endpoint?<\/strong><\/h4>\n<p>You can do it on your own, but for ESP32 you can use our ESP32 component <a aria-label=\"Link esp_prometheus_exporter\" href=\"https:\/\/github.com\/ci4rail\/esp_prometheus_exporter\" rel=\"noreferrer noopener\" target=\"_blank\" class=\"fui-Link ___1rxvrpe f2hkw1w f3rmtva f1ewtqcl fyind8e f1k6fduh f1w7gpdv fk6fouc fjoy568 figsok6 f1hu3pq6 f11qmguv f19f4twv f1tyq0we f1g0x7ka fhxju0i f1qch9an f1cnd47f fqv5qza f1vmzxwi f1o700av f13mvf36 f1cmlufx f9n3di6 f1ids18y f1tx3yz7 f1deo86v f1eh06m1 f1iescvh fhgqx19 f1olyrje f1p93eir f1nev41a f1h8hb77 f1lqvz6u f10aw75t fsle3fq f17ae5zn\" title=\"https:\/\/github.com\/ci4rail\/esp_prometheus_exporter\">esp_prometheus_exporter<\/a>, which is a small open source library that implements the Prometheus endpoint with the exposition format and a set of APIs to define and update the metrics. The <a aria-label=\"Link README\" href=\"https:\/\/github.com\/ci4rail\/esp_prometheus_exporter\/blob\/main\/README.md\" rel=\"noreferrer noopener\" target=\"_blank\" class=\"fui-Link ___1rxvrpe f2hkw1w f3rmtva f1ewtqcl fyind8e f1k6fduh f1w7gpdv fk6fouc fjoy568 figsok6 f1hu3pq6 f11qmguv f19f4twv f1tyq0we f1g0x7ka fhxju0i f1qch9an f1cnd47f fqv5qza f1vmzxwi f1o700av f13mvf36 f1cmlufx f9n3di6 f1ids18y f1tx3yz7 f1deo86v f1eh06m1 f1iescvh fhgqx19 f1olyrje f1p93eir f1nev41a f1h8hb77 f1lqvz6u f10aw75t fsle3fq f17ae5zn\" title=\"https:\/\/github.com\/ci4rail\/esp_prometheus_exporter\/blob\/main\/readme.md\">README<\/a> explains how to use it.<\/p>\n<p>Use this component as a git submodule in your project and place it in the <code>components<\/code> folder.<\/p>\n<h4><strong>Setup Prometheus<\/strong><\/h4>\n<p>In this example, we will use a docker container to run Prometheus and run it with docker compose.<\/p>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Why monitoring? You have just completed your firmware for your microcontroller and have a bunch of them installed at your customer&#8217;s site. They seem to work, but you are unsure how they behave in the long run? You want to know if they are online, how much memory they uses and how many times they [&hellip;]<\/p>\n","protected":false},"author":4,"featured_media":279530,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":"","_et_gb_content_width":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[74],"tags":[],"dipi_cpt_category":[],"class_list":["post-279720","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog-beitraege"],"_links":{"self":[{"href":"https:\/\/www.ci4rail.com\/de\/wp-json\/wp\/v2\/posts\/279720","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ci4rail.com\/de\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ci4rail.com\/de\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ci4rail.com\/de\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ci4rail.com\/de\/wp-json\/wp\/v2\/comments?post=279720"}],"version-history":[{"count":10,"href":"https:\/\/www.ci4rail.com\/de\/wp-json\/wp\/v2\/posts\/279720\/revisions"}],"predecessor-version":[{"id":279779,"href":"https:\/\/www.ci4rail.com\/de\/wp-json\/wp\/v2\/posts\/279720\/revisions\/279779"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.ci4rail.com\/de\/wp-json\/wp\/v2\/media\/279530"}],"wp:attachment":[{"href":"https:\/\/www.ci4rail.com\/de\/wp-json\/wp\/v2\/media?parent=279720"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ci4rail.com\/de\/wp-json\/wp\/v2\/categories?post=279720"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ci4rail.com\/de\/wp-json\/wp\/v2\/tags?post=279720"},{"taxonomy":"dipi_cpt_category","embeddable":true,"href":"https:\/\/www.ci4rail.com\/de\/wp-json\/wp\/v2\/dipi_cpt_category?post=279720"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}