In Data Engineer’s Lunch #22: Prometheus, guest speaker Will Angel covers the topic of using Prometheus for data engineering. Prometheus is a monitoring system & time-series database The live recording of the Data Engineer’s Lunch, which includes a more in-depth discussion, is also embedded below in case you were not able to attend live. If you would like to attend a Data Engineer’s Lunch live, it is hosted every Monday at noon EST. Register here now!
Prometheus
Prometheus is an open-source software application used for event monitoring and alerting. It records real-time metrics in a time series database built using an HTTP pull model. It allows flexible queries and real-time alerting. Here at Anant, we understand the importance of having a great monitoring system in place in order to rest easy with the knowledge that our programs and our client’s programs are operating smoothly and efficiently. Prometheus is a great option as its data model is multi-dimensional with time series identified by metric name and key-value pairs. It leverages the PromQL query language in order to facilitate these actions. Prometheus’s single server nodes are autonomous and time-series collection happens over HTTP using a pull model. Intermediary gateways support pushing time-series and targets are discovered by static configuration or service discovery. Finally, Prometheus supports multiple modes of graphing and dashboarding.
Architecture
When to use and when not to use
Prometheus is leveraged best when the data coming in is multi-dimensional in nature, think IoT or microservice monitoring. Its strength lies in collecting and querying multi-dimensional data and allows for rapid diagnoses of issues. The stand-alone nature of its servers allows it to be reliable and can be called upon even when other parts of your system are down. Another added benefit is its popularity, which means there are many resources available for integration into your monitoring system.
Prometheus is great for monitoring the health of your system, but when you want more detailed records with 100% accuracy, it may not be your best solution.
As mentioned above, if you are looking for a more in-depth discussion, as well as a demo, be sure to check out the live recording of Data Engineer’s Lunch #22: Prometheus below!
Cassandra.Link
Cassandra.Link is a knowledge base that we created for all things Apache Cassandra. Our goal with Cassandra.Link was to not only fill the gap of Planet Cassandra but to bring the Cassandra community together. Feel free to reach out if you wish to collaborate with us on this project in any capacity.
We are a technology company that specializes in building business platforms. If you have any questions about the tools discussed in this post or about any of our services, feel free to send us an email!
