Services creation¶
The ITSI module for Telegraf Kafka smart monitoring provides builtin services templates, relying on several base KPIs retrieving data from the metric store.
- Zookeeper monitoring
- Kafka brokers monitoring
- Kafka LinkedIn monitor
- Kafka topic monitoring
- Kafka connect monitoring
- Kafka sink task monitoring
- Kafka source task monitoring
- Kafka Consumers lag monitoring
- Confluent schema-registry monitoring
- Confluent Confluent ksql-server monitoring
- Confluent kafka-rest monitoring
As a general practice, if you first goal is designing the IT infrastructure in ITSI, a good generic recommendation is to create a main service container for your Kafka infrastructure.
As such, every service that will be designed will be linked to the main service. (the main service depends on them)
Monitoring Zookeeper servers¶
To monitor your Zookeeper servers, create a new service using the “Zookeeper monitoring” template service and select the proper filters for your entities:
- Configure / Service / Create new service / Zookeeper monitoring
Monitoring Kafka Brokers¶
To monitor your Zookeeper servers, create a new service using the “Kafka brokers monitoring” template service and select the proper filters for your entities:
- Configure / Service / Create new service / Kafka brokers monitoring
Monitoring Kafka Topics¶
To monitor one or more Kafka topics, create a new service using the “Kafka topic monitoring” template service and select the proper filters for your entities corresponding to your topics:
- Configure / Service / Create new service / Kafka topic monitoring
Monitoring Kafka Connect¶
To monitor Kafka Connect, create a new service using the “Kafka connect monitoring” template service and select the proper filters for your entities:
- Configure / Service / Create new service / Kafka connect monitoring
Monitoring Kafka Connect Sink taks¶
To monitor one of more Kafka Connect Sink connectors, create a new service using the “Kafka sink task monitoring” template service and select the proper filters for your entities:
Monitoring Kafka Connect Source taks¶
To monitor one of more Kafka Connect Source connectors, create a new service using the “Kafka source task monitoring” template service and select the proper filters for your entities:
Monitoring Kafka Consumers¶
To monitor one or more Kafka Consumers, create a new service using the “Kafka Consumers lag monitoring” template service and select the proper filters for your entities corresponding to your topics:
- Configure / Service / Create new service / Kafka lag monitoring
Monitoring Confluent schema-registry¶
To monitor one of more Confluent schema-registry nodes, create a new service using the “Kafka schema-registry monitoring” template service and select the proper filters for your entities:
Monitoring Confluent ksql-server¶
To monitor one of more Confluent ksql servers, create a new service using the “Confluent ksql-server monitoring” template service and select the proper filters for your entities:
Monitoring Confluent kafka-rest¶
To monitor one of more Confluent kafka-rest nodes, create a new service using the “Confluent kafka-rest monitoring” template service and select the proper filters for your entities:
End to end monitoring with LinkedIn Kafka monitor¶
To monitor your Kafka deployment using the LinkedIn Kafka monitor, create a new service using the “Kafka LinkedIn monitor” template service and select the proper filters for your entities:
- Configure / Service / Create new service / Kafka LinkedIn monitor