Integrate Sleuth and Zipkin to your Spring Boot Microservices for distributed logging
What does Sleuth do?
You can visit here for more details, these bullets are from the same link:
Transparently create and inject a correlation ID into your service calls if one doesn’t exist.
Manage the propagation of the correlation ID to outbound service calls so that the correlation ID for a transaction is automatically added to outbound calls.
Add the correlation information to Spring’s MDC logging so that the generated correlation ID is automatically logged by Spring Boots default SL4J and Logback implementation.
Optionally, publish the tracing information in the service call to the Zipkin — distributed tracing platform.
Since “traceId” and “spanId” (correlation ID) is added to MDC, you can integrate it with ELK via Logstash dependency in pom.xml.
What does Zipkin do?
Explanation from here:
Zipkin is an open source data-visualization tool that can show the flow of a transaction across multiple services. Zipkin allows you to break a transaction down into its component pieces and visually identify where there might be performance hotspots
Zipkin uses in-memory database by default but you can change this to Cassandra, ES, etc. Details are provided here.
You can download Zipkin as jar or as a Docker image. You can follow these instructions here if you want to use the jar version. I preferred to use Docker version.
Kafka and Zipkin Docker
Even if the distributed tracing server is down, all these microservices can put information on the queue and we can pick up information from the queue in the distributed tracing server (to add more resiliency, as explained here). I preferred to use Kafka instead of RabbitMQ.
To accomplish this, I will use Docker images.
Download docker-compose-kafka.yml and docker-compose.yml to a preferred folder path (for details and other Docker compose files, you can visit here).
docker compose -f ~/dev/senoritadev-workspace/spring-microservices/docker/docker-compose-kafka.yml upIntegrating to our Spring Boot Microservices
This is as easy as adding only three extra dependencies to your projects (including your api-gateway project, Spring Cloud Gateway — provided with spring-cloud-starter-gateway dependency, if exists):
Add these to your properties to your application.properties (pay attention to zipkin-kafka Docker’s exposed ports as localhost:19092 is set to spring.kafka.bootstrap-servers):
spring.sleuth.sampler.probability=1.0
spring.zipkin.sender.type=kafka
spring.kafka.bootstrap-servers=localhost:19092Every request is logged if we set sampler probability to 1.0.
I did not add any other configuration for Kafka in my application.properties file. The properties shown below will be the default values (you can visit here for more details).
spring.sleuth.kafka.enabled -- true
Enable instrumenting of Apache Kafka clients.spring.sleuth.messaging.kafka.enabled -- true
Enable tracing of Kafka.spring.sleuth.messaging.kafka.mapper.enabled -- true
Enable DefaultKafkaHeaderMapper tracing for Kafka.spring.sleuth.messaging.kafka.remote-service-name -- kafka
Kafka remote service name.spring.sleuth.messaging.kafka.streams.enabled -- false
Should Kafka Streams be turned on.spring.zipkin.kafka.topic -- zipkin
Name of the Kafka topic where spans should be sent to Zipkin.
Time to Test
Eureka http://localhost:8761/
Kafka
To check if data is being sent to Kafka after performing a REST API call:
kafka-console-consumer --bootstrap-server localhost:19092 --topic zipkinZipkin http://localhost:9411/zipkin/
Voilà!
You can visit my Microservices project (implemented with Spring Cloud) GitHub url for sample code:
Happy Coding!
