Overview

Over the last few years, Kafka has become the de-facto platform for stream processing and messaging: from microservice communication fabric, through inbound data ingestion to streaming analytics, Kafka can serve all of these use cases. In this course, developers can expect to gain the skills and knowledge required to build Scala applications that use Kafka as their backbone. On the Kafka side, the course covers essential Kafka concepts, application-level administration of Kafka clusters and important tooling; on the application side, the course covers the usage of Kafka in Scala using stream processing paradigms, both for microservice messaging and for streaming analytics.

Who Should Attend

Developers, data engineers and software architects interested in using Kafka for microservice communication, data ingestion, and stream processing.

Prerequisites

Attendees should have basic experience with usage of the Scala language.

Topics

• Overview of Kafka’s storage hierarchy - clusters, brokers, topics, partitions
• Overview of ZIO and ZIO Streams
• Using Kafka’s Consumer, ZIO Streams and ZIO Kafka to consume topics as streams
• Using consumer groups to create scalable applications
• Producing data to Kafka topics using ZIO Kafka
• Data serialization and deserialization when working with ZIO Kafka
• Latency/reliability trade-offs in the Kafka producer
• CLI tools for administering Kafka clusters
• Important parameters for Kafka topics
• Replicating data between microservices using key compacted topics
• Error-handling and disaster recovery for Kafka-based applications
• Monitoring Kafka-based applications
• Concurrency models for Kafka clients