Apache Kafka: Real-time Data Streaming, Messaging & Event-driven Architecture

Real-time data pipelining and event-driven architecture.

2 days

What does the training include?

In this course, you'll get a thorough and hands-on introduction to Apache Kafka: the leading platform for real-time data streaming, messaging, and event-driven architectures. You'll learn how Kafka works, how to build scalable streaming applications, and how to integrate Kafka into modern data platforms and microservices. Through hands-on labs, you'll work with producers, consumers, connectors, and stream processors, designing reliable, high-throughput data pipelines in Python or Java.

What you'll learn

The core principles of Apache Kafka and why it's ideal for real-time data streaming.
Key concepts: producers, consumers, topics, partitions, offsets, and consumer groups.
Architectures for scalability, reliability, and fault tolerance.
Working with Kafka Connect for data movement and database integrations.
Real-time stream processing with the Kafka Streams API.
Kafka within microservices and event-sourcing patterns.
Advanced concepts such as exactly-once semantics, log compaction, and Kafka Raft.

Programme

Part 1 – Introduction to Kafka and Real-time Streaming

Producers, consumers, topics, and partitions.

Part 2 – Architecture & Integration

Scalability, reliability, replication, and fault tolerance.

Part 3 – Kafka Connect

Source and sink connectors, database integration, schema evolution.

Part 4 – Kafka Streams

Stateful processing, windowing, and real-time transformations.

Part 5 – Kafka in Microservices

Event-sourcing, messaging patterns, and best practices.

Part 6 – Advanced Concepts

Exactly-once semantics, log compaction, Kafka Raft.

Part 7 – Best Practices & Q&A

For whom?

Data engineers and data architects.
Software engineers working with microservices or event-driven architectures.
Organisations looking to use Kafka for real-time analytics, data movement, or integrations.
Anyone who wants to master real-time data processing with Apache Kafka.

Prerequisites

Basic programming knowledge (e.g. Python or Java).
Familiarity with distributed systems or data processing is helpful.
No prior Kafka experience required.

What will you learn?

Set up and manage Kafka clusters and clients independently.
Build robust streaming applications with producers, consumers, and Kafka Streams.
Integrate Kafka into existing systems using best-practice architectures.
Move data efficiently with Kafka Connect and schema management.
Apply event-driven design within microservice environments.
Design scalable, production-ready real-time data pipelines.