How to Install Apache Kafka in Docker?
Introduction
Modern IT is fully developed in terms of services and products. There are times when large amounts of data need to be streamed. It is called event-streaming, more precisely. There are many technologies used for that. Apache Kafka is an open-source tool developed by the Apache Foundation. It is needed on high-end gaming servers for log monitoring, machine learning, analytics, etc. It has a wide application in IT.
Installation
Let’s see how we can install Apache Kafka inside a Docker container. Docker is so popular and lightweight.
Step 1 – Install Docker.
Install Docker and verify it’s working. Run the below command on the command-line interface (CLI).
#docker -v
If not installed, do install it from the official doc: https://docs.docker.com/engine/install/
Step 2 – Create an independent directory.
Create a directory under /opt/ or wherever you want it to be installed.
#mkdir /opt/kafka
#cd /opt/kafka
Step 3 – Create a docker-compose file.
It’s time to write docker-compose.yml, where you can define the entire application stack in a single file. Create the docker-compose file with the below directives.
version: '2'
services:
zookeeper:
image: confluentinc/cp-zookeeper:7.4.4
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
ports:
- 22181:2181
kafka:
image: confluentinc/cp-kafka:7.4.4
depends_on:
- zookeeper
ports:
- 29092:29092
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
The Zookeeper and Kafka are bound together. It runs on port 22181 and 29092 respectively. Rest assured, all additional settings are in place.
Step 4 – Start the container using docker-compose.
You can simply start it using the command below and see the output.
#docker-compose up -d
It is successful. Time to verify that Docker is running.
#docker ps
![](https://lh7-us.googleusercontent.com/docsz/AD_4nXfzjWDOvR-XiBb2wDxzaswSkMGOnGNyOaMrZzRzx0FF6IB8rqYcij0AcOXENnP_xVJ5_JVw1LUiEPt1gzlTlNb4bvrzG2gwJFxas16PxOQC5UFOv-VDITnftMJJsRK5hw7HOYaLgExTy575FL0gPwgSxpRh0e06OjndOrNW8JWuoYz9MK61ltk?key=cFYduWwt0haHrd6fKVwfBg)
Step 5 – Verify that Kafka is running.
You can verify it using netcat (nc) command.
#nc -zv localhost 22181
Step 6 – Connect to Kafka.
Kafka is running, use any kafka client to connect to it. Remember it is a single node cluster.
Conclusion
Apache Kafka is a large data handling and streaming utility. It is open-source and reliable. It can be installed on VPS and dedicated servers in single-node and multi-node configurations.