Install Kafka Docker

2/13/2022by admin
  1. Install Kafka Docker Windows
  2. Install Kafka Docker

For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container. If you have not installed Docker, download the Community edition and follow the instructions for your OS. Kafka writes data to a scalable disk structure and replicates for fault-tolerance. Producers can wait for write acknowledgments. Stream processing with Kafka Streams API, enables complex aggregations or joins of input streams onto an output stream of processed data. Traditional messaging models are queue and publish-subscribe. Kafka is recommended for production deployments. This queue is used on the most of ThingsBoard production environments now. It is useful for both on-prem and private cloud deployments. It is also useful if you like to stay independent from your cloud provider. However, some providers also have managed services for Kafka. See AWS MSK for example. A minimal Docker image based on Alpine Linux with a complete package index and only 5 MB in size!

In this video, I will show you how to install Apache Kafka using Docker and Docker compose in any operating system.Github Code:

  • $ docker run -rm -network kafka-net ches/kafka -topic USERCREATEDTOPIC -from-beginning -bootstrap-server kafka:9092; The output on the terminal should be similar to: So we have seen how to install Apache Kafka in Docker container and make it work.
  • As we said, in any case, if you want to install and run Kafka you should run a ZooKeeper server. Before running ZooKeep container using docker, we create a docker network for our cluster: Now we should run a ZooKeeper container from Bitnami ZooKeeper image: By default, ZooKeeper runs on port 2181 and we expose that port using -p param so that.

For local development and testing, you can run Pulsar in standalonemode on your own machine within a Docker container.

If you have not installed Docker, download the Community editionand follow the instructions for your OS.

Start Pulsar in Docker

  • For MacOS, Linux, and Windows:

A few things to note about this command:

  • The data, metadata, and configuration are persisted on Docker volumes in order to not start 'fresh' everytime the container is restarted. For details on the volumes you can use docker volume inspect <sourcename>
  • For Docker on Windows make sure to configure it to use Linux containers

If you start Pulsar successfully, you will see INFO-level log messages like this:


When you start a local standalone cluster, a public/defaultnamespace is created automatically. The namespace is used for development purposes. All Pulsar topics are managed within namespaces.For more information, see Topics.

Use Pulsar in Docker

Pulsar offers client libraries for Java, Go, Pythonand C++. If you're running a local standalone cluster, you canuse one of these root URLs to interact with your cluster:

  • pulsar://localhost:6650
  • http://localhost:8080

The following example will guide you get started with Pulsar quickly by using the Pythonclient API.

Install the Pulsar Python client library directly from PyPI:

Consume a message

Create a consumer and subscribe to the topic:

Install Apache Kafka In Docker Container

Produce a message

Now start a producer to send some test messages:

Get the topic statistics

In Pulsar, you can use REST, Java, or command-line tools to control every aspect of the system.For details on APIs, refer to Admin API Overview.

Install Kafka In Docker Container

In the simplest example, you can use curl to probe the stats for a particular topic:

Install Kafka In Docker Windows 10

The output is something like this:

To install packages in a docker container, the packages should be defined in the Dockerfile. If you want to install packages in the Container, use the RUN statement followed by exact download command. $ RUN pip install //IN Windows $ RUN apt-get install //in Ubuntu $ RUN yum install //CentOS/RHEL. Mac & Windows: install the Docker Toolbox to get Docker installed. As the generated files are in your shared folder, they will not be deleted if you stop your Docker container. However, if you don’t want Docker to keep downloading all the Maven and NPM dependencies every time you start the container, you should commit its state or mount a volume.

Docker Java Example

We shall learn following items in this Docker Java Example :

Install Kafka Docker Windows

Build Docker Image with Java Application

1. Create a directory

A separate directory is useful to organise docker applications. For this Java Example, create a directory somewhere with name of your choice. We shall use the name java-application

2. Create Java Application

Create a simple Java File, in the directory java-application, with name containing the following content.

3. Dockerfile

Create a file with name Dockerfile. Dockerfile contains instructions to prepare Docker image with our Java Application.

Following is the content of Dockerfile.

4. Verify contents of java-application directory

5. Build docker image

Login as root user. Navigate into java-application directory and run the following command. Instructions in the Dockerfile are executed.

Please observe that there is dot (.) at the end of the command. Docker image is successfully built.

6. Check the docker image

To display available docker images, run the following command.

Run Docker Java Example

Run the following command to run the java-application Docker image in a container.

The Java Application has run, and the print statement could be seen in the console.

Save Docker Image to a tar file


Save the Docker Image file to a tar file, so that the image file could be copied to other machines through disk storage devices like pen-drive, etc.

Install Kafka And Zookeeper In Dockerfile

Run the following command to save Docker image as a tar file.

Saving might take few seconds. Wait for the command to complete.

Copy and Run the Docker Image file in another machine

Install Kafka On Docker Windows 10

You may copy the Docker image tar file to another computer.

Install Kafka Docker

Run the following command to load the Docker image into the Docker.

Replace /home/arjun/workspace/java-application.tar with your file location.

You may run the image using the same command we used to run the image file after building.

Install Kafka In Docker Download


Install Kafka In Docker Container

In this Docker Tutorial – Docker Java Example, we have learnt to build a Docker Image with Java Application and also how to save the image to a file and transfer it to other computers or servers.

Comments are closed.