Apache Kudu Quickstart

Follow these instructions to set up and run a local Kudu Cluster using Docker, and get started using Apache Kudu in minutes.

Note: This is intended for demonstration purposes only and shouldn’t be used for production or performance/scale testing.

Install Docker

Follow the Docker install documentation to install docker in your Linux, Mac, or Windows environment.

You may also want to read through the Docker getting started guide, but that isn’t a requirement.

Clone the Repository

Clone the Apache Kudu repository using Git and change to the kudu directory:

$ git clone https://github.com/apache/kudu
$ cd kudu

Start the Quickstart Cluster

Set KUDU_QUICKSTART_IP

Set the KUDU_QUICKSTART_IP environment variable to your ip address:

$ export KUDU_QUICKSTART_IP=$(ifconfig | grep "inet " | grep -Fv 127.0.0.1 |  awk '{print $2}' | tail -1)

Bring up the Cluster

Then use docker-compose to start a cluster with 3 master servers and 5 tablet servers. When inside the docker network/containers the master addresses will be kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251 and when on the host machine you can specify the master addresses with localhost:7051,localhost:7151,localhost:7251.

docker-compose -f docker/quickstart.yml up

Note: You can include the -d flag to run the cluster in the background.

View the Web-UI

Once the cluster is started you can view the master web-ui by visiting localhost:8050.

Check the cluster health

Use the command below to get a bash shell in the kudu-master-1 container:

docker exec -it $(docker ps -aqf "name=kudu-master-1") /bin/bash

You can now run the Kudu ksck tool to verify the cluster is healthy:

kudu cluster ksck kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251

Alternatively, if you have a kudu binary available on your host machine, you can run ksck there via:

export KUDU_USER_NAME=kudu
kudu cluster ksck localhost:7051,localhost:7151,localhost:7251

Note: Setting KUDU_USER_NAME=kudu simplifies using Kudu from various user accounts in a non-secure environment.

Running a Brief Example

Now that a Kudu cluster is up and running, examples and integrations can be run against the cluster. The commands below run the java-example against the quickstart cluster:

export KUDU_USER_NAME=kudu
cd examples/java/java-example
mvn package
java -DkuduMasters=localhost:7051,localhost:7151,localhost:7251 -jar target/kudu-java-example-1.0-SNAPSHOT.jar

More Examples

More complete walkthroughs using the quickstart Kudu cluster can be found in the examples/quickstart directory. For convenience you can browse them on Github.

Destroying the Cluster

Once you are done with the quickstart cluster you can shutdown in a couple of ways. If you ran docker-compose without the -d flag, you can use ctrl + c to stop the cluster.

If you ran docker-compose with the -d flag, you can use the following to gracefully shutdown the cluster:

docker-compose -f docker/quickstart.yml down

Another alternative is to stop all of the Kudu containers via:

docker stop $(docker ps -aqf "name=kudu")

If you want to remove the cluster state you can also remove the docker containers and volumes via:

docker rm $(docker ps -aqf "name=kudu")
docker volume rm $(docker volume ls --filter name=kudu -q)

Troubleshooting

Viewing the logs

To view the logs you can use the docker logs command. Below is an example that will show the logs one of the tablet servers:

docker logs $(docker ps -aqf "name=kudu-tserver-1")

Changing the Kudu version

To change the version of Kudu Docker images used you can override the default value of latest by setting the KUDU_QUICKSTART_VERSION environment variable.

export KUDU_QUICKSTART_VERSION="1.9.0"

Changing the Kudu configuration

To change the configuration flags passed to the master and tablet servers you can edit the docker/quickstart.yml file before starting the cluster.

Killing a Kudu master

Due to KUDU-1620 master hosts are always expected to be reachable.

Next Steps