If you’d prefer to install Heron on Kubernetes without using the Helm package manager, see the Heron on Kubernetes by hand document.

Helm is an open source package manager for Kubernetes that enables you to quickly and easily install even the most complex software systems on Kubernetes. Heron has a Helm chart that you can use to install Heron on Kubernetes using just a few commands. The chart can be used to install Heron on the following platforms:

Requirements

In order to install Heron on Kubernetes using Helm, you’ll need to have an existing Kubernetes cluster on one of the supported platforms (which includes bare metal installations).

Installing the Helm client

In order to get started, you need to install Helm on your machine. Installation instructions for macOS and Linux are below.

Helm for macOS

You can install Helm on macOS using Homebrew:

$ brew install kubernetes-helm

Helm for Linux

You can install Helm on Linux using a simple installation script:

$ curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get > install-helm.sh
$ chmod 700 install-helm.sh
$ ./install-helm.sh

Installing Helm in your Kubernetes cluster

To run Helm on Kubernetes, you need to first make sure that kubectl is using the right configuration context for your cluster. To check which context is being used:

$ kubectl config current-context

If the context is correct, then you can get Helm running using just one command:

$ helm init

If the output of that command includes Happy Helming! then Helm is ready to go.

Installing Heron on Kubernetes

Once you’ve installed the Helm client on your machine and gotten Helm running in your Kubernetes cluster, you need to make your client aware of the heron-charts Helm repository, which houses the chart for Heron:

$ helm repo add heron-charts https://storage.googleapis.com/heron-charts
"heron-charts" has been added to your repositories

Now you can install the Heron package:

$ helm install heron-charts/heron

This will install Heron and provide the installation with a random name like jazzy-anaconda. To provide the installation with a name, such as heron-kubernetes:

$ helm install heron-charts/heron \
  --name heron-kubernetes

Specifying a platform

The default platform for running Heron on Kubernetes is Minikube. To specify a different platform, you can use the --set platform=PLATFORM flag. Here’s an example:

$ helm install heron-charts/heron \
  --set platform=gke

The available platforms are:

Platform Tag
Minikube minikube
Google Kubernetes Engine gke
Amazon Web Services aws
Bare metal baremetal

Minikube

To run Heron on Minikube, you need to first install Minikube. Once Minikube is installed, you can start it by running minikube start. Please note, however, that Heron currently requires the following resources:

  • 7 GB of memory
  • 5 CPUs
  • 20 GB of disk space

To start up Minikube with the minimum necessary resources:

$ minikube start \
  --memory=7168 \
  --cpus=5 \
  --disk-size=20g

Once Minikube is running, you can then install Heron in one of two ways:

# Use the Minikube default
$ helm install heron-charts/heron

# Explicitly select Minikube
$ helm install heron-charts/heron \
  --set platform=minikube

Google Kubernetes Engine

The resources required to run Heron on Google Kubernetes Engine vary based on your use case. To run a basic Heron cluster intended for development and experimentation, you’ll need at least:

To create a cluster with those resources using the gcloud tool:

$ gcloud container clusters create heron-gke-dev-cluster \
  --num-nodes=3 \
  --machine-type=n1-standard-2

For a production-ready cluster you’ll want a larger cluster with:

To create such a cluster:

$ gcloud container clusters create heron-gke-prod-cluster \
  --num-nodes=8 \
  --machine-type=n1-standard-8

Once the cluster has been successfully created, you’ll need to install that cluster’s credentials locally so that they can be used by kubectl. You can do this in just one command:

$ gcloud container clusters get-credentials heron-gke-dev-cluster # or heron-gke-prod-cluster

Once, the cluster is running (that could take a few minutes), you can initialize Helm on the cluster:

$ helm init

Then, you’ll need to adjust some RBAC permissions for your cluster:

$ kubectl create serviceaccount tiller \
  --namespace kube-system \
$ kubectl create clusterrolebinding tiller-cluster-rule \
  --clusterrole cluster-admin \
  --serviceaccount kube-system:tiller
$ kubectl patch deploy tiller-deploy \
  --namespace kube-system \
  --patch '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'

Finally, you can install Heron:

$ helm install heron-charts/heron \
  --set platform=gke
Resource configurations

Helm enables you to supply sets of variables via YAML files. There are currently a handful of different resource configurations that can be applied to your Heron on GKE cluster upon installation:

Configuration Description
small.yaml Smaller Heron cluster intended for basic testing, development, and experimentation
large.yaml Larger Heron cluster intended for production usage

To apply the small configuration, for example:

$ helm install heron-charts/heron \
  --set platform=gke \
  --values https://raw.githubusercontent.com/twitter/heron/master/deploy/kubernetes/gcp/small.yaml

Amazon Web Services

To run Heron on Kubernetes on Amazon Web Services (AWS), you’ll need to

$ helm install heron-charts/heron \
  --set platform=aws

Bare metal

To run Heron on a bare metal Kubernetes cluster:

$ helm install heron-charts/heron \
  --set platform=baremetal

Running topologies on Heron on Kubernetes

Once you have a Heron cluster up and running on Kubernetes via Helm, you can use the heron CLI tool like normal if you set the proper URL for the Heron API server. When running Heron on Kubernetes, that URL is:

http://localhost:8001/api/v1/proxy/namespaces/default/services/heron-kubernetes-heron-apiserver:9000/api/v1/version

To set that URL:

$ heron config heron-kubernetes set service_url \
  http://localhost:8001/api/v1/proxy/namespaces/default/services/heron-kubernetes-heron-apiserver:9000/api/v1/version

When setting the heron CLI configuration, make sure that the cluster name matches the name of the Helm installation. This can be either the name auto-generated by Helm or the name you supplied via the --name flag upon installation (in some of the examples above, the heron-kubernetes name was used). Make sure to adjust the name accordingly if necessary.

To test your cluster, you can submit an example topology:

$ heron submit heron-kubernetes \
  ~/.heron/examples/heron-streamlet-examples.jar \
  com.twitter.heron.examples.streamlet.WindowedWordCountTopology \
  WindowedWordCount