This document contains information about the Heron codebase intended primarily for developers who want to contribute to Heron. The Heron codebase lives on github.
If you’re looking for documentation about developing topologies for a Heron cluster, see Building Topologies instead.
The primary programming languages for Heron are C++, Java, and Python.
Java 8 is used primarily for Heron’s topology API, and Heron Instance. It is currently the only language in which topologies can be written. Instructions can be found in Building Topologies, while documentation for the Java API can be found here. Please note that Heron topologies do not require Java 8 and can be written in Java 7 or later.
Cluster coordination — Heron relies heavily on ZooKeeper for cluster coordination for distributed deployment, be it for Aurora or for a custom scheduler that you build. More information on ZooKeeper components in the codebase can be found in the State Management section below.
heron/common contains a variety of
utilities for each of Heron’s languages, including useful constants, file
utilities, networking interfaces, and more.
Heron supports two cluster schedulers out of the box:
Aurora and a local
scheduler. The Java code for each of those
schedulers can be found in
, while the underlying scheduler API can be found here
If you’d like to implement your own custom metrics handler (known as a metrics sink), see Implementing a Custom Metrics Sink.
Heron’s API for writing topologies is written in Java. The code for this API can
be found in
Heron enables you to run topologies in
for debugging purposes.
The Java API for simulator can be found in
User Interface Components
Heron has a tool called
heron that is used to both provide a CLI interface
for managing topologies and to perform much of
the heavy lifting behind assembling physical topologies in your cluster.
The Python code for
heron can be found in
Sample configurations for different Heron schedulers
- Local scheduler config can be found in
- Aurora scheduler config can be found
Like Heron Tracker, Heron UI is a web server written in Python that relies on
the Tornado framework. You can add new
HTTP routes to Heron UI in
main.py and corresponding
handlers in the
The Python code for the Heron Shell can be
heron/shell. The HTTP handlers and
web server are defined in
main.py while the HTML,
There are a wide variety of tests for Heron that are scattered throughout the codebase. For more info see Testing Heron.