A topology specifies components like spouts and bolts, as well as the relation between components and proper configurations.
Install Heron APIs for development
Before getting started writing a topology, you need to install the Heron API and import its library into your own topology project.
Go to the releases page for Heron and download the Heron API installation script for your platform. The name of the script for Mac OS X (
darwin), for example, would be
Once you’ve downloaded, run it with the
After successful installation, import
~/.heronapi/heron-storm.jarinto your project as a dependency. This allows you to use the Heron APIs that are necessary to develop your own topology.
Alternatively, you can integrate the latest Heron API by including
the following lines in your project’s
<dependency> <groupId>com.twitter.heron</groupId> <artifactId>heron-storm</artifactId> <version>0.14.5</version> </dependency>
Writing your own topology
After defining the spouts and bolts, a topology can be composed using
TopologyBuilder has two major methods to specify the components:
setBolt(String id, IRichBolt bolt, Number parallelismHint):
idis the unique identifier that assigned to a bolt,
boltis the one previously composed, and
parallelismHintis a number that specifying the number of instances of this bolt.
setSpout(String id, IRichSpout spout, Number parallelismHint):
idis the unique identifier that assigned to a spout,
spoutis the one previously composed, and
parallelismHintis a number that specifying the number of instances of this spout.
A simple example is as follows:
TopologyBuilder builder = new TopologyBuilder(); builder.setSpout("word", new TestWordSpout(), 5); builder.setBolt("exclaim", new ExclamationBolt(), 4);
In addition to the component specification, how to transmit Tuples between the components must also be specified. This is defined by different grouping strategies:
- Fields Grouping: Tuples are transmitted to bolts based on a given field. Tuples with the same field will always go to the same bolt.
- Global Grouping: All the Tuples are transmitted to a single instance of a bolt with the lowest task id.
- Shuffle Grouping: Tuples are randomly transmitted to different instances of a bolt.
- None Grouping: Currently, it equals to shuffle grouping.
- All Grouping: All Tuples are transmitted to all instances of a bolt.
- Custom Grouping: User-defined grouping strategy.
The following snippet is a simple example of specifying shuffle grouping
word spout and
builder.setBolt("exclaim", new ExclamationBolt(), 4) .shuffleGrouping("word");
Once the components and the grouping are specified, the topology can be built.
HeronTopology topology = builder.createTopology();