Finatra Kafka Streams¶
Finatra has native integration with Kafka Streams to easily build Kafka Streams applications on top of a TwitterServer.
Note
Versions of finatra-kafka and finatra-kafka-streams that are published against Scala 2.12 use Kafka 2.2, versions of that are published against Scala 2.13 use Kafka 2.5. This simplified cross-version support is ephemeral until we can drop Kafka 2.2.
Features¶
- Intuitive DSL for topology creation, compatible with the Kafka Streams DSL
- Full Kafka Streams metric integration, exposed as TwitterServer Metrics
- RocksDB integration
- Queryable State
- Rich testing functionality
Basics¶
With KafkaStreamsTwitterServer,
a fully functional service can be written by simply configuring the Kafka Streams Builder via the configureKafkaStreams() lifecycle method. See the examples section.
Transformers¶
Implement custom transformers using FinatraTransformer.
Aggregations¶
- There are several included aggregating transformers, which may be used when configuring a
StreamsBuilder aggregatesamplesum
Stores¶
RocksDB¶
In addition to using state stores, you may also use a RocksDB-backed store. This affords all of the advantages of using RocksDB, including efficient range scans.
Queryable State¶
Finatra Kafka Streams supports directly querying state from a store. This can be useful for creating a service that serves data aggregated within a local Topology. You can use static partitioning to query an instance deterministically known to hold a key.
See how queryable state is used in the following example.
