The SummerBolt takes two related options: CacheSize and MaxWaitingFutures.
CacheSize sets the number of key-value pairs that the SinkBolt will accept
(and sum into an internal map) before committing out to the online store.
To perform this commit, the SinkBolt iterates through the map of aggregated
kv pairs and performs a "+" on the store for each pair, sequencing these
"+" calls together using the Future monad. If the store has high latency,
these calls might take a bit of time to complete.
MaxWaitingFutures(count) handles this problem by realizing a future
representing the "+" of kv-pair n only when kvpair n + 100 shows up in the bolt,
effectively pushing back against latency bumps in the host.
The allowed latency before a future is forced is equal to
(MaxWaitingFutures * execute latency).
If you can use Future.value below, do so. The double Future is here to deal with
cases that need to complete operations after or before doing a FlatMapOperation or
doing a store merge
The SummerBolt takes two related options: CacheSize and MaxWaitingFutures. CacheSize sets the number of key-value pairs that the SinkBolt will accept (and sum into an internal map) before committing out to the online store.
To perform this commit, the SinkBolt iterates through the map of aggregated kv pairs and performs a "+" on the store for each pair, sequencing these "+" calls together using the Future monad. If the store has high latency, these calls might take a bit of time to complete.
MaxWaitingFutures(count) handles this problem by realizing a future representing the "+" of kv-pair n only when kvpair n + 100 shows up in the bolt, effectively pushing back against latency bumps in the host.
The allowed latency before a future is forced is equal to (MaxWaitingFutures * execute latency).