Class

com.twitter.algebird

SetSizeHashAggregator

Related Doc: package algebird

Permalink

case class SetSizeHashAggregator[A](hllBits: Int, maxSetSize: Int = 10)(implicit hash: Hash128[A]) extends SetSizeAggregatorBase[A] with Product with Serializable

Use a Hash128 when converting to HLL, rather than an implicit conversion to Array[Byte] Unifying with SetSizeAggregator would be nice, but since they only differ in an implicit parameter, scala seems to be giving me errors.

Source
HyperLogLog.scala
Linear Supertypes
Serializable, Product, Equals, SetSizeAggregatorBase[A], EventuallyMonoidAggregator[A, HLL, Set[A], Long], MonoidAggregator[A, Either[HLL, Set[A]], Long], AbstractEventuallyAggregator[A, HLL, Set[A], Long], Aggregator[A, Either[HLL, Set[A]], Long], Serializable, AnyRef, Any
Type Hierarchy
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SetSizeHashAggregator
  2. Serializable
  3. Product
  4. Equals
  5. SetSizeAggregatorBase
  6. EventuallyMonoidAggregator
  7. MonoidAggregator
  8. AbstractEventuallyAggregator
  9. Aggregator
  10. Serializable
  11. AnyRef
  12. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SetSizeHashAggregator(hllBits: Int, maxSetSize: Int = 10)(implicit hash: Hash128[A])

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def andThenPresent[D](present2: (Long) ⇒ D): MonoidAggregator[A, Either[HLL, Set[A]], D]

    Permalink

    Like calling andThen on the present function

    Like calling andThen on the present function

    Definition Classes
    MonoidAggregatorAggregator
  5. def append(l: Either[HLL, Set[A]], r: A): Either[HLL, Set[A]]

    Permalink
    Definition Classes
    Aggregator
  6. def appendAll(items: TraversableOnce[A]): Either[HLL, Set[A]]

    Permalink
    Definition Classes
    MonoidAggregator
  7. def appendAll(old: Either[HLL, Set[A]], items: TraversableOnce[A]): Either[HLL, Set[A]]

    Permalink
    Definition Classes
    Aggregator
  8. def apply(inputs: TraversableOnce[A]): Long

    Permalink

    This may error if inputs are empty (for Monoid Aggregators it never will, instead you see present(Monoid.zero[B])

    This may error if inputs are empty (for Monoid Aggregators it never will, instead you see present(Monoid.zero[B])

    Definition Classes
    Aggregator
  9. def applyCumulatively[In <: TraversableOnce[A], Out](inputs: In)(implicit bf: CanBuildFrom[In, Long, Out]): Out

    Permalink

    This returns the cumulative sum of its inputs, in the same order.

    This returns the cumulative sum of its inputs, in the same order. If the inputs are empty, the result will be empty too.

    Definition Classes
    Aggregator
  10. def applyOption(inputs: TraversableOnce[A]): Option[Long]

    Permalink

    This returns None if the inputs are empty

    This returns None if the inputs are empty

    Definition Classes
    Aggregator
  11. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  12. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  13. def composePrepare[A2](prepare2: (A2) ⇒ A): MonoidAggregator[A2, Either[HLL, Set[A]], Long]

    Permalink

    Like calling compose on the prepare function

    Like calling compose on the prepare function

    Definition Classes
    MonoidAggregatorAggregator
  14. def convert(set: Set[A]): HLL

    Permalink
  15. def cumulativeIterator(inputs: Iterator[A]): Iterator[Long]

    Permalink

    This returns the cumulative sum of its inputs, in the same order.

    This returns the cumulative sum of its inputs, in the same order. If the inputs are empty, the result will be empty too.

    Definition Classes
    Aggregator
  16. def either[A2, B2, C2](that: MonoidAggregator[A2, B2, C2]): MonoidAggregator[Either[A, A2], (Either[HLL, Set[A]], B2), (Long, C2)]

    Permalink

    Build a MonoidAggregator that either takes left or right input and outputs the pair from both

    Build a MonoidAggregator that either takes left or right input and outputs the pair from both

    Definition Classes
    MonoidAggregator
  17. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  18. def filterBefore[A1 <: A](pred: (A1) ⇒ Boolean): MonoidAggregator[A1, Either[HLL, Set[A]], Long]

    Permalink

    Only aggregate items that match a predicate

    Only aggregate items that match a predicate

    Definition Classes
    MonoidAggregator
  19. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  20. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  21. val hllBits: Int

    Permalink
  22. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  23. def join[A2 <: A, B2, C2](that: Aggregator[A2, B2, C2]): Aggregator[A2, (Either[HLL, Set[A]], B2), (Long, C2)]

    Permalink

    This allows you to run two aggregators on the same data with a single pass

    This allows you to run two aggregators on the same data with a single pass

    Definition Classes
    Aggregator
  24. val leftSemigroup: HyperLogLogMonoid

    Permalink
  25. def lift: MonoidAggregator[A, Option[Either[HLL, Set[A]]], Option[Long]]

    Permalink
    Definition Classes
    Aggregator
  26. val maxSetSize: Int

    Permalink
  27. lazy val monoid: EventuallyMonoid[HLL, Set[A]]

    Permalink
  28. def mustConvert(set: Set[A]): Boolean

    Permalink
  29. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  30. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  31. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  32. def prepare(a: A): Right[Nothing, Set[A]]

    Permalink
  33. def present(b: Either[HLL, Set[A]]): Long

    Permalink
  34. def presentLeft(hll: HLL): Long

    Permalink
  35. final def reduce(items: TraversableOnce[Either[HLL, Set[A]]]): Either[HLL, Set[A]]

    Permalink

    This may error if items is empty.

    This may error if items is empty. To be safe you might use reduceOption if you don't know that items is non-empty

    Definition Classes
    MonoidAggregatorAggregator
  36. def reduce(l: Either[HLL, Set[A]], r: Either[HLL, Set[A]]): Either[HLL, Set[A]]

    Permalink

    combine two inner values

    combine two inner values

    Definition Classes
    Aggregator
  37. def reduceOption(items: TraversableOnce[Either[HLL, Set[A]]]): Option[Either[HLL, Set[A]]]

    Permalink

    This is the safe version of the above.

    This is the safe version of the above. If the input in empty, return None, else reduce the items

    Definition Classes
    Aggregator
  38. val rightAggregator: MonoidAggregator[A, Set[A], Long]

    Permalink
  39. def semigroup: Monoid[Either[HLL, Set[A]]]

    Permalink
    Definition Classes
    MonoidAggregatorAggregator
  40. def sumBefore: MonoidAggregator[TraversableOnce[A], Either[HLL, Set[A]], Long]

    Permalink

    This maps the inputs to Bs, then sums them, effectively flattening the inputs to the MonoidAggregator

    This maps the inputs to Bs, then sums them, effectively flattening the inputs to the MonoidAggregator

    Definition Classes
    MonoidAggregator
  41. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  42. def toFold: Fold[A, Option[Long]]

    Permalink

    An Aggregator can be converted to a Fold, but not vice-versa Note, a Fold is more constrained so only do this if you require joining a Fold with an Aggregator to produce a Fold

    An Aggregator can be converted to a Fold, but not vice-versa Note, a Fold is more constrained so only do this if you require joining a Fold with an Aggregator to produce a Fold

    Definition Classes
    Aggregator
  43. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  44. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  45. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  46. def zip[A2, B2, C2](ag2: MonoidAggregator[A2, B2, C2]): MonoidAggregator[(A, A2), (Either[HLL, Set[A]], B2), (Long, C2)]

    Permalink

    This allows you to join two aggregators into one that takes a tuple input, which in turn allows you to chain .composePrepare onto the result if you have an initial input that has to be prepared differently for each of the joined aggregators.

    This allows you to join two aggregators into one that takes a tuple input, which in turn allows you to chain .composePrepare onto the result if you have an initial input that has to be prepared differently for each of the joined aggregators.

    The law here is: ag1.zip(ag2).apply(as.zip(bs)) == (ag1(as), ag2(bs))

    Definition Classes
    MonoidAggregator
  47. def zip[A2, B2, C2](ag2: Aggregator[A2, B2, C2]): Aggregator[(A, A2), (Either[HLL, Set[A]], B2), (Long, C2)]

    Permalink

    This allows you to join two aggregators into one that takes a tuple input, which in turn allows you to chain .composePrepare onto the result if you have an initial input that has to be prepared differently for each of the joined aggregators.

    This allows you to join two aggregators into one that takes a tuple input, which in turn allows you to chain .composePrepare onto the result if you have an initial input that has to be prepared differently for each of the joined aggregators.

    The law here is: ag1.zip(ag2).apply(as.zip(bs)) == (ag1(as), ag2(bs))

    Definition Classes
    Aggregator

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from SetSizeAggregatorBase[A]

Inherited from EventuallyMonoidAggregator[A, HLL, Set[A], Long]

Inherited from MonoidAggregator[A, Either[HLL, Set[A]], Long]

Inherited from AbstractEventuallyAggregator[A, HLL, Set[A], Long]

Inherited from Aggregator[A, Either[HLL, Set[A]], Long]

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped