Trait

com.twitter.scalding.typed

HashJoinable

Related Doc: package typed

Permalink

trait HashJoinable[K, +V] extends CoGroupable[K, V] with KeyedPipe[K]

If we can HashJoin, then we can CoGroup, but not vice-versa i.e., HashJoinable is a strict subset of CoGroupable (CoGrouped, for instance is CoGroupable, but not HashJoinable).

Source
HashJoinable.scala
Linear Supertypes
KeyedPipe[K], CoGroupable[K, V], Serializable, HasDescription, HasReducers, AnyRef, Any
Known Subclasses
Type Hierarchy
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. HashJoinable
  2. KeyedPipe
  3. CoGroupable
  4. Serializable
  5. HasDescription
  6. HasReducers
  7. AnyRef
  8. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def descriptions: Seq[String]

    Permalink
    Definition Classes
    HasDescription
  2. abstract def joinFunction: (K, Iterator[Tuple], Seq[Iterable[Tuple]]) ⇒ Iterator[V]

    Permalink

    This function is not type-safe for others to call, but it should never have an error.

    This function is not type-safe for others to call, but it should never have an error. By construction, we never call it with incorrect types. It would be preferable to have stronger type safety here, but unclear how to achieve, and since it is an internal function, not clear it would actually help anyone for it to be type-safe

    Attributes
    protected
    Definition Classes
    CoGroupable
  3. abstract def keyOrdering: Ordering[K]

    Permalink
    Definition Classes
    KeyedPipe
  4. abstract def mapped: TypedPipe[(K, Any)]

    Permalink
    Definition Classes
    KeyedPipe
  5. abstract def reducers: Option[Int]

    Permalink
    Definition Classes
    HasReducers

Concrete Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  6. def cogroup[R1, R2](smaller: CoGroupable[K, R1])(fn: (K, Iterator[V], Iterable[R1]) ⇒ Iterator[R2]): CoGrouped[K, R2]

    Permalink

    Smaller is about average values/key not total size (that does not matter, but is clearly related).

    Smaller is about average values/key not total size (that does not matter, but is clearly related).

    Note that from the type signature we see that the right side is iterated (or may be) over and over, but the left side is not. That means that you want the side with fewer values per key on the right. If both sides are similar, no need to worry. If one side is a one-to-one mapping, that should be the "smaller" side.

    Definition Classes
    CoGroupable
  7. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  8. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  9. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  10. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  11. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  12. def hashCogroupOn[V1, R](mapside: TypedPipe[(K, V1)])(joiner: (K, V1, Iterable[V]) ⇒ Iterator[R]): TypedPipe[(K, R)]

    Permalink

    This fully replicates this entire Grouped to the argument: mapside.

    This fully replicates this entire Grouped to the argument: mapside. This means that we never see the case where the key is absent in the pipe. This means implementing a right-join (from the pipe) is impossible. Note, there is no reduce-phase in this operation. The next issue is that obviously, unlike a cogroup, for a fixed key, each joiner will NOT See all the tuples with those keys. This is because the keys on the left are distributed across many machines See hashjoin: http://docs.cascading.org/cascading/2.0/javadoc/cascading/pipe/HashJoin.html

  13. def inputs: List[TypedPipe[(K, Any)]]

    Permalink

    A HashJoinable has a single input into to the cogroup

    A HashJoinable has a single input into to the cogroup

    Definition Classes
    HashJoinableCoGroupable
  14. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  15. def join[W](smaller: CoGroupable[K, W]): CoGrouped[K, (V, W)]

    Permalink
    Definition Classes
    CoGroupable
  16. def leftJoin[W](smaller: CoGroupable[K, W]): CoGrouped[K, (V, Option[W])]

    Permalink
    Definition Classes
    CoGroupable
  17. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  18. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  19. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  20. def outerJoin[W](smaller: CoGroupable[K, W]): CoGrouped[K, (Option[V], Option[W])]

    Permalink
    Definition Classes
    CoGroupable
  21. def rightJoin[W](smaller: CoGroupable[K, W]): CoGrouped[K, (Option[V], W)]

    Permalink
    Definition Classes
    CoGroupable
  22. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  23. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  24. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  25. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from KeyedPipe[K]

Inherited from CoGroupable[K, V]

Inherited from Serializable

Inherited from HasDescription

Inherited from HasReducers

Inherited from AnyRef

Inherited from Any

Ungrouped