package scalding

Linear Supertypes
AnyRef, Any
  1. Alphabetic
  2. By inheritance
  1. scalding
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Learn more about member selection
  1. Public
  2. All

Type Members

  1. trait ExternalService[K, +V] extends Service[K, V]

    This represents a service that is *external* to the current job.

  2. type FactoryInput = (Interval[Timestamp], Mode)

    The Platform recursively passes this input around to describe a step forward: requested input time span, and scalding Mode

  3. type FailureReason = String

    These are printed/logged only when we can't make any progress

  4. type FlowInput = (FlowDef, Mode)

    When it is time to run build the final flow, this is what scalding needs.

    When it is time to run build the final flow, this is what scalding needs. It is modified in the Reader[FlowInput, T]

  5. case class FlowPlanException(errs: List[FailureReason]) extends Exception with Product with Serializable

    Puts a type on the final exception thrown by the scalding Planner

  6. type FlowProducer[+T] = Reader[(FlowDef, Mode), T]

    This is a function that modifies a flow to return T generally T will be some kind of TypedPipe

  7. type FlowToPipe[+T] = Reader[(FlowDef, Mode), TimedPipe[T]]

    We so commonly talk about producing TimedPipe we define this

  8. sealed trait InternalService[K, +V] extends Service[K, V]

    This represents a join against data that is materialized by a store in the current job

  9. type KeyValuePipe[+K, +V] = TypedPipe[(Timestamp, (K, V))]

  10. class LocalIterableSource[+T] extends IterableSource[T]

  11. class LoopState[T] extends WaitingState[T]

  12. class Memo[T] extends Serializable

  13. class MockMappable[T] extends scalding.Source with Mappable[T]

  14. type PipeFactory[+T] = StateWithError[(Interval[Timestamp], Mode), List[FailureReason], FlowToPipe[T]]

    We are usually producing Pipes in the Planner

  15. class PipeFactoryOps[+T] extends AnyRef

  16. type PlannerOutput[+T] = StateWithError[(Interval[Timestamp], Mode), List[FailureReason], T]

    The recursive planner produces these objects which are Monads

  17. class Scalding extends Platform[Scalding] with Serializable

  18. case class ScaldingEnv(jobName: String, inargs: Array[String]) extends Env with Product with Serializable

  19. trait ScaldingExecutionConfig extends ChillExecutionConfig[Scalding]

  20. sealed trait Service[K, +V] extends Serializable

  21. trait Sink[T] extends AnyRef

  22. trait Store[K, V] extends Serializable

  23. case class StoreIntermediateData[K, V](sink: Sink[(K, V)]) extends Serializable with Product with Serializable

  24. case class StoreService[K, V](store: BatchedStore[K, V]) extends InternalService[K, V] with Store[K, V] with Product with Serializable

  25. class TestService[K, V] extends BatchedService[K, V]

  26. class TestSink[T] extends Sink[T]

    This is a test sink that assumes single threaded testing with cascading local mode

  27. class TestStore[K, V] extends BatchedStore[K, V]

  28. class TestStoreService[K, V] extends StoreService[K, V]

  29. type TimedPipe[+T] = TypedPipe[(Timestamp, T)]

    How we represent the streams in scalding

  30. type Try[+T] = Either[List[FailureReason], T]

  31. case class WriteDot(filename: String) extends Product with Serializable

    Use this option to write the logical graph that cascading produces before Map/Reduce planning.

  32. case class WriteStepsDot(filename: String) extends Product with Serializable

    Use this option to write map/reduce graph that cascading produces Use the job name as the key

Value Members

  1. object Executor

  2. object LookupJoin extends Serializable

    lookupJoin simulates the behavior of a realtime system attempting to leftJoin (K, V) pairs against some other value type (JoinedV) by performing realtime lookups on a key-value Store.

  3. object Scalding extends Serializable

  4. val ScaldingConfig: BatchConfig.type

  5. object TestSource

  6. object TestStore extends Serializable

  7. object TestStoreService extends Serializable

  8. object TestUtil

  9. package batch

  10. implicit def flowDefFromTuple(implicit fm: (FlowDef, Mode)): FlowDef

  11. implicit def modeFromTuple(implicit fm: (FlowDef, Mode)): Mode

  12. package service

  13. package source

  14. package store

  15. implicit def toPipeFactoryOps[T](pipeF: PipeFactory[T]): PipeFactoryOps[T]

  16. def toTry(e: Throwable, msg: String = ""): Try[Nothing]

Inherited from AnyRef

Inherited from Any