Object

com.twitter.scalding

Dsl

Related Doc: package scalding

Permalink

object Dsl extends FieldConversions with Serializable

This object has all the implicit functions and values that are used to make the scalding DSL, which includes the functions for automatically creating cascading.tuple.Fields objects from scala tuples of Strings, Symbols or Ints, as well as the cascading.pipe.Pipe enrichment to RichPipe which adds the scala.collections-like API to Pipe.

It's useful to import Dsl._ when you are writing scalding code outside of a Job.

Source
Dsl.scala
Linear Supertypes
Serializable, FieldConversions, LowPriorityFieldConversions, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Dsl
  2. Serializable
  3. FieldConversions
  4. LowPriorityFieldConversions
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def anyToFieldArg(f: Any): Comparable[_]

    Permalink
    Attributes
    protected
    Definition Classes
    LowPriorityFieldConversions
  5. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  6. def asList(f: Fields): List[Comparable[_]]

    Permalink
    Definition Classes
    FieldConversions
  7. def asSet(f: Fields): Set[Comparable[_]]

    Permalink
    Definition Classes
    FieldConversions
  8. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  9. def defaultMode(fromFields: Fields, toFields: Fields): Fields

    Permalink

    Rather than give the full power of cascading's selectors, we have a simpler set of rules encoded below: 1) if the input is non-definite (ALL, GROUP, ARGS, etc...) ALL is the output.

    Rather than give the full power of cascading's selectors, we have a simpler set of rules encoded below: 1) if the input is non-definite (ALL, GROUP, ARGS, etc...) ALL is the output. Perhaps only fromFields=ALL will make sense 2) If one of from or to is a strict super set of the other, SWAP is used. 3) If they are equal, REPLACE is used. 4) Otherwise, ALL is used.

    Definition Classes
    FieldConversions
  10. final def ensureUniqueFields(left: Fields, right: Fields, rightPipe: Pipe): (Fields, Pipe)

    Permalink
    Definition Classes
    FieldConversions
  11. implicit def enumValueToFields(x: Value): Fields

    Permalink
    Definition Classes
    FieldConversions
  12. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  13. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  14. implicit def fieldFields[T <: TraversableOnce[Field[_]]](f: T): RichFields

    Permalink
    Definition Classes
    FieldConversions
  15. implicit def fieldToFields(f: Field[_]): RichFields

    Permalink
    Definition Classes
    FieldConversions
  16. implicit def fields[T <: TraversableOnce[Symbol]](f: T): Fields

    Permalink
    Definition Classes
    FieldConversions
  17. implicit def fieldsToRichFields(fields: Fields): RichFields

    Permalink

    We can't set the field Manifests because cascading doesn't (yet) expose field type information in the Fields API.

    We can't set the field Manifests because cascading doesn't (yet) expose field type information in the Fields API.

    Definition Classes
    FieldConversions
  18. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  19. implicit def flowDefToRichFlowDef(fd: FlowDef): RichFlowDef

    Permalink

    Enrichment on FlowDef

  20. implicit def fromEnum[T <: Enumeration](enumeration: T): Fields

    Permalink

    Multi-entry fields.

    Multi-entry fields. This are higher priority than Product conversions so that List will not conflict with Product.

    Definition Classes
    FieldConversions
  21. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  22. def getField(f: Fields, idx: Int): Fields

    Permalink
    Definition Classes
    FieldConversions
  23. def hasInts(f: Fields): Boolean

    Permalink
    Definition Classes
    FieldConversions
  24. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  25. implicit def intFields[T <: TraversableOnce[Int]](f: T): Fields

    Permalink
    Definition Classes
    FieldConversions
  26. implicit def intToFields(x: Int): Fields

    Permalink
    Definition Classes
    FieldConversions
  27. implicit def integerToFields(x: Integer): Fields

    Permalink
    Definition Classes
    FieldConversions
  28. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  29. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  30. final def newSymbol(avoid: Set[Symbol], guess: Symbol, trial: Int = 0): Symbol

    Permalink
    Definition Classes
    FieldConversions
    Annotations
    @tailrec()
  31. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  32. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  33. implicit def parseAnySeqToFields[T <: TraversableOnce[Any]](anyf: T): Fields

    Permalink

    Useful to convert f : Any* to Fields.

    Useful to convert f : Any* to Fields. This handles mixed cases ("hey", 'you). Not sure we should be this flexible, but given that Cascading will throw an exception before scheduling the job, I guess this is okay.

    Definition Classes
    FieldConversions
  34. implicit def pipeToRichPipe(pipe: Pipe): RichPipe

    Permalink
  35. implicit def productToFields(f: Product): Fields

    Permalink

    Handles treating any TupleN as a Fields object.

    Handles treating any TupleN as a Fields object. This is low priority because List is also a Product, but this method will not work for List (because List is Product2(head, tail) and so productIterator won't work as expected. Lists are handled by an implicit in FieldConversions, which have higher priority.

    Definition Classes
    LowPriorityFieldConversions
  36. implicit def strFields[T <: TraversableOnce[String]](f: T): Fields

    Permalink
    Definition Classes
    FieldConversions
  37. implicit def stringToFields(x: String): Fields

    Permalink
    Definition Classes
    FieldConversions
  38. implicit def symbolToFields(x: Symbol): Fields

    Permalink

    '* means Fields.ALL, otherwise we take the .name

    '* means Fields.ALL, otherwise we take the .name

    Definition Classes
    FieldConversions
  39. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  40. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  41. implicit def tuple2ToFieldsPair[T, U](pair: (T, U))(implicit tf: (T) ⇒ Fields, uf: (U) ⇒ Fields): (Fields, Fields)

    Permalink
    Definition Classes
    FieldConversions
  42. implicit def unitToFields(u: Unit): Fields

    Permalink
    Definition Classes
    FieldConversions
  43. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  44. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  45. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from FieldConversions

Inherited from AnyRef

Inherited from Any

Ungrouped