Class

com.twitter.scalding

InvalidSourceTap

Related Doc: package scalding

Permalink

class InvalidSourceTap extends SourceTap[JobConf, RecordReader[_, _]]

InvalidSourceTap used in createTap method when we want to defer the failures to validateTaps method.

This is used because for Job classes, createTap method on sources is called when the class is initialized. In most cases though, we want any exceptions to be thrown by validateTaps method, which is called subsequently during flow planning.

hdfsPaths represents user-supplied list that was detected as not containing any valid paths.

Source
Source.scala
Linear Supertypes
SourceTap[JobConf, RecordReader[_, _]], Tap[JobConf, RecordReader[_, _], Void], Traceable, FlowElement, Serializable, AnyRef, Any
Type Hierarchy
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. InvalidSourceTap
  2. SourceTap
  3. Tap
  4. Traceable
  5. FlowElement
  6. Serializable
  7. AnyRef
  8. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new InvalidSourceTap(hdfsPaths: Iterable[String])

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  6. def commitResource(arg0: JobConf): Boolean

    Permalink
    Definition Classes
    SourceTap → Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  7. def createResource(arg0: JobConf): Boolean

    Permalink
    Definition Classes
    SourceTap → Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  8. def createResource(arg0: FlowProcess[JobConf]): Boolean

    Permalink
    Definition Classes
    Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  9. def deleteResource(arg0: JobConf): Boolean

    Permalink
    Definition Classes
    SourceTap → Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  10. def deleteResource(arg0: FlowProcess[JobConf]): Boolean

    Permalink
    Definition Classes
    Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  11. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  12. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    Tap → AnyRef → Any

  13. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  14. def flowConfInit(arg0: Flow[JobConf]): Unit

    Permalink
    Definition Classes
    Tap

  15. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  16. def getConfigDef(): ConfigDef

    Permalink
    Definition Classes
    Tap → FlowElement

  17. def getFullIdentifier(arg0: JobConf): String

    Permalink
    Definition Classes
    Tap

  18. def getFullIdentifier(arg0: FlowProcess[JobConf]): String

    Permalink
    Definition Classes
    Tap

  19. def getIdentifier(): String

    Permalink
    Definition Classes
    InvalidSourceTap → Tap

  20. def getModifiedTime(conf: JobConf): Long

    Permalink
    Definition Classes
    InvalidSourceTap → Tap

  21. def getModifiedTime(arg0: FlowProcess[JobConf]): Long

    Permalink
    Definition Classes
    Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  22. def getScheme(): NullScheme[JobConf, RecordReader[_, _], Void, Nothing, Nothing]

    Permalink
    Definition Classes
    InvalidSourceTap → Tap

  23. def getSinkFields(): Fields

    Permalink
    Definition Classes
    SourceTap → Tap

  24. def getSinkMode(): SinkMode

    Permalink
    Definition Classes
    Tap

  25. def getSourceFields(): Fields

    Permalink
    Definition Classes
    Tap

  26. def getStepConfigDef(): ConfigDef

    Permalink
    Definition Classes
    Tap → FlowElement

  27. def getTrace(): String

    Permalink
    Definition Classes
    Tap → Traceable

  28. def hasConfigDef(): Boolean

    Permalink
    Definition Classes
    Tap → FlowElement

  29. def hasStepConfigDef(): Boolean

    Permalink
    Definition Classes
    Tap → FlowElement

  30. def hashCode(): Int

    Permalink
    Definition Classes
    InvalidSourceTap → Tap → AnyRef → Any

  31. val hdfsPaths: Iterable[String]

    Permalink

  32. def isEquivalentTo(arg0: FlowElement): Boolean

    Permalink
    Definition Classes
    Tap → FlowElement

  33. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  34. def isKeep(): Boolean

    Permalink
    Definition Classes
    Tap

  35. def isReplace(): Boolean

    Permalink
    Definition Classes
    Tap

  36. final def isSink(): Boolean

    Permalink
    Definition Classes
    SourceTap → Tap

  37. def isSource(): Boolean

    Permalink
    Definition Classes
    Tap

  38. def isTemporary(): Boolean

    Permalink
    Definition Classes
    Tap

  39. def isUpdate(): Boolean

    Permalink
    Definition Classes
    Tap

  40. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  41. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  42. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  43. def openForRead(flow: FlowProcess[JobConf], input: RecordReader[_, _]): TupleEntryIterator

    Permalink
    Definition Classes
    InvalidSourceTap → Tap

  44. def openForRead(arg0: FlowProcess[JobConf]): TupleEntryIterator

    Permalink
    Definition Classes
    Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  45. def openForWrite(arg0: FlowProcess[JobConf], arg1: Void): TupleEntryCollector

    Permalink
    Definition Classes
    SourceTap → Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  46. def openForWrite(arg0: FlowProcess[JobConf]): TupleEntryCollector

    Permalink
    Definition Classes
    Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  47. def outgoingScopeFor(arg0: Set[Scope]): Scope

    Permalink
    Definition Classes
    Tap → FlowElement

  48. def presentSinkFields(arg0: FlowProcess[JobConf], arg1: Fields): Unit

    Permalink
    Definition Classes
    Tap

  49. def presentSourceFields(arg0: FlowProcess[JobConf], arg1: Fields): Unit

    Permalink
    Definition Classes
    Tap

  50. def resolveIncomingOperationArgumentFields(arg0: Scope): Fields

    Permalink
    Definition Classes
    Tap → FlowElement

  51. def resolveIncomingOperationPassThroughFields(arg0: Scope): Fields

    Permalink
    Definition Classes
    Tap → FlowElement

  52. def resourceExists(conf: JobConf): Boolean

    Permalink
    Definition Classes
    InvalidSourceTap → Tap

  53. def resourceExists(arg0: FlowProcess[JobConf]): Boolean

    Permalink
    Definition Classes
    Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  54. def retrieveSinkFields(arg0: FlowProcess[JobConf]): Fields

    Permalink
    Definition Classes
    Tap

  55. def retrieveSourceFields(arg0: FlowProcess[JobConf]): Fields

    Permalink
    Definition Classes
    Tap

  56. def rollbackResource(arg0: JobConf): Boolean

    Permalink
    Definition Classes
    SourceTap → Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  57. def setScheme(arg0: Scheme[JobConf, RecordReader[_, _], Void, _, _]): Unit

    Permalink
    Attributes
    protected[cascading.tap]
    Definition Classes
    Tap

  58. def sinkConfInit(arg0: FlowProcess[JobConf], arg1: JobConf): Unit

    Permalink
    Definition Classes
    SourceTap → Tap

  59. def sourceConfInit(flow: FlowProcess[JobConf], conf: JobConf): Unit

    Permalink
    Definition Classes
    InvalidSourceTap → Tap

  60. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  61. def toString(): String

    Permalink
    Definition Classes
    Tap → AnyRef → Any

  62. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  63. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  64. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from SourceTap[JobConf, RecordReader[_, _]]

Inherited from Tap[JobConf, RecordReader[_, _], Void]

Inherited from Traceable

Inherited from FlowElement

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped