Execution
Related Docs:
object Execution
| package scalding
This is the internal method that must be implemented Given a config, mode, and cache of evaluations for this config and mode, return the new cache with as much evaluation as possible before the future completes, and a future of the result, counters and cache after the future is complete
This is the internal method that must be implemented Given a config, mode, and cache of evaluations for this config and mode, return the new cache with as much evaluation as possible before the future completes, and a future of the result, counters and cache after the future is complete
Scala uses the filter method in for syntax for pattern matches that can fail.
Scala uses the filter method in for syntax for pattern matches that can fail. If this filter is false, the result of run will be an exception in the future
First run this Execution, then move to the result of the function
This is the same as flatMap(identity)
Reads the counters and resets them to zero.
Reads the counters and resets them to zero. Probably what you want in a loop that is using counters to check for convergence.
Reads the counters into the value, but does not reset them.
Reads the counters into the value, but does not reset them. You may want .getAndResetCounters.
Lift an Execution into a Try
Lift an Execution into a Try
When this function is called the Execution should never be failed instead only the Try.
Apply a pure function to the result.
Apply a pure function to the result. This may not be called if subsequently the result is discarded with .unit For side effects see onComplete.
This function is called when the current run is completed.
This function is called when the current run is completed. This is only a side effect (see unit return).
ALSO You must .run the result. If you throw away the result of this call, your fn will never be called. When you run the result, the Future you get will not be complete unless fn has completed running. If fn throws, it will be handled be the scala.concurrent.ExecutionContext.reportFailure NOT by returning a Failure in the Future.
This allows you to handle a failure by giving a replacement execution in some cases.
This allows you to handle a failure by giving a replacement execution in some cases. This execution may be a retry if you know that your execution can have spurious errors, or it could be a constant or an alternate way to compute. Be very careful creating looping retries that could hammer your cluster when the data is missing or when when there is some real problem with your job logic.
Resets the counters back to zero.
Resets the counters back to zero. This is useful if you want to reset before a zip or a call to flatMap
This causes the Execution to occur.
This causes the Execution to occur. The result is not cached, so each call to run will result in the computation being re-run. Avoid calling this until the last possible moment by using flatMap, zip and recoverWith.
Seriously: pro-style is for this to be called only once in a program.
This is convenience for when we don't care about the result.
This is convenience for when we don't care about the result. like .map(_ => ())
This waits synchronously on run, using the global execution context Avoid calling this if possible, prefering run or just Execution composition.
This waits synchronously on run, using the global execution context Avoid calling this if possible, prefering run or just Execution composition. Every time someone calls this, be very suspect. It is always code smell. Very seldom should you need to wait on a future.
This is here to silence warnings in for comprehensions, but is identical to .filter.
This is here to silence warnings in for comprehensions, but is identical to .filter.
Users should never directly call this method, call .filter
Execution[T] represents and computation that can be run and will produce a value T and keep track of counters incremented inside of TypedPipes using a Stat.
Execution[T] is the recommended way to compose multistep computations that involve branching (if/then), intermediate calls to remote services, file operations, or looping (e.g. testing for convergence).
Library functions are encouraged to implement functions from TypedPipes or ValuePipes to Execution[R] for some result R. Refrain from calling run in library code. Let the caller of your library call run.
Note this is a Monad, meaning flatMap composes in series as you expect. It is also an applicative functor, which means zip (called join in some libraries) composes two Executions is parallel. Prefer zip to flatMap if you want to run two Executions in parallel.