Package

com.twitter.scalding

mathematics

Permalink

package mathematics

Content Hierarchy
Visibility
  1. Public
  2. All

Type Members

  1. class BigToSmall extends MatrixJoiner

    Permalink
  2. class BlockMatrix[RowT, GroupT, ColT, ValT] extends AnyRef

    Permalink

    BlockMatrix is 3 dimensional matrix where the rows are grouped It is useful for when we want to multiply groups of vectors only between themselves.

    BlockMatrix is 3 dimensional matrix where the rows are grouped It is useful for when we want to multiply groups of vectors only between themselves. For example, grouping users by countries and calculating products only between users from the same country

  3. class ColVector[RowT, ValT] extends Serializable with WrappedPipe

    Permalink
  4. class DefaultMatrixJoiner extends MatrixJoiner2

    Permalink

    This uses standard join if the matrices are comparable size and large, otherwise, if one is much smaller than the other, we use a hash join

  5. sealed trait Degree extends AnyRef

    Permalink
  6. class DiagonalMatrix[IdxT, ValT] extends WrappedPipe with Serializable

    Permalink
  7. class DimsumInCosine[N] extends TypedSimilarity[N, (Weight, L2Norm), Double]

    Permalink
  8. class DiscoInCosine[N] extends TypedSimilarity[N, InDegree, Double]

    Permalink

    Params: minCos: the minimum cosine similarity you care about accuracy for delta: the error on the approximated cosine (e.g.

    Params: minCos: the minimum cosine similarity you care about accuracy for delta: the error on the approximated cosine (e.g. 0.05 = 5%) boundedProb: the probability we have larger than delta error see: http://arxiv.org/pdf/1206.2082v2.pdf for more details

  9. case class Edge[+N, +E](from: N, to: N, data: E) extends scala.Product with Serializable

    Permalink

    Represents an Edge in a graph with some edge data

  10. class ExactInCosine[N] extends TypedSimilarity[N, InDegree, Double]

    Permalink

    This algothm is just matrix multiplication done by hand to make it clearer when we do the sampling implementation

  11. case class FiniteHint(rows: BigInt = 1L, cols: BigInt = 1L) extends SizeHint with scala.Product with Serializable

    Permalink
  12. case class HadamardProduct[R, C, V](left: Matrix2[R, C, V], right: Matrix2[R, C, V], ring: Ring[V]) extends Matrix2[R, C, V] with scala.Product with Serializable

    Permalink
  13. class Histogram extends AnyRef

    Permalink
  14. case class InDegree(degree: Int) extends Degree with scala.Product with Serializable

    Permalink
  15. case class L2Norm(norm: Double) extends scala.Product with Serializable

    Permalink
  16. class LiteralScalar[ValT] extends Serializable

    Permalink
  17. class Matrix[RowT, ColT, ValT] extends WrappedPipe with Serializable

    Permalink
  18. sealed trait Matrix2[R, C, V] extends Serializable

    Permalink

    This is the future Matrix API.

    This is the future Matrix API. The old one will be removed in scalding 0.10.0 (or 1.0.0).

    Create Matrix2 instances with methods in the Matrix2 object. Note that this code optimizes the order in which it evaluates matrices, and replaces equivalent terms to avoid recomputation. Also, this code puts the parenthesis in the optimal place in terms of size according to the sizeHints. For instance: (A*B)*C == A*(B*C) but if B is a 10 x 106 matrix, and C is 106 x 100, it is better to do the B*C product first in order to avoid storing as much intermediate output.

    NOTE THIS REQUIREMENT: for each formula, you can only have one Ring[V] in scope. If you evaluate part of the formula with one Ring, and another part with another, you must go through a TypedPipe (call toTypedPipe) or the result may not be correct.

  19. abstract class MatrixCrosser extends Serializable

    Permalink
  20. abstract class MatrixJoiner extends Serializable

    Permalink

    Abstracts the approach taken to join the two matrices

  21. trait MatrixJoiner2 extends Serializable

    Permalink

    This trait allows users to plug in join algorithms where they are needed to improve products and propagations.

    This trait allows users to plug in join algorithms where they are needed to improve products and propagations. The default works well in most cases, but highly skewed matrices may need some special handling

  22. case class MatrixLiteral[R, C, V](toTypedPipe: TypedPipe[(R, C, V)], sizeHint: SizeHint)(implicit rowOrd: Ordering[R], colOrd: Ordering[C]) extends Matrix2[R, C, V] with scala.Product with Serializable

    Permalink
  23. class MatrixMappableExtensions[T] extends AnyRef

    Permalink

    This is the enrichment pattern on Mappable[T] for converting to Matrix types

  24. class MatrixPipeExtensions extends AnyRef

    Permalink

    Matrix class - represents an infinite (hopefully sparse) matrix.

    Matrix class - represents an infinite (hopefully sparse) matrix. any elements without a row are interpretted to be zero. the pipe hold ('rowIdx, 'colIdx, 'val) where in principle each row/col/value type is generic, with the constraint that ValT is a Ring[T] In practice, RowT and ColT are going to be Strings, Integers or Longs in the usual case.

    WARNING: It is NOT OKAY to use the same instance of Matrix/Row/Col with DIFFERENT Monoids/Rings/Fields. If you want to change, midstream, the Monoid on your ValT, you have to construct a new Matrix. This is due to caching of internal computation graphs.

    RowVector - handles matrices of row dimension one. It is the result of some of the matrix methods and has methods that return ColVector and diagonal matrix

    ColVector - handles matrices of col dimension one. It is the result of some of the matrix methods and has methods that return RowVector and diagonal matrix

  25. trait MatrixProduct[Left, Right, Result] extends Serializable

    Permalink
  26. case class OneC[R, V]()(implicit rowOrd: Ordering[R]) extends Matrix2[R, Unit, V] with scala.Product with Serializable

    Permalink

    Infinite column vector - only for intermediate computations

  27. case class OneR[C, V]()(implicit colOrd: Ordering[C]) extends Matrix2[Unit, C, V] with scala.Product with Serializable

    Permalink

    Infinite row vector - only for intermediate computations

  28. case class OutDegree(degree: Int) extends Degree with scala.Product with Serializable

    Permalink
  29. class Poisson extends AnyRef

    Permalink

    Generating Poisson-distributed random variables according to Donald Knuth's algorithm as shown on Wikipedia's Poisson Distribution page

  30. case class Product[R, C, C2, V](left: Matrix2[R, C, V], right: Matrix2[C, C2, V], ring: Ring[V], expressions: Option[Map[Matrix2[R, C2, V], TypedPipe[(R, C2, V)]]] = None)(implicit joiner: MatrixJoiner2) extends Matrix2[R, C2, V] with scala.Product with Serializable

    Permalink

    Class representing a matrix product

    Class representing a matrix product

    left

    multiplicand

    right

    multiplier

    expressions

    a HashMap of common subtrees; None if possibly not optimal (did not go through optimize), Some(...) with a HashMap that was created in optimize

  31. class RowVector[ColT, ValT] extends Serializable with WrappedPipe

    Permalink
  32. class Scalar[ValT] extends WrappedPipe with Serializable

    Permalink
  33. trait Scalar2[V] extends Serializable

    Permalink

    A representation of a scalar value that can be used with Matrices

  34. case class SetSimilarity(intersection: Int, sizeLeft: Int, sizeRight: Int) extends scala.Product with Serializable

    Permalink
  35. sealed abstract class SizeHint extends AnyRef

    Permalink
  36. class SmallToBig extends MatrixJoiner

    Permalink
  37. case class SparseHint(sparsity: Double, rows: BigInt, cols: BigInt) extends SizeHint with scala.Product with Serializable

    Permalink
  38. case class Sum[R, C, V](left: Matrix2[R, C, V], right: Matrix2[R, C, V], mon: Monoid[V]) extends Matrix2[R, C, V] with scala.Product with Serializable

    Permalink
  39. trait TypedSimilarity[N, E, S] extends Serializable

    Permalink
  40. case class ValuePipeScalar[V](value: typed.ValuePipe[V]) extends Scalar2[V] with scala.Product with Serializable

    Permalink
  41. case class Weight(weight: Double) extends scala.Product with Serializable

    Permalink
  42. trait WrappedPipe extends AnyRef

    Permalink

Value Members

  1. object AnyCrossSmall extends MatrixCrosser with scala.Product with Serializable

    Permalink
  2. object AnyCrossTiny extends MatrixCrosser with scala.Product with Serializable

    Permalink
  3. object AnyToTiny extends MatrixJoiner with scala.Product with Serializable

    Permalink
  4. object Combinatorics

    Permalink

    Serve as a repo for self-contained combinatorial functions with no dependencies such as combinations, aka n choose k, nCk permutations , aka nPk subset sum : numbers that add up to a finite sum weightedSum: For weights (a,b,c, ...), want integers (x,y,z,...) to satisfy constraint |ax + by + cz + ...

    Serve as a repo for self-contained combinatorial functions with no dependencies such as combinations, aka n choose k, nCk permutations , aka nPk subset sum : numbers that add up to a finite sum weightedSum: For weights (a,b,c, ...), want integers (x,y,z,...) to satisfy constraint |ax + by + cz + ... - result | < error ...

  5. object GraphOperations extends Serializable

    Permalink
  6. object Matrix extends Serializable

    Permalink
  7. object Matrix2 extends Serializable

    Permalink
  8. object MatrixJoiner2 extends Serializable

    Permalink
  9. object MatrixProduct extends Serializable

    Permalink

    TODO: Muliplication is the expensive stuff.

    TODO: Muliplication is the expensive stuff. We need to optimize the methods below: This object holds the implicits to handle matrix products between various types

  10. object NoClue extends SizeHint with scala.Product with Serializable

    Permalink
  11. object Scalar2 extends Serializable

    Permalink
  12. object SizeHint

    Permalink
  13. object SizeHintOrdering extends Ordering[SizeHint] with Serializable

    Permalink

    Allows us to sort matrices by approximate type

  14. object TinyToAny extends MatrixJoiner with scala.Product with Serializable

    Permalink
  15. object TypedSimilarity extends Serializable

    Permalink

Ungrouped