Package com.twitter.common.text.tokenizer

Class Summary
LatinTokenizer Tokenizes text written in Latin alphabets such as English, French, German.
LatinTokenizer.Builder  
RegexTokenizer Tokenizes text based on regular expressions of word delimiters and punctuation characters.
RegexTokenizer.AbstractBuilder<N extends RegexTokenizer,T extends RegexTokenizer.AbstractBuilder<N,T>>  
RegexTokenizer.Builder Builder for RegexTokenizer.