com.twitter.common.text
Class DefaultTextTokenizer

java.lang.Object
  extended by com.twitter.common.text.TextTokenizer
      extended by com.twitter.common.text.DefaultTextTokenizer

public class DefaultTextTokenizer
extends TextTokenizer

Default implementation of a tokenizer for processing tweets. For sample usage, please consult annotated code example in TokenizerUsageExample.


Nested Class Summary
static class DefaultTextTokenizer.Builder
          Builder for DefaultTextTokenizer
 
Field Summary
 
Fields inherited from class com.twitter.common.text.TextTokenizer
tokenizationStream
 
Method Summary
 TokenStream applyDefaultChain(TokenStream tokenizer)
           
 
Methods inherited from class com.twitter.common.text.TextTokenizer
getDefaultTokenStream, tokenize, tokenizeToStrings
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Method Detail

applyDefaultChain

public TokenStream applyDefaultChain(TokenStream tokenizer)
Specified by:
applyDefaultChain in class TextTokenizer