com.twitter.common.text.detector
Class RegexDetector

java.lang.Object
  extended by org.apache.lucene.util.AttributeSource
      extended by com.twitter.common.text.token.TokenStream
          extended by com.twitter.common.text.token.TokenProcessor
              extended by com.twitter.common.text.detector.RegexDetector
Direct Known Subclasses:
PunctuationDetector

public class RegexDetector
extends TokenProcessor

Updates TypeAttribute of a token if the term matches a given regular expression.


Nested Class Summary
static class RegexDetector.AbstractBuilder<N extends RegexDetector,T extends RegexDetector.AbstractBuilder<N,T>>
           
static class RegexDetector.Builder
           
 
Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource
org.apache.lucene.util.AttributeSource.AttributeFactory, org.apache.lucene.util.AttributeSource.State
 
Constructor Summary
protected RegexDetector(TokenStream inputStream)
           
 
Method Summary
 boolean incrementToken()
          Consumers call this method to advance the stream to the next token.
protected  void setRegexPattern(Pattern regex)
           
protected  void setType(TokenType type)
           
 
Methods inherited from class com.twitter.common.text.token.TokenProcessor
getInputStream, getInstanceOf, reset
 
Methods inherited from class com.twitter.common.text.token.TokenStream
toStringList
 
Methods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString
 
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
 

Constructor Detail

RegexDetector

protected RegexDetector(TokenStream inputStream)
Method Detail

setRegexPattern

protected void setRegexPattern(Pattern regex)

setType

protected void setType(TokenType type)

incrementToken

public boolean incrementToken()
Description copied from class: TokenStream
Consumers call this method to advance the stream to the next token.

Specified by:
incrementToken in class TokenStream
Returns:
false for end of stream; true otherwise