com.twitter.common.text.combiner
Class LookAheadTokenCombiner

java.lang.Object
  extended by org.apache.lucene.util.AttributeSource
      extended by com.twitter.common.text.token.TokenStream
          extended by com.twitter.common.text.token.TokenProcessor
              extended by com.twitter.common.text.combiner.LookAheadTokenCombiner
Direct Known Subclasses:
DotContractedTokenCombiner

public abstract class LookAheadTokenCombiner
extends TokenProcessor

Combiner that looks ahead to the next token and combines it with the current token based on specified conditions.


Nested Class Summary
 
Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource
org.apache.lucene.util.AttributeSource.AttributeFactory, org.apache.lucene.util.AttributeSource.State
 
Constructor Summary
LookAheadTokenCombiner(TokenStream inputStream)
           
 
Method Summary
abstract  boolean canBeCombinedWithNextToken(CharSequence term)
           
abstract  boolean canBeCombinedWithPreviousToken(CharSequence term)
           
 boolean incrementToken()
          Consumers call this method to advance the stream to the next token.
protected  void setType(TokenType type)
           
 
Methods inherited from class com.twitter.common.text.token.TokenProcessor
getInputStream, getInstanceOf, reset
 
Methods inherited from class com.twitter.common.text.token.TokenStream
toStringList
 
Methods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString
 
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
 

Constructor Detail

LookAheadTokenCombiner

public LookAheadTokenCombiner(TokenStream inputStream)
Method Detail

incrementToken

public boolean incrementToken()
Description copied from class: TokenStream
Consumers call this method to advance the stream to the next token.

Specified by:
incrementToken in class TokenStream
Returns:
false for end of stream; true otherwise

setType

protected void setType(TokenType type)

canBeCombinedWithNextToken

public abstract boolean canBeCombinedWithNextToken(CharSequence term)

canBeCombinedWithPreviousToken

public abstract boolean canBeCombinedWithPreviousToken(CharSequence term)