com.twitter.common.text.token
Class TokenStreamDuplicator

java.lang.Object
  extended by org.apache.lucene.util.AttributeSource
      extended by com.twitter.common.text.token.TokenStream
          extended by com.twitter.common.text.token.TokenProcessor
              extended by com.twitter.common.text.token.TokenStreamDuplicator

public class TokenStreamDuplicator
extends TokenProcessor

Creates a duplicate a TokenStream. A TokenStream returned by duplicate method provides the same output as the original TokenStream, but without analyzing the input CharSequence again.


Nested Class Summary
protected static class TokenStreamDuplicator.DuplicatedTokenStream
           
 
Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource
org.apache.lucene.util.AttributeSource.AttributeFactory, org.apache.lucene.util.AttributeSource.State
 
Constructor Summary
TokenStreamDuplicator(TokenStream inputStream)
          Constructs a new TokenStreamDuplicator.
 
Method Summary
 TokenStream duplicate()
          Returns a new TokenStream which provides the same output as the original TokenStream.
 boolean incrementToken()
          Consumers call this method to advance the stream to the next token.
 void reset(CharSequence input)
          Resets this TokenStream (and also downstream tokens if they exist) to parse a new input.
 
Methods inherited from class com.twitter.common.text.token.TokenProcessor
getInputStream, getInstanceOf
 
Methods inherited from class com.twitter.common.text.token.TokenStream
toStringList
 
Methods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString
 
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
 

Constructor Detail

TokenStreamDuplicator

public TokenStreamDuplicator(TokenStream inputStream)
Constructs a new TokenStreamDuplicator.

Parameters:
inputStream - TokenStream to duplicate.
Method Detail

reset

public void reset(CharSequence input)
Description copied from class: TokenStream
Resets this TokenStream (and also downstream tokens if they exist) to parse a new input.

Overrides:
reset in class TokenProcessor
Parameters:
input - new text to parse.

incrementToken

public boolean incrementToken()
Description copied from class: TokenStream
Consumers call this method to advance the stream to the next token.

Specified by:
incrementToken in class TokenStream
Returns:
false for end of stream; true otherwise

duplicate

public TokenStream duplicate()
Returns a new TokenStream which provides the same output as the original TokenStream.

Returns:
a duplicated TokenStream