public class ElisionFilter extends TokenFilter
TokenStream
. For example, "l'avion" (the plane) will be
tokenized as "avion" (plane).
Note that StandardTokenizer
sees " ' " as a space, and cuts it out.
AttributeSource.AttributeFactory, AttributeSource.State
input
Modifier | Constructor and Description |
---|---|
protected |
ElisionFilter(TokenStream input)
Constructs an elision filter with standard stop words
|
|
ElisionFilter(TokenStream input,
java.util.Set articles)
Constructs an elision filter with a Set of stop words
|
|
ElisionFilter(TokenStream input,
java.lang.String[] articles)
Constructs an elision filter with an array of stop words
|
Modifier and Type | Method and Description |
---|---|
boolean |
incrementToken()
Increments the
TokenStream with a TermAttribute without elisioned start |
Token |
next()
Deprecated.
Will be removed in Lucene 3.0. This method is final, as it should
not be overridden. Delegates to the backwards compatibility layer.
|
Token |
next(Token reusableToken)
Deprecated.
Will be removed in Lucene 3.0. This method is final, as it should
not be overridden. Delegates to the backwards compatibility layer.
|
void |
setArticles(java.util.Set articles) |
close, end, reset
getOnlyUseNewAPI, setOnlyUseNewAPI
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString
protected ElisionFilter(TokenStream input)
public ElisionFilter(TokenStream input, java.util.Set articles)
public ElisionFilter(TokenStream input, java.lang.String[] articles)
public void setArticles(java.util.Set articles)
public final boolean incrementToken() throws java.io.IOException
TokenStream
with a TermAttribute
without elisioned startincrementToken
in class TokenStream
Note that this method will be defined abstract in Lucene 3.0.
java.io.IOException
public final Token next(Token reusableToken) throws java.io.IOException
TokenStream
This implicitly defines a "contract" between consumers (callers of this method) and producers (implementations of this method that are the source for tokens):
Token
before calling this method again.Token.clear()
before setting the fields in
it and returning itToken
after it
has been returned: the caller may arbitrarily change it. If the producer
needs to hold onto the Token
for subsequent calls, it must clone()
it before storing it. Note that a TokenFilter
is considered a
consumer.next
in class TokenStream
reusableToken
- a Token
that may or may not be used to return;
this parameter should never be null (the callee is not required to
check for null before using it, but it is a good idea to assert that
it is not null.)Token
in the stream or null if end-of-stream was hitjava.io.IOException
public final Token next() throws java.io.IOException
TokenStream
Token
in the stream, or null at EOS.next
in class TokenStream
java.io.IOException
Copyright © 2000-2018 Apache Software Foundation. All Rights Reserved.