public final class SentenceTokenizer
extends org.apache.lucene.analysis.Tokenizer
The output tokens can then be broken into words with WordTokenFilter
WARNING: The status of the analyzers/smartcn analysis.cn.smart package is experimental. The APIs and file formats introduced here might change in the future and will not be supported anymore in such a case.
Constructor and Description |
---|
SentenceTokenizer(org.apache.lucene.util.AttributeSource.AttributeFactory factory,
java.io.Reader reader) |
SentenceTokenizer(org.apache.lucene.util.AttributeSource source,
java.io.Reader reader) |
SentenceTokenizer(java.io.Reader reader) |
Modifier and Type | Method and Description |
---|---|
void |
end() |
boolean |
incrementToken() |
void |
reset() |
void |
reset(java.io.Reader input) |
getOnlyUseNewAPI, next, next, setOnlyUseNewAPI
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString
public SentenceTokenizer(java.io.Reader reader)
public SentenceTokenizer(org.apache.lucene.util.AttributeSource source, java.io.Reader reader)
public SentenceTokenizer(org.apache.lucene.util.AttributeSource.AttributeFactory factory, java.io.Reader reader)
public boolean incrementToken() throws java.io.IOException
incrementToken
in class org.apache.lucene.analysis.TokenStream
java.io.IOException
public void reset() throws java.io.IOException
reset
in class org.apache.lucene.analysis.TokenStream
java.io.IOException
public void reset(java.io.Reader input) throws java.io.IOException
reset
in class org.apache.lucene.analysis.Tokenizer
java.io.IOException
public void end() throws java.io.IOException
end
in class org.apache.lucene.analysis.TokenStream
java.io.IOException
Copyright © 2000-2018 Apache Software Foundation. All Rights Reserved.