public class StandardAnalyzer extends Analyzer
StandardTokenizer
with StandardFilter
, LowerCaseFilter
and StopFilter
, using a list of
English stop words.
You must specify the required Version
compatibility when creating StandardAnalyzer:
Modifier and Type | Field and Description |
---|---|
static int |
DEFAULT_MAX_TOKEN_LENGTH
Default maximum allowed token length
|
static java.lang.String[] |
STOP_WORDS
Deprecated.
Use
STOP_WORDS_SET instead |
static java.util.Set |
STOP_WORDS_SET
An unmodifiable set containing some common English words that are usually not
useful for searching.
|
overridesTokenStreamMethod
Constructor and Description |
---|
StandardAnalyzer()
Deprecated.
Use
StandardAnalyzer(Version) instead. |
StandardAnalyzer(boolean replaceInvalidAcronym)
Deprecated.
Remove in 3.X and make true the only valid value
|
StandardAnalyzer(java.io.File stopwords)
Deprecated.
Use
StandardAnalyzer(Version, File)
instead |
StandardAnalyzer(java.io.File stopwords,
boolean replaceInvalidAcronym)
Deprecated.
Remove in 3.X and make true the only valid value
|
StandardAnalyzer(java.io.Reader stopwords)
Deprecated.
Use
StandardAnalyzer(Version, Reader)
instead |
StandardAnalyzer(java.io.Reader stopwords,
boolean replaceInvalidAcronym)
Deprecated.
Remove in 3.X and make true the only valid value
|
StandardAnalyzer(java.util.Set stopWords)
Deprecated.
Use
StandardAnalyzer(Version, Set)
instead |
StandardAnalyzer(java.util.Set stopwords,
boolean replaceInvalidAcronym)
Deprecated.
Remove in 3.X and make true the only valid value
|
StandardAnalyzer(java.lang.String[] stopWords)
Deprecated.
Use
StandardAnalyzer(Version, Set) instead |
StandardAnalyzer(java.lang.String[] stopwords,
boolean replaceInvalidAcronym)
Deprecated.
Remove in 3.X and make true the only valid value
|
StandardAnalyzer(Version matchVersion)
Builds an analyzer with the default stop words (
STOP_WORDS ). |
StandardAnalyzer(Version matchVersion,
java.io.File stopwords)
Builds an analyzer with the stop words from the given file.
|
StandardAnalyzer(Version matchVersion,
java.io.Reader stopwords)
Builds an analyzer with the stop words from the given reader.
|
StandardAnalyzer(Version matchVersion,
java.util.Set stopWords)
Builds an analyzer with the given stop words.
|
Modifier and Type | Method and Description |
---|---|
static boolean |
getDefaultReplaceInvalidAcronym()
Deprecated.
This will be removed (hardwired to true) in 3.0
|
int |
getMaxTokenLength() |
boolean |
isReplaceInvalidAcronym()
Deprecated.
This will be removed (hardwired to true) in 3.0
|
TokenStream |
reusableTokenStream(java.lang.String fieldName,
java.io.Reader reader)
Deprecated.
Use
tokenStream(java.lang.String, java.io.Reader) instead |
static void |
setDefaultReplaceInvalidAcronym(boolean replaceInvalidAcronym)
Deprecated.
This will be removed (hardwired to true) in 3.0
|
void |
setMaxTokenLength(int length)
Set maximum allowed token length.
|
void |
setReplaceInvalidAcronym(boolean replaceInvalidAcronym)
Deprecated.
This will be removed (hardwired to true) in 3.0
|
TokenStream |
tokenStream(java.lang.String fieldName,
java.io.Reader reader)
|
close, getOffsetGap, getPositionIncrementGap, getPreviousTokenStream, setOverridesTokenStreamMethod, setPreviousTokenStream
public static final java.lang.String[] STOP_WORDS
STOP_WORDS_SET
insteadpublic static final java.util.Set STOP_WORDS_SET
public static final int DEFAULT_MAX_TOKEN_LENGTH
public StandardAnalyzer()
StandardAnalyzer(Version)
instead.STOP_WORDS_SET
).public StandardAnalyzer(Version matchVersion)
STOP_WORDS
).matchVersion
- Lucene version to match See abovepublic StandardAnalyzer(java.util.Set stopWords)
StandardAnalyzer(Version, Set)
insteadpublic StandardAnalyzer(Version matchVersion, java.util.Set stopWords)
matchVersion
- Lucene version to match See abovestopWords
- stop wordspublic StandardAnalyzer(java.lang.String[] stopWords)
StandardAnalyzer(Version, Set)
insteadpublic StandardAnalyzer(java.io.File stopwords) throws java.io.IOException
StandardAnalyzer(Version, File)
insteadjava.io.IOException
WordlistLoader.getWordSet(File)
public StandardAnalyzer(Version matchVersion, java.io.File stopwords) throws java.io.IOException
matchVersion
- Lucene version to match See abovestopwords
- File to read stop words fromjava.io.IOException
WordlistLoader.getWordSet(File)
public StandardAnalyzer(java.io.Reader stopwords) throws java.io.IOException
StandardAnalyzer(Version, Reader)
insteadjava.io.IOException
WordlistLoader.getWordSet(Reader)
public StandardAnalyzer(Version matchVersion, java.io.Reader stopwords) throws java.io.IOException
matchVersion
- Lucene version to match See abovestopwords
- Reader to read stop words fromjava.io.IOException
WordlistLoader.getWordSet(Reader)
public StandardAnalyzer(boolean replaceInvalidAcronym)
replaceInvalidAcronym
- Set to true if this analyzer should replace mischaracterized acronyms in the StandardTokenizer
See https://issues.apache.org/jira/browse/LUCENE-1068public StandardAnalyzer(java.io.Reader stopwords, boolean replaceInvalidAcronym) throws java.io.IOException
stopwords
- The stopwords to usereplaceInvalidAcronym
- Set to true if this analyzer should replace mischaracterized acronyms in the StandardTokenizer
See https://issues.apache.org/jira/browse/LUCENE-1068java.io.IOException
public StandardAnalyzer(java.io.File stopwords, boolean replaceInvalidAcronym) throws java.io.IOException
stopwords
- The stopwords to usereplaceInvalidAcronym
- Set to true if this analyzer should replace mischaracterized acronyms in the StandardTokenizer
See https://issues.apache.org/jira/browse/LUCENE-1068java.io.IOException
public StandardAnalyzer(java.lang.String[] stopwords, boolean replaceInvalidAcronym) throws java.io.IOException
stopwords
- The stopwords to usereplaceInvalidAcronym
- Set to true if this analyzer should replace mischaracterized acronyms in the StandardTokenizer
See https://issues.apache.org/jira/browse/LUCENE-1068java.io.IOException
public StandardAnalyzer(java.util.Set stopwords, boolean replaceInvalidAcronym) throws java.io.IOException
stopwords
- The stopwords to usereplaceInvalidAcronym
- Set to true if this analyzer should replace mischaracterized acronyms in the StandardTokenizer
See https://issues.apache.org/jira/browse/LUCENE-1068java.io.IOException
public static boolean getDefaultReplaceInvalidAcronym()
public static void setDefaultReplaceInvalidAcronym(boolean replaceInvalidAcronym)
replaceInvalidAcronym
- Set to true to have new
instances of StandardTokenizer replace mischaracterized
acronyms by default. Set to false to preserve the
previous (before 2.4) buggy behavior. Alternatively,
set the system property
org.apache.lucene.analysis.standard.StandardAnalyzer.replaceInvalidAcronym
to false.
See https://issues.apache.org/jira/browse/LUCENE-1068public TokenStream tokenStream(java.lang.String fieldName, java.io.Reader reader)
tokenStream
in class Analyzer
public void setMaxTokenLength(int length)
public int getMaxTokenLength()
setMaxTokenLength(int)
public TokenStream reusableTokenStream(java.lang.String fieldName, java.io.Reader reader) throws java.io.IOException
tokenStream(java.lang.String, java.io.Reader)
insteadAnalyzer
reusableTokenStream
in class Analyzer
java.io.IOException
public boolean isReplaceInvalidAcronym()
public void setReplaceInvalidAcronym(boolean replaceInvalidAcronym)
replaceInvalidAcronym
- Set to true if this Analyzer is replacing mischaracterized acronyms in the StandardTokenizer
See https://issues.apache.org/jira/browse/LUCENE-1068Copyright © 2000-2019 Apache Software Foundation. All Rights Reserved.