public final class Tokenizer
extends java.lang.Object
Modifier and Type | Field and Description |
---|---|
static int |
BARE_NAME_STATE
State in which a name is NOT to be merged with what comes next, for example "("
|
int |
currentToken
The number identifying the most recently read token
|
int |
currentTokenStartOffset
The position in the input expression where the current token starts
|
java.lang.String |
currentTokenValue
The string value of the most recently read token
|
static int |
DEFAULT_STATE
Initial default state of the Tokenizer
|
java.lang.String |
input
The string being parsed
|
int |
inputOffset
The current position within the input string
|
static int |
OPERATOR_STATE
State in which the next thing to be read is an operator
|
static int |
SEQUENCE_TYPE_STATE
State in which the next thing to be read is a SequenceType
|
int |
startLineNumber
The starting line number (for XPath in XSLT, the line number in the stylesheet)
|
Constructor and Description |
---|
Tokenizer() |
Modifier and Type | Method and Description |
---|---|
int |
getColumnNumber()
Get the column number of the current token
|
int |
getColumnNumber(int offset)
Return the column number corresponding to a given offset in the expression
|
long |
getLineAndColumn(int offset)
Get the line and column number corresponding to a given offset in the input expression,
as a long value with the line number in the top half
and the column number in the lower half
|
int |
getLineNumber()
Get the line number of the current token
|
int |
getLineNumber(int offset)
Return the line number corresponding to a given offset in the expression
|
int |
getState()
Get the current tokenizer state
|
void |
lookAhead()
Look ahead by one token.
|
void |
next()
Get the next token from the input expression.
|
char |
nextChar()
Read next character directly.
|
java.lang.String |
recentText()
Get the most recently read text (for use in an error message)
|
void |
setState(int state)
Set the tokenizer into a special state
|
void |
tokenize(java.lang.String input,
int start,
int end,
int lineNumber)
Prepare a string for tokenization.
|
void |
treatCurrentAsOperator()
Force the current token to be treated as an operator if possible
|
void |
unreadChar()
Step back one character.
|
public static final int DEFAULT_STATE
public static final int BARE_NAME_STATE
public static final int SEQUENCE_TYPE_STATE
public static final int OPERATOR_STATE
public int startLineNumber
public int currentToken
public java.lang.String currentTokenValue
public int currentTokenStartOffset
public java.lang.String input
public int inputOffset
public int getState()
public void setState(int state)
state
- the new statepublic void tokenize(java.lang.String input, int start, int end, int lineNumber) throws XPathException
input
- the string to be tokenizedstart
- start point within the stringend
- end point within the string (last character not read):
-1 means end of stringlineNumber
- the linenumber in the source where the expression appearsXPathException
- if a lexical error occurs, e.g. unmatched
string quotespublic void next() throws XPathException
XPathException
- if a lexical error is detectedpublic void treatCurrentAsOperator()
public void lookAhead() throws XPathException
XPathException
- if a lexical error occurspublic char nextChar() throws java.lang.StringIndexOutOfBoundsException
java.lang.StringIndexOutOfBoundsException
- if an attempt is made to read beyond
the end of the string. This will only occur in the event of a syntax error in the
input.public void unreadChar()
public java.lang.String recentText()
public int getLineNumber()
public int getColumnNumber()
public long getLineAndColumn(int offset)
offset
- the byte offset in the expressionpublic int getLineNumber(int offset)
offset
- the byte offset in the expressionpublic int getColumnNumber(int offset)
offset
- the byte offset in the expression