genscript.defaultreaders

Class PreScanTokenizer

java.lang.Object

genscript.parse.ReaderStack

genscript.parse.IndexReader

genscript.syntax.tokens.Tokenizer

genscript.defaultreaders.PreScanTokenizer

All Implemented Interfaces:

ReaderInterface

public class PreScanTokenizer
extends Tokenizer

Field Summary

Fields inherited from class genscript.syntax.tokens.Tokenizer

innerSource, tokenStack

Constructor Summary

Constructors

Constructor and Description
PreScanTokenizer(IndexReader innerSource)

Method Summary

Modifier and Type	Method and Description
int	getBottomLevelPosition(int position)
int	getBufferSize()
int	getIntervalEndPos()
int	getIntervalStartPos()
int	getPosition()
java.lang.String	intervalToString(int startShift, int endShift)
boolean	isEndOfBuffer()
IndexReader	newReader() Clones this Reader without copying the buffer itself
void	parseStart(NTProductions startParser)
Token	read() Returns the next index in the source code and moves the marker.
void	setIntervalEndPos(int endPos)
void	setIntervalStartPos(int startPos)
void	setPosition(int position) Sets the Position of the inner Buffer.
void	skipSpace()
void	stepBack() Makes one step back in the buffer.
java.lang.String	toString()
SourcePosition	toTextPosition(int position) Returns the position of the readable input source code.

Methods inherited from class genscript.syntax.tokens.Tokenizer

clear, getBottomLevelSource, getFilename, isTokenStackEmpty, pollTokenValue, pushTokenValue, tokenStackToString

Methods inherited from class genscript.parse.IndexReader

canStepBack, getBottomLevelIntervalStartPos, getBottomLevelPosition, getErrorPosition, hasErrorPosition, intervalToString, setErrorPosition, setIntervalPositions, switchParser

Methods inherited from class genscript.parse.ReaderStack

getMarkedStackSize, getResultStackSize, getStackMark, popResult, pushResult, resetStack, resultStackToString, setStackMark

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

PreScanTokenizer

public PreScanTokenizer(IndexReader innerSource)

Method Detail

read

public Token read()
           throws ParseException

Description copied from interface: ReaderInterface

Returns the next index in the source code and moves the marker.

Specified by:

read in interface ReaderInterface

Specified by:

read in class IndexReader

Returns:

the next index

Throws:

ParseException

stepBack

public void stepBack()

Description copied from class: IndexReader

Makes one step back in the buffer.

Specified by:

stepBack in class IndexReader

newReader

public IndexReader newReader()

Description copied from class: IndexReader

Clones this Reader without copying the buffer itself

Specified by:

newReader in class IndexReader

skipSpace

public void skipSpace()

Specified by:

skipSpace in class IndexReader

parseStart

public void parseStart(NTProductions startParser)
                throws ParseException

Overrides:

parseStart in class IndexReader

Throws:

ParseException

getPosition

public int getPosition()

Specified by:

getPosition in interface ReaderInterface

Specified by:

getPosition in class IndexReader

Returns:

the current position of the marker

setPosition

public void setPosition(int position)

Description copied from class: IndexReader

Sets the Position of the inner Buffer. The next read() call will start from the set position.

Specified by:

setPosition in interface ReaderInterface

Specified by:

setPosition in class IndexReader

Parameters:

position - the position to set the marker

isEndOfBuffer

public boolean isEndOfBuffer()

Specified by:

isEndOfBuffer in interface ReaderInterface

Specified by:

isEndOfBuffer in class IndexReader

Returns:

True, if and only if the end of the inner Buffer is reached

toTextPosition

public SourcePosition toTextPosition(int position)

Description copied from interface: ReaderInterface

Returns the position of the readable input source code.

Specified by:

toTextPosition in interface ReaderInterface

Specified by:

toTextPosition in class IndexReader

Parameters:

position - the position to transform into the text position

Returns:

the text position

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

getBottomLevelPosition

public int getBottomLevelPosition(int position)

Specified by:

getBottomLevelPosition in class IndexReader

getIntervalStartPos

public final int getIntervalStartPos()

Specified by:

getIntervalStartPos in class IndexReader

setIntervalStartPos

public void setIntervalStartPos(int startPos)

Specified by:

setIntervalStartPos in class IndexReader

getIntervalEndPos

public final int getIntervalEndPos()

Specified by:

getIntervalEndPos in class IndexReader

setIntervalEndPos

public void setIntervalEndPos(int endPos)

Specified by:

setIntervalEndPos in class IndexReader

intervalToString

public java.lang.String intervalToString(int startShift,
                                         int endShift)

Specified by:

intervalToString in class IndexReader

getBufferSize

public int getBufferSize()

Specified by:

getBufferSize in interface ReaderInterface

Specified by:

getBufferSize in class IndexReader

Returns:

the number of tokens/bytes of the buffer