Main Page   Namespace List   Class Hierarchy   Alphabetical List   Compound List   File List   Namespace Members   Compound Members   File Members   Related Pages  

CharScanner Class Reference

#include <CharScanner.hpp>

Inheritance diagram for CharScanner:

TokenStream List of all members.

Public Methods

 CharScanner (InputBuffer &cb, bool case_sensitive)
 CharScanner (InputBuffer *cb, bool case_sensitive)
 CharScanner (const LexerSharedInputState &state, bool case_sensitive)
virtual ~CharScanner ()
virtual int LA (unsigned int i)
virtual void append (char c)
virtual void append (const ANTLR_USE_NAMESPACE(std) string &s)
virtual void commit ()
virtual void consume ()
virtual void consumeUntil (int c)
virtual void consumeUntil (const BitSet &set)
virtual unsigned int mark ()
 Mark the current position and return a id for it.

virtual void rewind (unsigned int pos)
 Rewind the scanner to a previously marked position.

virtual void match (int c)
 See if input contains character 'c' throw MismatchedCharException if not.

virtual void match (const BitSet &b)
virtual void match (const char *s)
virtual void match (const ANTLR_USE_NAMESPACE(std) string &s)
virtual void matchNot (int c)
virtual void matchRange (int c1, int c2)
virtual bool getCaseSensitive () const
virtual void setCaseSensitive (bool t)
virtual bool getCaseSensitiveLiterals () const=0
virtual int getLine () const
 Get the line the scanner currently is in (starts at 1).

virtual void setLine (int l)
 set the line number

virtual int getColumn () const
 Get the column the scanner currently is in (starts at 1).

virtual void setColumn (int c)
 set the column number

virtual const ANTLR_USE_NAMESPACE (std) string &getFilename() const
 get the filename for the file currently used

virtual void setFilename (const ANTLR_USE_NAMESPACE(std) string &f)
 Set the filename the scanner is using (used in error messages).

virtual bool getCommitToPath () const
virtual void setCommitToPath (bool commit)
virtual const ANTLR_USE_NAMESPACE (std) string &getText() const
virtual void setText (const ANTLR_USE_NAMESPACE(std) string &s)
virtual void resetText ()
virtual RefToken getTokenObject () const
virtual void newline ()
virtual void tab ()
int setTabsize (int size)
 set the tabsize. Returns the old tabsize

int getTabSize () const
 Return the tabsize used by the scanner.

virtual void panic ()
virtual void panic (const ANTLR_USE_NAMESPACE(std) string &s)
virtual void reportError (const RecognitionException &e)
virtual void reportError (const ANTLR_USE_NAMESPACE(std) string &s)
virtual void reportWarning (const ANTLR_USE_NAMESPACE(std) string &s)
virtual InputBuffer & getInputBuffer ()
virtual LexerSharedInputState getInputState ()
virtual void setInputState (LexerSharedInputState state)
virtual void setTokenObjectFactory (factory_type factory)
 Set the factory for created tokens.

virtual int testLiteralsTable (int ttype) const
virtual int testLiteralsTable (const ANTLR_USE_NAMESPACE(std) string &txt, int ttype) const
virtual int toLower (int c) const
 Override this method to get more specific case handling.

virtual void uponEOF ()
virtual void traceIndent ()
 Methods used to change tracing behavior.

virtual void traceIn (const char *rname)
virtual void traceOut (const char *rname)

Static Public Attributes

const int EOF_CHAR = EOF

Protected Types

typedef RefToken(* factory_type )()

Protected Methods

 ANTLR_USE_NAMESPACE (std) string text
 ANTLR_USE_NAMESPACE (std) map< ANTLR_USE_NAMESPACE(std) string
virtual RefToken makeToken (int t)
 Create a new RefToken of type t.


Protected Attributes

bool saveConsumedInput
factory_type tokenFactory
 Factory for tokens.

bool caseSensitive
 Is this lexer case sensitive.

 int
CharScannerLiteralsLess literals
RefToken _returnToken
 used to return tokens w/o using return val

LexerSharedInputState inputState
 Input state, gives access to input stream, shared among different lexers.

bool commitToPath
int tabsize
 tab size the scanner uses.

int traceDepth

Detailed Description

Superclass of generated lexers

Member Typedef Documentation

typedef RefToken(* CharScanner::factory_type)() [protected]
 

Constructor & Destructor Documentation

CharScanner::CharScanner (  InputBuffer &    cb,
bool    case_sensitive
) 
 

CharScanner::CharScanner (  InputBuffer *    cb,
bool    case_sensitive
) 
 

CharScanner::CharScanner (  const LexerSharedInputState &    state,
bool    case_sensitive
) 
 

virtual CharScanner::~CharScanner (    )  [inline, virtual]
 

Member Function Documentation

CharScanner::ANTLR_USE_NAMESPACE (  std    )  [protected]
 

CharScanner::ANTLR_USE_NAMESPACE (  std    )  [protected]
 

Text of current token flag indicating wether consume saves characters

virtual const CharScanner::ANTLR_USE_NAMESPACE (  std    )  const [inline, virtual]
 

return a copy of the current text buffer

virtual const CharScanner::ANTLR_USE_NAMESPACE (  std    )  const [inline, virtual]
 

get the filename for the file currently used

virtual void CharScanner::append (  const ANTLR_USE_NAMESPACE(std) string &    s )  [inline, virtual]
 

virtual void CharScanner::append (  char    c )  [inline, virtual]
 

virtual void CharScanner::commit (    )  [inline, virtual]
 

void CharScanner::consume (    )  [virtual]
 

virtual void CharScanner::consumeUntil (  const BitSet &    set )  [inline, virtual]
 

Consume chars until one matches the given set

virtual void CharScanner::consumeUntil (  int    c )  [inline, virtual]
 

Consume chars until one matches the given char

virtual bool CharScanner::getCaseSensitive (    )  const [inline, virtual]
 

virtual bool CharScanner::getCaseSensitiveLiterals (    )  [pure virtual]
 

virtual int CharScanner::getColumn (    )  const [inline, virtual]
 

Get the column the scanner currently is in (starts at 1).

virtual bool CharScanner::getCommitToPath (    )  const [inline, virtual]
 

virtual InputBuffer& CharScanner::getInputBuffer (    )  [inline, virtual]
 

virtual LexerSharedInputState CharScanner::getInputState (    )  [inline, virtual]
 

virtual int CharScanner::getLine (    )  const [inline, virtual]
 

Get the line the scanner currently is in (starts at 1).

int CharScanner::getTabSize (    )  const [inline]
 

Return the tabsize used by the scanner.

virtual RefToken CharScanner::getTokenObject (    )  const [inline, virtual]
 

int CharScanner::LA (  unsigned int    i )  [inline, virtual]
 

virtual RefToken CharScanner::makeToken (  int    t )  [inline, protected, virtual]
 

Create a new RefToken of type t.

virtual unsigned int CharScanner::mark (    )  [inline, virtual]
 

Mark the current position and return a id for it.

virtual void CharScanner::match (  const ANTLR_USE_NAMESPACE(std) string &    s )  [inline, virtual]
 

See if input contains string 's' throw MismatchedCharException if not

Note:
the string cannot match EOF

virtual void CharScanner::match (  const char *    s )  [inline, virtual]
 

See if input contains string 's' throw MismatchedCharException if not

Note:
the string cannot match EOF

virtual void CharScanner::match (  const BitSet &    b )  [inline, virtual]
 

See if input contains element from bitset b throw MismatchedCharException if not

virtual void CharScanner::match (  int    c )  [inline, virtual]
 

See if input contains character 'c' throw MismatchedCharException if not.

virtual void CharScanner::matchNot (  int    c )  [inline, virtual]
 

See if input does not contain character 'c' throw MismatchedCharException if not

virtual void CharScanner::matchRange (  int    c1,
int    c2
)  [inline, virtual]
 

See if input contains character in range c1-c2 throw MismatchedCharException if not

virtual void CharScanner::newline (    )  [inline, virtual]
 

Used to keep track of line breaks, needs to be called from within generated lexers when a
\r is encountered.

void CharScanner::panic (  const ANTLR_USE_NAMESPACE(std) string &    s )  [virtual]
 

Terminate program using exit()

Deprecated:
will be removed in the next release. It is not used.

void CharScanner::panic (    )  [virtual]
 

Terminate program using exit()

Deprecated:
will be removed in the next release. It is not used.

void CharScanner::reportError (  const ANTLR_USE_NAMESPACE(std) string &    s )  [virtual]
 

Parser error-reporting function can be overridden in subclass

void CharScanner::reportError (  const RecognitionException &    e )  [virtual]
 

Report exception errors caught in nextToken()

void CharScanner::reportWarning (  const ANTLR_USE_NAMESPACE(std) string &    s )  [virtual]
 

Parser warning-reporting function can be overridden in subclass

virtual void CharScanner::resetText (    )  [inline, virtual]
 

virtual void CharScanner::rewind (  unsigned int    pos )  [inline, virtual]
 

Rewind the scanner to a previously marked position.

virtual void CharScanner::setCaseSensitive (  bool    t )  [inline, virtual]
 

virtual void CharScanner::setColumn (  int    c )  [inline, virtual]
 

set the column number

virtual void CharScanner::setCommitToPath (  bool    commit )  [inline, virtual]
 

virtual void CharScanner::setFilename (  const ANTLR_USE_NAMESPACE(std) string &    f )  [inline, virtual]
 

Set the filename the scanner is using (used in error messages).

virtual void CharScanner::setInputState (  LexerSharedInputState    state )  [inline, virtual]
 

set the input state for the lexer.

Note:
state is a reference counted object, hence no reference

virtual void CharScanner::setLine (  int    l )  [inline, virtual]
 

set the line number

int CharScanner::setTabsize (  int    size )  [inline]
 

set the tabsize. Returns the old tabsize

virtual void CharScanner::setText (  const ANTLR_USE_NAMESPACE(std) string &    s )  [inline, virtual]
 

virtual void CharScanner::setTokenObjectFactory (  factory_type    factory )  [inline, virtual]
 

Set the factory for created tokens.

virtual void CharScanner::tab (    )  [inline, virtual]
 

Advance the current column number by an appropriate amount according to the tabsize. This method needs to be explicitly called from the lexer rules encountering tabs.

virtual int CharScanner::testLiteralsTable (  const ANTLR_USE_NAMESPACE(std) string &    txt,
int    ttype
)  const [inline, virtual]
 

Test the text passed in against the literals table Override this method to perform a different literals test This is used primarily when you want to test a portion of a token

virtual int CharScanner::testLiteralsTable (  int    ttype )  const [inline, virtual]
 

Test the token text against the literals table Override this method to perform a different literals test

virtual int CharScanner::toLower (  int    c )  const [inline, virtual]
 

Override this method to get more specific case handling.

void CharScanner::traceIn (  const char *    rname )  [virtual]
 

void CharScanner::traceIndent (    )  [virtual]
 

Methods used to change tracing behavior.

void CharScanner::traceOut (  const char *    rname )  [virtual]
 

virtual void CharScanner::uponEOF (    )  [inline, virtual]
 

This method is called by YourLexer::nextToken() when the lexer has hit EOF condition. EOF is NOT a character. This method is not called if EOF is reached during syntactic predicate evaluation or during evaluation of normal lexical rules, which presumably would be an IOException. This traps the "normal" EOF condition.

uponEOF() is called after the complete evaluation of the previous token and only if your parser asks for another token beyond that last non-EOF token.

You might want to throw token or char stream exceptions like: "Heh, premature eof" or a retry stream exception ("I found the end of this file, go back to referencing file").

Member Data Documentation

RefToken CharScanner::_returnToken [protected]
 

used to return tokens w/o using return val

bool CharScanner::caseSensitive [protected]
 

Is this lexer case sensitive.

bool CharScanner::commitToPath [protected]
 

Used during filter mode to indicate that path is desired. A subsequent scan error will report an error as usual if acceptPath=true;

const int CharScanner::EOF_CHAR = EOF [static]
 

LexerSharedInputState CharScanner::inputState [protected]
 

Input state, gives access to input stream, shared among different lexers.

CharScanner::int [protected]
 

CharScannerLiteralsLess CharScanner::literals [protected]
 

bool CharScanner::saveConsumedInput [protected]
 

int CharScanner::tabsize [protected]
 

tab size the scanner uses.

factory_type CharScanner::tokenFactory [protected]
 

Factory for tokens.

int CharScanner::traceDepth [protected]
 

The documentation for this class was generated from the following files:
Generated on Wed Nov 3 12:59:25 2004 for Lemur Toolkit by doxygen1.2.18