All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
G3D::TextInput::Settings Class Reference

#include <TextInput.h>

Public Member Functions

 Settings ()

Public Attributes

bool cppBlockComments
bool cppLineComments
bool otherLineComments
bool escapeSequencesInStrings
char otherCommentCharacter
char otherCommentCharacter2
bool generateCommentTokens
bool generateNewlineTokens
bool signedNumbers
bool singleQuotedStrings
char singleQuoteCharacter
std::string sourceFileName
int startingLineNumberOffset
bool msvcFloatSpecials
bool simpleFloatSpecials
bool proofSymbols
bool caseSensitive
Set< std::string > trueSymbols
Set< std::string > falseSymbols

Detailed Description

Tokenizer configuration options.

Constructor & Destructor Documentation

G3D::TextInput::Settings::Settings ( )
75  :
76  cppBlockComments(true),
77  cppLineComments(true),
78  otherLineComments(true),
82  generateCommentTokens(false),
83  generateNewlineTokens(false),
84  signedNumbers(true),
85  singleQuotedStrings(true),
89  msvcFloatSpecials(true),
90  simpleFloatSpecials(true),
91  proofSymbols(false),
92  caseSensitive(true)
93 {
94  trueSymbols.insert("true");
95  falseSymbols.insert("false");
96 }
Set< std::string > trueSymbols
Definition: TextInput.h:398
bool caseSensitive
Definition: TextInput.h:391
bool proofSymbols
Definition: TextInput.h:385
char otherCommentCharacter2
Definition: TextInput.h:284
char otherCommentCharacter
Definition: TextInput.h:279
bool escapeSequencesInStrings
Definition: TextInput.h:269
Set< std::string > falseSymbols
Definition: TextInput.h:401
bool cppBlockComments
Definition: TextInput.h:241
int startingLineNumberOffset
Definition: TextInput.h:331
bool msvcFloatSpecials
Definition: TextInput.h:364
std::string sourceFileName
Definition: TextInput.h:325
bool otherLineComments
Definition: TextInput.h:259
char singleQuoteCharacter
Definition: TextInput.h:316
bool insert(const T &member)
Definition: Set.h:62
bool generateCommentTokens
Definition: TextInput.h:293
bool singleQuotedStrings
Definition: TextInput.h:311
bool generateNewlineTokens
Definition: TextInput.h:301
bool cppLineComments
Definition: TextInput.h:249
bool simpleFloatSpecials
Definition: TextInput.h:368
bool signedNumbers
Definition: TextInput.h:305

+ Here is the call graph for this function:

Member Data Documentation

bool G3D::TextInput::Settings::caseSensitive

When parsing booleans and msvcFloatSpecials, is case significant? Default is {true}

bool G3D::TextInput::Settings::cppBlockComments

If true, C-style slash-star marks a multi-line comment.

See generateCommentTokens for rules on how this is applied.

Default is true.

bool G3D::TextInput::Settings::cppLineComments

If true, // begins a single line comment.

See generateCommentTokens for rules on how this is applied.

Default is true.

bool G3D::TextInput::Settings::escapeSequencesInStrings

If true, \r, \n, \t, \0, \\ and other escape sequences inside strings are converted to the equivalent C++ escaped character. If false, backslashes are treated literally. It is convenient to set to false if reading Windows paths, for example, like c:\foo\bar.

Default is true.

Set<std::string> G3D::TextInput::Settings::falseSymbols

See trueSymbols. Default is {false}

bool G3D::TextInput::Settings::generateCommentTokens

If true, comments enabled by cppBlockComments, cppLineComments and otherLineComments will generate their respective tokens. If false, the same settings will enable parsing and ignoring comments

Default is false.

bool G3D::TextInput::Settings::generateNewlineTokens

If true, newlines will generate tokens. If false, newlines will be discarded as whitespace when parsed outside of other tokens.

Default is false.

bool G3D::TextInput::Settings::msvcFloatSpecials

Parse "-1.#IND00" as the floating point number returned by G3D::nan(), "-1.#INF00" as - G3D::inf(), and "1.#INF00" as G3D::inf().

Note that the C99 standard specifies that a variety of formats like "nan" are to be used; these are supported by G3D::TextInput::Settings::simpleFloatSpecials.

An alternative to specifying msvcFloatSpecials is to read numbers as:

  Token x =;
  Token y = t.peek();
  if ((x.string() == "-1.") && 
      (y.string() == "#INF00") && 
      (y.character() == x.character() + 3) &&
      (y.line() == x.line()) {;
      return nan();
  // ... similar cases for inf

If the single-comment character was #, the floating point special format overrides the comment and will be parsed instead.

If signedNumbers is false msvcFloatSpecials will not be parsed.

Default is true.

char G3D::TextInput::Settings::otherCommentCharacter

If not '\0', specifies a character that begins single line comments ('#' and '' are popular choices). This is independent of the cppLineComments flag. If the character appears in text with a backslash in front of it, it is considered escaped and is not treated as a comment character.

Default is '\0'.

char G3D::TextInput::Settings::otherCommentCharacter2

Another (optional) 1-comment character. Useful for files that support multiple comment syntaxes. Default is '\0'.

bool G3D::TextInput::Settings::otherLineComments

If true, otherCommentCharacter and otherCommentCharacter2 are used to begin single line comments in the same way cppLineComments is.

See generateCommentTokens for rules on how this is applied.

Default is true.

bool G3D::TextInput::Settings::proofSymbols

Parse the following set of useful proof symbols:

=> ::> <:: :> <: |- ::= := <-

Default is false.

bool G3D::TextInput::Settings::signedNumbers

If true, "-1" parses as the number -1 instead of the symbol "-" followed by the number 1. Default is true.

bool G3D::TextInput::Settings::simpleFloatSpecials

Parses "+inf', "-inf", "inf", "nan" as floats instead of symbols. Defaults to true.

char G3D::TextInput::Settings::singleQuoteCharacter

The character to use as a single quote. Defaults to "'" (backquote), occasionally useful to set to "`" (forward quote) or to "," (comma) for reading CSV files.

bool G3D::TextInput::Settings::singleQuotedStrings

If true, strings can be marked with single quotes (e.g., 'aaa'). If false, the quote character is parsed as a symbol. Default is true. Backquote (`) is always parsed as a symbol.

std::string G3D::TextInput::Settings::sourceFileName

If set to a non-empty string, that string will be used in place of the real file name (or in place of a pseudonym constructed from the buffer if given FROM_STRING) in tokens and exceptions.

Default is empty.

int G3D::TextInput::Settings::startingLineNumberOffset

Added to the line number reported by peekLineNumber and in exceptions. Useful for concatenating files that are parsed separately. Default is zero.

Set<std::string> G3D::TextInput::Settings::trueSymbols

All symbols that will become the 'true' boolean token. See also caseSensitive. Clear this value to disable parsing of true booleans.

Default is {true}.

The documentation for this class was generated from the following files: