Module token defines the various kinds of tokens and related classes used by ABC. This modules also provides a factory for producing these objects. ABC tokens contain a lot of information, including references to the original source code snippet from which the token originated, the token's formation history from preprocessor directives, and so on.
ClassDescriptionA character token represents a character constant in a source program, e.g.,
'a'.A post-preprocessor token.Represents a finite sequence of
TokenSourceinterface by adding some additional functionality: getting the macro information, and methods to get the number of tokens produced so far and to retrieve any token produced so far by index.A
Concatenationrepresents the concatenation of a sequence of tokens.A
FileIndexerkeeps track of all source files opened during an invocation of ABC.A formation is a record of the history of events that went into the formation of a token.A FunctionMacro represents a C preprocessor function-like macro, which has the from
#define f(X1,X2,...) ....An Inclusion represents the application of a preprocessor
#includedirective.An abstract representation of a preprocessor macro.A MacroExpansion represents an instance of the expansion of a preprocessor (object or function) macro, which replaces one token by a sequence of new tokens.An Object macro represents a C preprocessor object macro: a directive of the form
#define X ...where
Xis the name of the macro and the
...is a list of replacement tokens.A "Source" represents a range of CTokens (post-preprocessor tokens) from the token stream that forms the input to the parser.Information object for a source file processed by ABC.Represents use of the '#' operator in a function-like macro application.A string literal object results from processing a string literal in the source code.The different kinds of strings, based on the kind of characters they contain.A StringToken is formed from a sequence of one or more CTokens.A factory for producing all the objects under the control of the token module.This class is the entry point for the "token" module.Utility class providing static methods dealing with Token objects.Represents the formation of new tokens by a transformer.An exception for which the source (in the source program being compiled) is unknown.