Module token defines the various kinds of tokens and related classes used by ABC. This modules also provides a factory for producing these objects. ABC tokens contain a lot of information, including references to the original source code snippet from which the token originated, the token's formation history from preprocessor directives, and so on.

  • Class
    A character token represents a character constant in a source program, e.g., 'a'.
    A post-preprocessor token.
    Represents a finite sequence of CivlcToken.
    Extends ANTLR's TokenSource interface by adding some additional functionality: getting the macro information, and methods to get the number of tokens produced so far and to retrieve any token produced so far by index.
    A Concatenation represents the concatenation of a sequence of tokens.
    A FileIndexer keeps track of all source files opened during an invocation of ABC.
    A formation is a record of the history of events that went into the formation of a token.
    A FunctionMacro represents a C preprocessor function-like macro, which has the from #define f(X1,X2,...) ....
    An Inclusion represents the application of a preprocessor #include directive.
    An abstract representation of a preprocessor macro.
    The body of a Macro definition consists of a sequence of Macro.ReplacementUnits, each of which comprises a preprocessing token (a non-whitespace token known as the "replacement token" in C11) plus some possible whitespace.
    A MacroExpansion represents an instance of the expansion of a preprocessor (object or function) macro, which replaces one token by a sequence of new tokens.
    An Object macro represents a C preprocessor object macro: a directive of the form #define X ... where X is the name of the macro and the ... is a list of replacement tokens.
    A "Source" represents a range of CTokens (post-preprocessor tokens) from the token stream that forms the input to the parser.
    Information object for a source file processed by ABC.
    Represents use of the '#' operator in a function-like macro application.
    A string literal object results from processing a string literal in the source code.
    The different kinds of strings, based on the kind of characters they contain.
    A StringToken is formed from a sequence of one or more CTokens.
    A factory for producing all the objects under the control of the token module.
    This class is the entry point for the "token" module.
    Utility class providing static methods dealing with Token objects.
    Represents the formation of new tokens by a transformer.
    An exception for which the source (in the source program being compiled) is unknown.