Module dev.civl.abc
Package dev.civl.abc.token.IF

Interface TokenFactory

public interface TokenFactory
A factory for producing all the objects under the control of the token module. These includes instances of the following types (and their subtypes):

  • Method Details

    • newMacroExpansion

      MacroExpansion newMacroExpansion(CivlcToken startToken, Macro macro, int index)
      Returns a standard macro expansion formation object. This is a formation that represents a token created through the process of macro expansion.
      Parameters:
      startToken - the token within the macro application expression that led to the formation of the new token; this could be either the macro name itself or a token from one of the arguments
      macro - the Macro object given an abstraction represnetation of the macro
      index - the index of the replacement token (numbered from 0) in the macro replacement list that led to the final token
      Returns:
      a new formation incorporating the specified values

    • newBuiltinMacroExpansion

      Formation newBuiltinMacroExpansion(CivlcToken macroToken)
      Returns a new built-in macro expansion formation object. This represents a token formed by expanding one of the predefined object macros such as "__FILE__" or "__LINE__".
      Parameters:
      macroToken - the original token, such as "__FILE__" or "__LINE__" that is being expanded
      Returns:
      a new Formation object representing the expansion of such a macro

    • newStringification

      Stringification newStringification(FunctionMacro macro, int index, List<CivlcToken> argument)
      Formation of a string literal token through the use of the preprocessor "#" operator during the application of a function-like macro.
      Parameters:
      macro - the function-like macro being applied
      index - the index of the replacement token (numbered from 0) in the macro replacement list involved in the formation of the new token; this replacement token will necessarily be a parameter immediately following a "#" token
      argument - the sequence of non-whitespace tokens comprising the argument in the macro invocation
      Returns:
      a new formation incorporating specified values

    • newConcatenation

      Concatenation newConcatenation(List<CivlcToken> tokens)
      A formation of a token by either (1) concatenating 0 or more tokens using the preprocessor "##" operator, or (2) concatenating 1 or more string literal tokens immediately after preprocessing.
      Parameters:
      tokens - list of tokens to concatenate; should not include whitespace

    • newInclusion

      Inclusion newInclusion(SourceFile file)
      Inclusion record for original source file.
      Parameters:
      file - the file which was included, which should be the original source file (the root of the inclusion tree)
      Returns:
      a new inclusion record

    • newInclusion

      Inclusion newInclusion(SourceFile file, CivlcToken includeToken)

    • newSystemFormation

      Formation newSystemFormation(String identifier)
      Creates a new formation which represents some code added by the system itself, as opposed to code that emanated from an actual source file. The identifier should be a short string indicating what part of the system created the code. Examples: "The CIVL-MPI Transformer". The identifier will be used to form a "fake" File, which will be used to form a SourceFile, and that is what will be returned by the formation's Formation.getLastFile() method.
      Parameters:
      identifier - short string indicating what part of the system created this code; used in messages
      Returns:
      a new system formation object

    • newTransformFormation

      Formation newTransformFormation(String transformerName, String method)

    • newCivlcToken

      CivlcToken newCivlcToken(org.antlr.runtime.Token token, Formation formation, CivlcToken.TokenVocabulary tokenVocab)

    • newCivlcToken

      CivlcToken newCivlcToken(int type, String text, Formation formation, CivlcToken.TokenVocabulary tokenVocab)

    • newCivlcToken

      CivlcToken newCivlcToken(org.antlr.runtime.CharStream input, int type, int channel, int start, int stop, Formation formation, int line, int charPositionInLine, CivlcToken.TokenVocabulary tokenVocab)
      Parameters:
      input - the character stream from which this token was formed
      type - the type of the new token, which is a unique integer ID assigned to each category of token by the lexer, e.g., IDENTIFIER
      channel - all tokens go to the parser (unless skip() is called in that rule) on a particular "channel". The parser tunes to a particular channel so that whitespace etc... can go to the parser on a "hidden" channel.
      start - the char position into the input buffer where this token starts
      stop - the char position into the input buffer where this token stops
      formation - the object specifying the complete history on how this token came to exist
      line - the line number on which the token occurs
      charPositionInLine - the index of the first character of the token in the line in which it occurs. Indexes start at 0 for first character in line.
      tokenVocab - the lexical vocabulary used by thisinvalid input: '<'
      Returns:
      a new instance of CivlcToken with fields as specified above

    • executionCharacter

      ExecutionCharacter executionCharacter(ExecutionCharacter.CharacterKind kind, int codePoint, char[] characters)

    • characterToken

      CharacterToken characterToken(CivlcToken token) throws SyntaxException
      Throws:
      SyntaxException

    • newStringToken

      StringToken newStringToken(CivlcToken token) throws SyntaxException
      Throws:
      SyntaxException

    • newStringToken

      StringToken newStringToken(List<CivlcToken> tokens) throws SyntaxException
      Constructs a new string literal token (instance of StringToken) by concatenating a list of string literal tokens. According to the C Standard, after preprocessing, adjacent string literal tokens are concatenated. (They may be separated by white space, which is ignored.)
      Parameters:
      tokens - sequence of non-null string literal tokens
      Returns:
      a single token obtained by concatenating the given ones
      Throws:
      SyntaxException - if the new string cannot be formed for some reason

    • newSource

      Source newSource(CivlcToken token)
      Returns a Source consisting of a single Token.
      Parameters:
      token - a non-null Token
      Returns:
      a Source consisting of the single given token

    • newSource

      Source newSource(CivlcToken first, CivlcToken last)
      Computes a Source comprising all Tokens from first to last, inclusive.
      Parameters:
      first - a non-null Token
      last - a non-null Token from the same token sequence as first
      Returns:
      a Source comprising all Tokens from first to last, inclusive

    • join

      Source join(Source source, CivlcToken token)
      Computes a minimal Source containing the given source and token. The given source and token must come from the same token sequence. This is useful when you want to increase a source by one token.
      Parameters:
      source - a non-null Source
      token - a non-null Token from the same token sequence as source
      Returns:
      a minimal Source containing the given source and token

    • join

      Source join(Source source1, Source source2)
      Computes a minimal Source containing the two given sources.
      Parameters:
      source1 - a non-null Source
      source2 - a non-null Source from the same token sequence as source1
      Returns:
      a minimal source containing the two given ones

    • newSyntaxException

      SyntaxException newSyntaxException(String message, Source source)

    • newSyntaxException

      SyntaxException newSyntaxException(String message, CivlcToken token)

    • newSyntaxException

      SyntaxException newSyntaxException(UnsourcedException e, Source source)

    • newSyntaxException

      SyntaxException newSyntaxException(UnsourcedException e, CivlcToken token)

    • newUnsourcedException

      UnsourcedException newUnsourcedException(String message)

    • newObjectMacro

      ObjectMacro newObjectMacro(org.antlr.runtime.tree.Tree definitionNode, SourceFile file)

    • newFunctionMacro

      FunctionMacro newFunctionMacro(org.antlr.runtime.tree.Tree definitionNode, SourceFile file)

    • getTokenSubsequence

      CivlcTokenSequence getTokenSubsequence(CivlcTokenSource fullSource, CivlcToken startToken, CivlcToken stopToken)

    • getEmptyTokenSubsequence

      CivlcTokenSequence getEmptyTokenSubsequence(CivlcTokenSource originalSource)

    • getCivlcTokenSourceByTokens

      CivlcTokenSource getCivlcTokenSourceByTokens(List<? extends org.antlr.runtime.Token> tokens, Formation formation)
      creates a CivlC Token Source based on a give list of tokens (not necessarily CivlC token). All given tokens will be transformed to CivlC tokens if they are not CivlC tokens, using the given formation.
      Parameters:
      tokens - the list of tokens
      formation - the formation to be used when transforming the given tokens
      Returns:
      a CivlC token source wrapping the given list of tokens

    • newFileIndexer

      FileIndexer newFileIndexer()