Module dev.civl.abc
Package dev.civl.abc.token.IF

Interface CivlcToken

All Superinterfaces:
org.antlr.runtime.Token
All Known Subinterfaces:
CharacterToken, StringToken
public interface CivlcToken extends org.antlr.runtime.Token

A post-preprocessor token. Each token belongs to at most one token source. A token source is essentially an abstraction for a sequence of tokens. The token has some index (position) in this sequence, and there are methods to get the index of this token, and to get the next element in the sequence.

A token also carries detailed information about its formation history. This can be obtained by method getFormation(), which returns a Formation.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Interface
    Description
    static enum 
    CivlcToken.TokenVocabulary
     

  • Field Summary

    Fields inherited from interface org.antlr.runtime.Token

    DEFAULT_CHANNEL, DOWN, EOF, EOR_TOKEN_TYPE, HIDDEN_CHANNEL, INVALID_TOKEN, INVALID_TOKEN_TYPE, MIN_TOKEN_TYPE, SKIP_TOKEN, UP

  • Method Summary

    Modifier and Type
    Method
    Description
    Formation
    getFormation()
    A record of how this token was formed, through macro expansions, token concatenation, include directives, etc.
    int
    getIndex()
    Returns the index of this token (among its siblings).
    CivlcToken
    getNext()
    Returns the "next" token, which, if non-null, should have index one greater than this token's index.
    SourceFile
    getSourceFile()
    Returns the file that actually contains the token.
    int
    getStartIndex()
    Sets this token's expandable bit to false.
    int
    getStopIndex()
     
    CivlcToken.TokenVocabulary
    getTokenVocab()
    Return this token's CivlcToken.TokenVocabulary
    String
    getWhiteText()
    Return this token's white text, which is used for Fortran lexer
    void
    setIndex(int index)
    Sets the index of this token.
    void
    setNext(CivlcToken nextToken)
    Sets the next token in the sequence.
    void
    setStartIndex(int index)
     
    void
    setStopIndex(int index)
     
    void
    setTokenVocab(CivlcToken.TokenVocabulary newTokenVocab)
    Set the language kind of this token, when it is converted to a language-based token.
    void
    setWhiteText(String text)
    Set this token's white text, which is used for Fortran lexer

    Methods inherited from interface org.antlr.runtime.Token

    getChannel, getCharPositionInLine, getInputStream, getLine, getText, getTokenIndex, getType, setChannel, setCharPositionInLine, setInputStream, setLine, setText, setTokenIndex, setType

  • Method Details

    • getFormation

      Formation getFormation()
      A record of how this token was formed, through macro expansions, token concatenation, include directives, etc.
      Returns:
      history of token formation

    • getSourceFile

      SourceFile getSourceFile()
      Returns the file that actually contains the token.
      Returns:
      the source file

    • setNext

      void setNext(CivlcToken nextToken)
      Sets the next token in the sequence. The next token, if non-null, should have index one greater than the index of this token.
      Parameters:
      nextToken -

    • getNext

      CivlcToken getNext()
      Returns the "next" token, which, if non-null, should have index one greater than this token's index. Could be null.
      Returns:
      the next token

    • setIndex

      void setIndex(int index)
      Sets the index of this token.
      Parameters:
      index -

    • getIndex

      int getIndex()
      Returns the index of this token (among its siblings). Tokens are indexed from 0.
      Returns:
      the index of this token

    • getStartIndex

      int getStartIndex()
      Sets this token's expandable bit to false.

    • getStopIndex

      int getStopIndex()

    • setStartIndex

      void setStartIndex(int index)

    • setStopIndex

      void setStopIndex(int index)

    • getWhiteText

      String getWhiteText()
      Return this token's white text, which is used for Fortran lexer
      Returns:
      the white text of this token.

    • setWhiteText

      void setWhiteText(String text)
      Set this token's white text, which is used for Fortran lexer
      Parameters:
      text -

    • getTokenVocab

      CivlcToken.TokenVocabulary getTokenVocab()
      Return this token's CivlcToken.TokenVocabulary
      Returns:
      this token's CivlcToken.TokenVocabulary

    • setTokenVocab

      void setTokenVocab(CivlcToken.TokenVocabulary newTokenVocab)
      Set the language kind of this token, when it is converted to a language-based token. The CivlcToken.TokenVocabulary should be changed from PREPROC to a real language king (i.e., CIVLC for Civl-C/C, FORTRAN for Fortran.)
      Parameters:
      newTokenVocab -