Fortran Translation Issues

Redesign the Preprocessor

1. Preprocessor

1). Input: Raw char stream read from source files

2). Output: A pp-token stream

2. PP Token Converter

1). Input: A pp-token stream

2). Output: A CIVL-C token stream.

3). Functionality:

Based on a specific programming language (C/CIVL-C or Fortran), the convertor analyzes each pp-token and generates a corresponding !CIVLCToken for each of them.

3. Language Based Parser

1). Input: A CIVLCToken stream

2). Output: A CIVL parse tree

Meeting Notes for Fortran Front-End

01-08-2019

0). The preprocessor will only create and handle pp-tokens (preprocess tokens).

1). All language specific tokens & keywords should be created after the preprocess procedure.

2). Language grammar based converters (C/CIVL-C & Fortran) should be developed for generating CIVLC tokens from pp-tokens.

3). Different parsers will be used for pars the CIVL-C token stream

1). A detailed plan of adding, removing and changing class/grammar files

2). Preprocessor grammar: (Both Prep Lexer and Parser grammars in one file)

CIVLC Lexer grammar: (Mainly for debugging) Defines all CIVLC Tokens

PP2CIVLTokenConverter Converts a pp-token to a CIVLC token, or combine several into a single CIVLC token like ..

CIVLC Parser grammar: (Using CIVLC Token)

Fortran Parser grammar: (Using CIVLC Token)

Front End Packages and Classes

edu.udel.cis.vsl.abc.front

edu.udel.cis.vsl.abc.front.IF

Front.java [Unchanged]

Preprocessor.java [Unchanged]

PreprocessorException.java [Moved] Move to package: edu.udel.cis.vsl.abc.front.IF.exception

PreprocessorExpressionException.java [Moved] Move to package: edu.udel.cis.vsl.abc.front.IF.exception

PreprocessorRuntimeException.java [Moved] Move to package: edu.udel.cis.vsl.abc.front.IF.exception

IllegalMacroArgumentException.java [Moved] Move to package: edu.udel.cis.vsl.abc.front.IF.exception

Parser.java [Unchanged]

ParseTree.java [Unchanged]

ParseException.java [Moved] Move to package: edu.udel.cis.vsl.abc.front.IF.exception

RuntimeParseException.java [Moved] Move to package: edu.udel.cis.vsl.abc.front.IF.exception

ASTBuilder [Unchanged]

CivlcTokenConstant [Unchanged]

edu.udel.cis.vsl.abc.front.IF.exception [Created]

edu.udel.cis.vsl.abc.front.common

edu.udel.cis.vsl.abc.front.common.astgen [Unchanged]

edu.udel.cis.vsl.abc.front.common.ptree [Unchanged]

edu.udel.cis.vsl.abc.front.common.preproc

!CTokenIterator.java

edu.udel.cis.vsl.abc.front.common.parse

OmpPragmaParser.java [Move] Move to package: edu.udel.cis.vsl.abc.front.IF

TreeUtils.java [Unchanged]

edu.udel.cis.vsl.abc.front.c

edu.udel.cis.vsl.abc.front.c.astgen [Unchanged]

edu.udel.cis.vsl.abc.front.c.ptree [Unchanged]

edu.udel.cis.vsl.abc.front.c.preproc

CommonCharacterStream.java [Unchanged]

FilteredStream.java [Unchanged]

CPreprocessor.java [Changed] Update for the change in 'PreprocessorTokenSource.java'

MacroExpander.java [Changed] Update for the change in 'PreprocessorTokenSource.java'

PreprocessorExpressionAnalyzer.java [Changed] Use pp-tokens.

PreprocessorSourceFileInfo.java [Unchanged]

PreprocessorTokenSource.java [Changed] It should implement PPTokenSource, which should implement Antlr's TokenSource.

PreprocessorUtils.java [Unchanged]

edu.udel.cis.vsl.abc.front.c.converter

PP2CivlcConverter.java [Created] It should implement CIVL interface CivlcTokenSource. Take a stream of pp-tokens and convert them into a CivlcToken. The constructor consumes a PPTokenSource.

edu.udel.cis.vsl.abc.front.fortran

edu.udel.cis.vsl.abc.front.fortran.astgen

FOmpPragmaHandler.java [Changed] Add functions for supporting more Fortran-OpenMP bindings.

FortranASTBuilder.java [Changed] Fix known issues/defects listed in 'FortranTransformations'; Add functions for supporting more Fortran behaviors.

FortranASTBuilderWorker.java [Unchanged]

edu.udel.cis.vsl.abc.front.fortran.ptree

FortranTree.java [Changed] Be unified as 'CParseTree.java'; Extend 'CommonParseTree'

edu.udel.cis.vsl.abc.front.fortran.preproc

FortranLexicalPrepass.java [Deleted] Should be deleted, because CPreprocessor will be used.

FortranPreprocessor.java [Changed] Saved for implementing FortranPreprocessor defined in standard.

FortranStream.java [Changed] Saved for 'FortranPreprocessor'

FortranTokenSource.java [Deleted] Should be deleted, because CivlcTokenSource will be used.

FortranTokenStream.java [Deleted] Should be deleted. The preprocessing procedure will be unified.

edu.udel.cis.vsl.abc.front.fortran.converter

PP2FortranConverter.java [Created] It should implement CIVL interface CivlcTokenSource. Take a stream of pp-tokens and convert them into a CivlcToken. The constructor consumes a PPTokenSource. (Or CivlFortranToken, which extends CivlCToken, if extra info is needed.)

edu.udel.cis.vsl.abc.front.fortran.parse

AbstractFortranParser [Deleted] Deleted for using a unified pattern like 'CParser'.

IActionEnums [Deleted] Deleted for using a unified pattern like 'CParser'.

IFortranParserAction [Deleted] Deleted for using a unified pattern like 'CParser'.

IFortranParser [Deleted] Deleted for using a unified pattern like 'CParser'.

FortranParserActionFactory [Deleted] Deleted for using a unified pattern like 'CParser'.

FortranParserActionNull [Deleted] Not in use.

FortranParserActionPrint [Deleted] Not in use.

FortranParserActionTreeMaker [Changed] Changed for using a unified pattern like 'CParser'; Remove the conversion from OpenFortranParser token system to the CivlCToken one; Add functions supporting more Fortran primitives/behaviors mentioned in 'FortranTransformations'

FortranParser [Changed] Changed for using a unified pattern like 'CParser'.

FOmpParser [Changed] Add functions for supporting more Fortran-OpenMP bindings.

Grammar Files

Preprocessor

PreprocessorLexer.g [Changed] Remove all C & CIVL-C keywords.

PreprocessorParser.g [Changed] Removed all language specific keywords, which will be treated as identifiers and be converted later.

PreprocessorExpressionParser.g [Changed] Removed all language specific keywords, which will be treated as identifiers and be converted later.

C & CIVL-C

CivlCParser.g [Changed] Remove the vocabulary used. Put all C & CIVL-C tokens here.

CivlCParser_alt.g [Changed] Use the token vocabulary used in CivlCParser.g

AcslParser.g [Changed] Use the token vocabulary used in CivlCParser.g

OmpLexer.g [Unchanged]

OmpParser.g [Unchanged]

Fortran

FortranLexer.g [Deleted] All Fortran tokens will be put in FortranParser08.g. And keywords will be converted in PP2CivlcConverter.

FortranParser03.g [Deleted] Not in use.

FortranParser08.g [Changed] The format of all grammar rules should be unified with C/CIVL-C grammar style used in CivlCParser.g [Renamed as FortranPArser90.g]

FortranParserExtras.g [Merged] Because supported Fortran features is a subset, then all supported grammar should be able to defined in a single file.

FortranParserRiceCAF.g [Deleted] Not in use.

OmpLexerF08.g [Changed] Use the token vocabulary of FortranParser08.g

OmpParserF08.g [Changed] Add more supported Fortran-OpenMP directives.