[flang] A rework of the cmake build components for in and out of tree builds.

In general all the basic functionality seems to work and removes some redundancy
and more complicated features in favor of borrowing infrastructure from LLVM
build configurations. Here's a quick summary of details and remaining issues:

  * Testing has spanned Ubuntu 18.04 & 19.10, CentOS 7, RHEL 8, and
    MacOS/darwin.  Architectures include x86_64 and Arm.  Without
    access to Window nothing has been tested there yet.

  * As we change file and directory naming schemes (i.e.,
    capitalization) some odd things can occur on MacOS systems with
    case preserving but not case senstive file system configurations.
    Can be painful and certainly something to watch out for as any
    any such changes continue.

  * Testing infrastructure still needs to be tuned up and worked on.
    Note that there do appear to be cases of some tests hanging (on
    MacOS in particular).  They appear unrelated to the build
    process.

  * Shared library configurations need testing (and probably fixing).

  * Tested both standalone and 'in-mono repo' builds.  Changes for
    supporting the mono repo builds will require LLVM-level changes that
    are straightforward when the time comes.

  * The configuration contains a work-around for LLVM's C++ standard mode
    passing down into Flang/F18 builds (i.e., LLVM CMake configuration would
    force a -std=c++11 flag to show up in command line arguments.  The
    current configuration removes that automatically and is more strict in
    following new CMake guidelines for enforcing C++17 mode across all the
    CMake files.

  * Cleaned up a lot of repetition in the command line arguments.  It
    is likely that more work is still needed to both allow for
    customization and working around CMake defailts (or those
    inherited from LLVM's configuration files). On some platforms agressive
    optimization flags (e.g. -O3) can actually break builds due to the inlining
    of templates in .cpp source files that then no longer are available for use
    cases outside those source files (shows up as link errors).   Sticking at -O2
    appears to fix this.  Currently this CMake configuration forces this in
    release mode but at the cost of stomping on any CMake, or user customized,
    settings for the release flags.

  * Made the lit tests non-source directory dependent where appropriate. This is
    done by configuring certain test shell files to refer to the correct paths
    whether an in or out of tree build is being performed. These configured
    files are output in the build directory. A %B substitution is introduced in
    lit to refer to the build directory, mirroring the %S substitution for the
    source directory, so that the tests can refer to the configured shell scripts.

Co-authored-by: David Truby <david.truby@arm.com>

Original-commit: flang-compiler/f18@d1c7184
Reviewed-on: flang-compiler/f18#1045

Author: pmccormick

flang/.gitignore

@@ -17,3 +17,5 @@ CMakeCache.txt
 */*/Makefile
 cmake_install.cmake
 formatted
+.DS_Store 
+.vs_code 

flang/CMakeLists.txt

@@ -150,43 +150,18 @@ or
 ```
 CXX=/opt/gcc-7.2/bin/g++-7.2 cmake ...
 ```
-There's a third option!
-The CMakeList.txt file uses the variable GCC
-as the path to the bin directory containing the C++ compiler.
-
-GCC can be defined on the cmake command line
-where `<GCC_DIRECTORY>` is the path to a GCC installation with bin, lib, etc:
-```
-cmake -DGCC=<GCC_DIRECTORY> ...
-```
 
 ### Building f18 with clang
 
 To build f18 with clang,
 cmake needs to know how to find clang++
 and the GCC library and tools that were used to build clang++.
 
-The CMakeList.txt file expects either CXX or BUILD_WITH_CLANG to be set.
-
 CXX should include the full path to clang++
 or clang++ should be found on your PATH.
 ```
 export CXX=clang++
 ```
-BUILD_WITH_CLANG can be defined on the cmake command line
-where `<CLANG_DIRECTORY>`
-is the path to a clang installation with bin, lib, etc:
-```
-cmake -DBUILD_WITH_CLANG=<CLANG_DIRECTORY>
-```
-Or GCC can be defined on the f18 cmake command line
-where `<GCC_DIRECTORY>` is the path to a GCC installation with bin, lib, etc:
-```
-cmake -DGCC=<GCC_DIRECTORY> ...
-```
-To use f18 after it is built,
-the environment variables PATH and LD_LIBRARY_PATH
-must be set to use GCC and its associated libraries.
 
 ### Installation Directory
 

flang/README.md

@@ -0,0 +1,141 @@
+macro(set_flang_windows_version_resource_properties name)
+  if (DEFINED windows_resource_file)
+    set_windows_version_resource_properties(${name} ${windows_resource_file}
+      VERSION_MAJOR ${FLANG_VERSION_MAJOR}
+      VERSION_MINOR ${FLANG_VERSION_MINOR}
+      VERSION_PATCHLEVEL ${FLANG_VERSION_PATCHLEVEL}
+      VERSION_STRING "${FLANG_VERSION} (${BACKEND_PACKAGE_STRING})"
+      PRODUCT_NAME "flang")
+  endif()
+endmacro()
+
+macro(add_flang_subdirectory name)
+  add_llvm_subdirectory(FLANG TOOL ${name})
+endmacro()
+
+macro(add_flang_library name)
+  cmake_parse_arguments(ARG
+    "SHARED"
+    ""
+    "ADDITIONAL_HEADERS"
+    ${ARGN})
+  set(srcs)
+  if (MSVC_IDE OR XCODE)
+    # Add public headers
+    file(RELATIVE_PATH lib_path
+      ${FLANG_SOURCE_DIR}/lib/
+      ${CMAKE_CURRENT_SOURCE_DIR})
+    if(NOT lib_path MATCHES "^[.][.]")
+      file( GLOB_RECURSE headers
+        ${FLANG_SOURCE_DIR}/include/flang/${lib_path}/*.h
+        ${FLANG_SOURCE_DIR}/include/flang/${lib_path}/*.def)
+      set_source_files_properties(${headers} PROPERTIES HEADER_FILE_ONLY ON)
+
+      if (headers)
+        set(srcs ${headers})
+      endif()
+    endif()
+  endif(MSVC_IDE OR XCODE)
+
+  if (srcs OR ARG_ADDITIONAL_HEADERS)
+    set(srcs
+      ADDITIONAL_HEADERS
+      ${srcs}
+      ${ARG_ADDITIONAL_HEADERS}) # It may contain unparsed unknown args.
+      
+  endif()
+
+  if (ARG_SHARED)
+    set(LIBTYPE SHARED)
+  else()
+    # llvm_add_library ignores BUILD_SHARED_LIBS if STATIC is explicitly set,
+    # so we need to handle it here.
+    if (BUILD_SHARED_LIBS)
+      set(LIBTYPE SHARED OBJECT)
+    else()
+      set(LIBTYPE STATIC OBJECT)
+    endif()
+    set_property(GLOBAL APPEND PROPERTY FLANG_STATIC_LIBS ${name})
+  endif()
+
+  llvm_add_library(${name} ${LIBTYPE} ${ARG_UNPARSED_ARGUMENTS} ${srcs})
+
+  if (TARGET ${name})
+    target_link_libraries(${name} INTERFACE ${LLVM_COMMON_LIBS})
+
+    if (NOT LLVM_INSTALL_TOOLCHAIN_ONLY OR ${name} STREQUAL "libflang")
+      set(export_to_flangtargets)
+      if (${name} IN_LIST LLVM_DISTRIBUTION_COMPONENTS OR
+          "flang-libraries" IN_LIST LLVM_DISTRIBUTION_COMPONENTS OR
+          NOT LLVM_DISTRIBUTION_COMPONENTS)
+        set(export_to_flangtargets EXPORT FlangTargets)
+        set_property(GLOBAL PROPERTY FLANG_HAS_EXPORTS True)
+      endif()
+
+      install(TARGETS ${name}
+        COMPONENT ${name}
+        ${export_to_flangtargets}
+        LIBRARY DESTINATION lib${LLVM_LIBDIR_SUFFIX}
+        ARCHIVE DESTINATION lib${LLVM_LIBDIR_SUFFIX}
+        RUNTIME DESTINATION bin)
+
+      if (NOT LLVM_ENABLE_IDE)
+        add_llvm_install_targets(install-${name}
+                                 DEPENDS ${name}
+                                 COMPONENT ${name})
+      endif()
+
+      set_property(GLOBAL APPEND PROPERTY FLANG_LIBS ${name})
+    endif()
+    set_property(GLOBAL APPEND PROPERTY FLANG_EXPORTS ${name})
+  else()
+    # Add empty "phony" target
+    add_custom_target(${name})
+  endif()
+
+  set_target_properties(${name} PROPERTIES FOLDER "Flang libraries")
+  set_flang_windows_version_resource_properties(${name})
+endmacro(add_flang_library)
+
+macro(add_flang_executable name)
+  add_llvm_executable(${name} ${ARGN})
+  set_target_properties(${name} PROPERTIES FOLDER "Flang executables")
+  set_flang_windows_version_resource_properties(${name})
+endmacro(add_flang_executable)
+
+macro(add_flang_tool name)
+  if (NOT FLANG_BUILD_TOOLS)
+    set(EXCLUDE_FROM_ALL ON)
+  endif()
+
+  add_flang_executable(${name} ${ARGN})
+  add_dependencies(${name} flang-resource-headers)
+
+  if (FLANG_BUILD_TOOLS)
+    set(export_to_flangtargets)
+    if (${name} IN_LIST LLVM_DISTRIBUTION_COMPONENTS OR
+        NOT LLVM_DISTRIBUTION_COMPONENTS)
+      set(export_to_flangtargets EXPORT FlangTargets)
+      set_property(GLOBAL PROPERTY FLANG_HAS_EXPORTS True)
+    endif()
+
+    install(TARGETS ${name}
+      ${export_to_flangtargets}
+      RUNTIME DESTINATION bin
+      COMPONENT ${name})
+
+    if(NOT LLVM_ENABLE_IDE)
+      add_llvm_install_targets(install-${name}
+                               DEPENDS ${name}
+                               COMPONENT ${name})
+    endif()
+    set_property(GLOBAL APPEND PROPERTY FLANG_EXPORTS ${name})
+  endif()
+endmacro()
+
+macro(add_flang_symlink name dest)
+  add_llvm_tool_symlink(${name} ${dest} ALWAYS_GENERATE)
+  # Always generate install targets
+  llvm_install_symlink(${name} ${dest} ALWAYS_GENERATE)
+endmacro()
+

flang/cmake/modules/AddFlang.cmake

@@ -0,0 +1,74 @@
+# Generate a list of CMake library targets so that other CMake projects can
+# link against them. LLVM calls its version of this file LLVMExports.cmake, but
+# the usual CMake convention seems to be ${Project}Targets.cmake.
+set(FLANG_INSTALL_PACKAGE_DIR lib${LLVM_LIBDIR_SUFFIX}/cmake/flang)
+set(flang_cmake_builddir "${CMAKE_BINARY_DIR}/${FLANG_INSTALL_PACKAGE_DIR}")
+
+# Keep this in sync with llvm/cmake/CMakeLists.txt!
+set(LLVM_INSTALL_PACKAGE_DIR lib${LLVM_LIBDIR_SUFFIX}/cmake/llvm)
+set(llvm_cmake_builddir "${LLVM_BINARY_DIR}/${LLVM_INSTALL_PACKAGE_DIR}")
+
+get_property(FLANG_EXPORTS GLOBAL PROPERTY FLANG_EXPORTS)
+export(TARGETS ${FLANG_EXPORTS} FILE ${flang_cmake_builddir}/FlangTargets.cmake)
+
+# Generate FlangConfig.cmake for the build tree.
+set(FLANG_CONFIG_CMAKE_DIR "${flang_cmake_builddir}")
+set(FLANG_CONFIG_LLVM_CMAKE_DIR "${llvm_cmake_builddir}")
+set(FLANG_CONFIG_EXPORTS_FILE "${flang_cmake_builddir}/FlangTargets.cmake")
+set(FLANG_CONFIG_INCLUDE_DIRS
+  "${FLANG_SOURCE_DIR}/include"
+  "${FLANG_BINARY_DIR}/include"
+  )
+configure_file(
+  ${CMAKE_CURRENT_SOURCE_DIR}/FlangConfig.cmake.in
+  ${flang_cmake_builddir}/FlangConfig.cmake
+  @ONLY)
+set(FLANG_CONFIG_CMAKE_DIR)
+set(FLANG_CONFIG_LLVM_CMAKE_DIR)
+set(FLANG_CONFIG_EXPORTS_FILE)
+
+# Generate FlangConfig.cmake for the install tree.
+set(FLANG_CONFIG_CODE "
+  # Compute the installation prefix from this LLVMConfig.cmake file location.
+  get_filename_component(FLANG_INSTALL_PREFIX \"\${CMAKE_CURRENT_LIST_FILE}\" PATH)")
+# Construct the proper number of get_filename_component(... PATH)
+# calls to compute the installation prefix.
+string(REGEX REPLACE "/" ";" _count "${FLANG_INSTALL_PACKAGE_DIR}")
+foreach(p ${_count})
+  set(FLANG_CONFIG_CODE "${FLANG_CONFIG_CODE}
+      get_filename_component(FLANG_INSTALL_PREFIX \"\${FLANG_INSTALL_PREFIX}\" PATH)")
+endforeach(p)
+
+set(FLANG_CONFIG_CMAKE_DIR "\${FLANG_INSTALL_PREFIX}/${FLANG_INSTALL_PACKAGE_DIR}")
+set(FLANG_CONFIG_LLVM_CMAKE_DIR "\${FLANG_INSTALL_PREFIX}/${LLVM_INSTALL_PACKAGE_DIR}")
+set(FLANG_CONFIG_EXPORTS_FILE "\${FLANG_CMAKE_DIR}/FlangTargets.cmake")
+set(FLANG_CONFIG_INCLUDE_DIRS "\${FLANG_INSTALL_PREFIX}/include")
+
+configure_file(
+  ${CMAKE_CURRENT_SOURCE_DIR}/FlangConfig.cmake.in
+  ${CMAKE_CURRENT_BINARY_DIR}/CMakeFiles/FlangConfig.cmake
+  @ONLY)
+
+set(FLANG_CONFIG_CODE)
+set(FLANG_CONFIG_CMAKE_DIR)
+set(FLANG_CONFIG_EXPORTS_FILE)
+
+if (NOT LLVM_INSTALL_TOOLCHAIN_ONLY)
+  get_property(flang_has_exports GLOBAL PROPERTY FLANG_HAS_EXPORTS)
+  if(flang_has_exports)
+    install(EXPORT FlangTargets DESTINATION ${FLANG_INSTALL_PACKAGE_DIR}
+            COMPONENT flang-cmake-exports)
+  endif()
+
+  install(FILES
+    ${CMAKE_CURRENT_BINARY_DIR}/CMakeFiles/FlangConfig.cmake
+    DESTINATION ${FLANG_INSTALL_PACKAGE_DIR}
+    COMPONENT flang-cmake-exports)
+
+  if(NOT LLVM_ENABLE_IDE)
+    # Add a dummy target so this can be used with LLVM_DISTRIBUTION_COMPONENTS
+    add_custom_target(flang-cmake-exports)
+    add_llvm_install_targets(install-flang-cmake-exports
+                             COMPONENT flang-cmake-exports)
+  endif()
+endif()

flang/cmake/modules/CMakeLists.txt

@@ -0,0 +1,13 @@
+# This file allows users to call find_package(Flang) and pick up our targets.
+
+@FLANG_CONFIG_CODE@
+
+find_package(LLVM REQUIRED CONFIG
+             HINTS "@FLANG_CONFIG_LLVM_CMAKE_DIR@")
+
+set(FLANG_EXPORTED_TARGETS "@FLANG_EXPORTS@")
+set(FLANG_CMAKE_DIR "FLANG_CONFIG_CMAKE_DIR@")
+set(FLANG_INCLUDE_DIRS "@FLANG_CONFIG_INCLUDE_DIRS@")
+
+# Provide all our library targets to users.
+include("@FLANG_CONFIG_EXPORTS_FILE@")

flang/cmake/modules/FlangConfig.cmake.in

@@ -0,0 +1 @@
+add_subdirectory(flang)

flang/include/CMakeLists.txt

@@ -0,0 +1,5 @@
+#define FLANG_VERSION @FLANG_VERSION@
+#define FLANG_VERSION_STRING "@FLANG_VERSION@"
+#define FLANG_VERSION_MAJOR @FLANG_VERSION_MAJOR@
+#define FLANG_VERSION_MINOR @FLANG_VERSION_MINOR@
+#define FLANG_VERSION_PATCHLEVEL @FLANG_VERSION_PATCHLEVEL@

flang/include/flang/Version.inc.in

@@ -1,11 +1,3 @@
-#===-- lib/CMakeLists.txt --------------------------------------------------===#
-#
-# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-# See https://llvm.org/LICENSE.txt for license information.
-# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-#
-#===------------------------------------------------------------------------===#
-
 add_subdirectory(Common)
 add_subdirectory(Evaluate)
 add_subdirectory(Decimal)

flang/lib/CMakeLists.txt

@@ -1,10 +1,3 @@
-#===-- lib/Common/CMakeLists.txt -------------------------------------------===#
-#
-# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-# See https://llvm.org/LICENSE.txt for license information.
-# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-#
-#===------------------------------------------------------------------------===#
 
 add_library(FortranCommon
   Fortran.cpp
@@ -13,6 +6,8 @@ add_library(FortranCommon
   idioms.cpp
 )
 
+target_compile_features(FortranCommon PUBLIC cxx_std_17)
+
 install (TARGETS FortranCommon
   ARCHIVE DESTINATION lib
   LIBRARY DESTINATION lib

flang/lib/Common/CMakeLists.txt

@@ -1,16 +1,11 @@
-#===-- lib/Decimal/CMakeLists.txt ------------------------------------------===#
-#
-# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-# See https://llvm.org/LICENSE.txt for license information.
-# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-#
-#===------------------------------------------------------------------------===#
 
 add_library(FortranDecimal
   binary-to-decimal.cpp
   decimal-to-binary.cpp
 )
 
+target_compile_features(FortranDecimal PUBLIC cxx_std_17)
+
 install (TARGETS FortranDecimal
   ARCHIVE DESTINATION lib
   LIBRARY DESTINATION lib

flang/lib/Decimal/CMakeLists.txt

@@ -1,10 +1,3 @@
-#===-- lib/Evaluate/CMakeLists.txt -----------------------------------------===#
-#
-# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-# See https://llvm.org/LICENSE.txt for license information.
-# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-#
-#===------------------------------------------------------------------------===#
 
 add_library(FortranEvaluate
   call.cpp
@@ -34,6 +27,8 @@ add_library(FortranEvaluate
   variable.cpp
 )
 
+target_compile_features(FortranEvaluate PUBLIC cxx_std_17)
+
 target_link_libraries(FortranEvaluate
   FortranCommon
   FortranDecimal

flang/lib/Evaluate/CMakeLists.txt

@@ -1,10 +1,3 @@
-#===-- lib/Parser/CMakeLists.txt -------------------------------------------===#
-#
-# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-# See https://llvm.org/LICENSE.txt for license information.
-# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-#
-#===------------------------------------------------------------------------===#
 
 add_library(FortranParser
   Fortran-parsers.cpp
@@ -32,6 +25,8 @@ add_library(FortranParser
   user-state.cpp
 )
 
+target_compile_features(FortranParser PRIVATE cxx_std_17)
+
 target_link_libraries(FortranParser
   FortranCommon
   LLVMSupport

flang/lib/Parser/CMakeLists.txt

@@ -1,10 +1,3 @@
-#===-- lib/Semantics/CMakeLists.txt ----------------------------------------===#
-#
-# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-# See https://llvm.org/LICENSE.txt for license information.
-# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-#
-#===------------------------------------------------------------------------===#
 
 add_library(FortranSemantics
   assignment.cpp
@@ -43,6 +36,8 @@ add_library(FortranSemantics
   unparse-with-symbols.cpp
 )
 
+target_compile_features(FortranSemantics PUBLIC cxx_std_17)
+
 target_link_libraries(FortranSemantics
   FortranCommon
   FortranEvaluate