APPLE-INTERNAL: Revert INSTR_PROF_*_VERSION to 1400/1316/1300 format

For more details, read the comments on rdar://84254277.

This revert is unfortunately very large. I've squashed everything
together to make it easier to see what's going on. This will be the
last time we have to do this though. :)

To make the coverage format compatible, we need, in InstrProfData.inc:

* `INSTR_PROF_RAW_VERSION 5`
* `INSTR_PROF_INDEX_VERSION 7`
* `INSTR_PROF_COVMAP_VERSION 4`

Prior to this revert we have

* `INSTR_PROF_RAW_VERSION 8`
* `INSTR_PROF_INDEX_VERSION 8`
* `INSTR_PROF_COVMAP_VERSION 5`

So, this revert includes every commit which directly impacts these
values *and* any changes in-between that

* Refactor code significantly
* Make assumptions about the new format.

This might seem like a bit of a sledgehammer. However, say, avoiding
reverting NFC refactors would introduce tricky merge conflicts. Fixing
those merge conflicts is likely to introduce bugs.

Testing:
* `ninja check-all` does not exhibit any new failures with this change.
* `llvm-profdata show` can handle raw profiles produced by building the
  LLVM test suite with clang-1316. Output is identical.
* `llvm-profdata merge` handles raw profiles produced with the 1316 format
  on the LLVM test suite with the same output as the 1316 `llvm-profdata`.

 Conflicts:
	clang/include/clang/Basic/CodeGenOptions.h
	clang/include/clang/Driver/Options.td
	clang/lib/CodeGen/BackendUtil.cpp
	clang/lib/CodeGen/CoverageMappingGen.cpp
	clang/lib/Driver/ToolChains/Clang.cpp
	clang/test/CoverageMapping/abspath.cpp
	clang/test/Profile/profile-prefix-map.c
	compiler-rt/include/profile/MemProfData.inc
	compiler-rt/lib/memprof/memprof_rawprofile.cpp
	compiler-rt/lib/profile/InstrProfilingInternal.h
	compiler-rt/lib/profile/InstrProfilingPlatformLinux.c
	compiler-rt/test/profile/Darwin/instrprof-debug-info-correlate.c
	compiler-rt/test/profile/Linux/binary-id.c
	compiler-rt/test/profile/Linux/instrprof-debug-info-correlate.c
	llvm/docs/CommandGuide/llvm-profdata.rst
	llvm/include/llvm/ProfileData/Coverage/CoverageMappingReader.h
	llvm/include/llvm/ProfileData/InstrProfCorrelator.h
	llvm/include/llvm/ProfileData/MemProf.h
	llvm/include/llvm/ProfileData/MemProfData.inc
	llvm/include/llvm/ProfileData/RawMemProfReader.h
	llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp
	llvm/lib/ProfileData/InstrProf.cpp
	llvm/lib/ProfileData/InstrProfCorrelator.cpp
	llvm/lib/ProfileData/InstrProfReader.cpp
	llvm/lib/ProfileData/RawMemProfReader.cpp
	llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp
	llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp
	llvm/test/Instrumentation/InstrProfiling/debug-info-correlate-coverage.ll
	llvm/test/Instrumentation/InstrProfiling/debug-info-correlate.ll
	llvm/test/Instrumentation/InstrProfiling/profiling.ll
	llvm/test/tools/llvm-profdata/Inputs/basic.memprofraw
	llvm/test/tools/llvm-profdata/Inputs/c-general.profraw
	llvm/test/tools/llvm-profdata/Inputs/multi.memprofraw
	llvm/test/tools/llvm-profdata/large-binary-id-size.test
	llvm/test/tools/llvm-profdata/malformed-not-space-for-another-header.test
	llvm/test/tools/llvm-profdata/malformed-num-counters-zero.test
	llvm/test/tools/llvm-profdata/malformed-ptr-to-counter-array.test
	llvm/test/tools/llvm-profdata/memprof-basic.test
	llvm/test/tools/llvm-profdata/memprof-multi.test
	llvm/unittests/ProfileData/CoverageMappingTest.cpp
	llvm/unittests/ProfileData/MemProfTest.cpp
	compiler-rt/lib/memprof/memprof_allocator.cpp

Cherry-picks:
* 703a0ea Revert "[Coverage] Store compilation dir separately in coverage mapping"

New (non-revert, non-cherry-picked) changes:
* Update icall-nocomdat.ll to work with old coverage format
* Remove coverage mapping directory from `ScanAndUpdateArgs`
* Update profiling.ll for old profiling/coverage format
* Regenerate `coverage_prefix_map` test for old coverage format
* CoverageFilenamesSectionWriter uses `ArrayRef<std::string>` for compatability with Swift's GenCoverage.cpp.

Reverts:
* 6dd6a61 [memprof] Deduplicate and outline frame storage in the memprof profile.
* 14415a3 "Apply clang-tidy fixes for readability-redundant-smartptr-get in InstrProfReader.cpp (NFC)"
* 0ca8ff4 "[llvm-profdata] Unify default cutoffs for detailed summary printing".
* 27a4f25 "Reland "[memprof] Store callsite metadata with memprof records.""
* f4b7944 "Revert "[memprof] Store callsite metadata with memprof records.""
* 0d362c9 "[memprof] Store callsite metadata with memprof records."
* 9c35303 "[InstrProf][NFC] Fix warning by removing typecast"
* fc97efa "Cleanup includes: ProfileData"
* b681799 "[instrprof] Rename the profile kind types to be more descriptive."
* 0a41849 "Reland "[memprof] Extend the index prof format to include memory profiles.""
* 19bdf44 "Revert "Reland "[memprof] Extend the index prof format to include memory profiles."""
* 807ba7a "Reland "[memprof] Extend the index prof format to include memory profiles.""
* a3beb34 "Reland "[InstrProf] Make the IndexedInstrProf header backwards compatible.""
* 9fd2cb2 "Revert "[InstrProf] Make the IndexedInstrProf header backwards compatible.""
* 85355a5 "Revert "Reland "[memprof] Extend the index prof format to include memory profiles."""
* de54e4a "Reland "[memprof] Extend the index prof format to include memory profiles.""
* 34a62f9 "[llvm-profdata] Fix use-after-move"
* 0f73fb1 "Revert "[memprof] Extend the index prof format to include memory profiles.""
* 43c2348 "[memprof] Extend the index prof format to include memory profiles."
* 14cc41a "[InstrProf] Make the IndexedInstrProf header backwards compatible."
* 216575e "Revert "Revert "[ProfileData] Read and symbolize raw memprof profiles."""
* dbf47d2 "Revert "[ProfileData] Read and symbolize raw memprof profiles.""
* 26f978d "[ProfileData] Read and symbolize raw memprof profiles."
* 14f4f63 "[memprof] Print out the summary in YAML format."
* d2df8d5 "[instrprof][NFC] Templatize the instrprof iterator."
* 7756b34 "[InstrProf][NFC] Remove stray option in InstrProfWriter"
* 186dcd4 "[instrprof][NFC] Refactor out the common logic for getProfileKind."
* eea002a "[InstrProf][NFC] Move function out of InstrProf.h"
* 11d3074 "[InstrProf] Add single byte coverage mode"
* 13d8947 "[InstrProf][NFC] Refactor Profile kind into a bitset enum."
* c9baa56 "[InstrProf][Correlate] Verify debug info with llvm-profdata show"
* 6d52391 "[InstrProf][Correlate] Improve error messages"
* f170595 "[InstrProf][Correlator] Do not compress names when reading debug info"
* ccb09a4 "Fix broken comment in InstrProfData.inc"
* 88d8177 "[InstrProf] Restore InstrProfData.inc to fix Fuchsia builds"
* f214737 "[InstrProf][NFC] Do not assume size of counter type"
* 4ecf15b "[llvm-profdata] Make -debug-info visible"
* ac719d7 "[InstrProf] Don't profile merge by default in lightweight mode"
* 65d7fd0 "[Try2][InstrProf] Add Correlator class to read debug info"
* bdc68ee "Revert "[InstrProf] Add Correlator class to read debug info""
* 95946d2 "[InstrProf] Add Correlator class to read debug info"
* 58d9c1a "[Try2][InstrProf] Attach debug info to counters"
* c809da7 "Revert "[InstrProf] Attach debug info to counters""
* 800bf8e "[InstrProf] Attach debug info to counters"
* 7cca33b "[memprof] Extend llvm-profdata to display MemProf profile summaries."
* 24c615f "[InstrProfData] Bump the raw profile version to 8"
* ee88b8d "[compiler-rt] Add more diagnostic to InstrProfError"
* 743f78e "[InstrProfiling] Fix warnings when building for Windows"
* a1532ed "[InstrProfiling] Make CountersPtr in __profd_ relative"
* b9f547e "[llvm][profile] Add padding after binary IDs"
* 1df7289 "[compiler-rt/profile] Include __llvm_profile_get_magic in module signature"
* 83302c8 "[profile] Fix profile merging with binary IDs"
* c579c65 "[compiler-rt][profile] Make corrupted-profile.c more robust"
* f984ac2 "[profile] Add binary id into profiles"
* f0a616efa42b33cd280d5a45c3c2e3aabe278bb5 "Revert "[llvm-profdata] Fix use-after-move""
* 4578233ce8250c206bb342e85632b1d2969039c5 "Revert "[instrprof][NFC] Templatize the instrprof iterator.""
* e8d4a36dc9ea5719c3bea227d3fde15a005f7443 "Revert "[llvm-profdata] Unify default cutoffs for detailed summary printing""
* 440d971 "Work around non-existence of ElfW(type) macro on FreeBSD"
* 8306968 "[memprof] Move the meminfo block struct to MemProfData.inc."
* a2ce97c "[memprof] Fix unit test build after refactoring shared header."
* f89319b "Reland "[memprof] Refactor out the MemInfoBlock into a macro based def.""

Author: Jessica Paquette

clang/include/clang/Basic/CodeGenOptions.h

@@ -170,9 +170,6 @@ class CodeGenOptions : public CodeGenOptionsBase {
   /// The string to embed in debug information as the current working directory.
   std::string DebugCompilationDir;
 
-  /// The string to embed in coverage mapping as the current working directory.
-  std::string CoverageCompilationDir;
-
   /// The string to embed in the debug information for the compile unit, if
   /// non-empty.
   std::string DwarfDebugFlags;

clang/include/clang/Driver/Options.td

@@ -1246,10 +1246,6 @@ def fdebug_compilation_dir_EQ : Joined<["-"], "fdebug-compilation-dir=">,
 def fdebug_compilation_dir : Separate<["-"], "fdebug-compilation-dir">,
     Group<f_Group>, Flags<[CC1Option, CC1AsOption, CoreOption]>,
     Alias<fdebug_compilation_dir_EQ>;
-def fcoverage_compilation_dir_EQ : Joined<["-"], "fcoverage-compilation-dir=">,
-    Group<f_Group>, Flags<[CC1Option, CC1AsOption, CoreOption]>,
-    HelpText<"The compilation directory to embed in the coverage mapping.">,
-    MarshallingInfoString<CodeGenOpts<"CoverageCompilationDir">>;
 def ffile_compilation_dir_EQ : Joined<["-"], "ffile-compilation-dir=">, Group<f_Group>,
     Flags<[CoreOption]>,
     HelpText<"The compilation directory to embed in the debug info and coverage mapping.">;

clang/lib/CodeGen/BackendUtil.cpp

@@ -96,16 +96,10 @@ using namespace llvm;
   llvm::PassPluginLibraryInfo get##Ext##PluginInfo();
 #include "llvm/Support/Extension.def"
 
-namespace llvm {
-extern cl::opt<bool> DebugInfoCorrelate;
-}
-
 namespace {
 
 // Default filename used for profile generation.
-std::string getDefaultProfileGenName() {
-  return DebugInfoCorrelate ? "default_%p.proflite" : "default_%m.profraw";
-}
+static constexpr StringLiteral DefaultProfileGenName = "default_%m.profraw";
 
 class EmitAssemblyHelper {
   DiagnosticsEngine &Diags;
@@ -707,7 +701,7 @@ void EmitAssemblyHelper::RunOptimizationPipeline(
   if (CodeGenOpts.hasProfileIRInstr())
     // -fprofile-generate.
     PGOOpt = PGOOptions(CodeGenOpts.InstrProfileOutput.empty()
-                            ? getDefaultProfileGenName()
+                            ? std::string(DefaultProfileGenName)
                             : CodeGenOpts.InstrProfileOutput,
                         "", "", PGOOptions::IRInstr, PGOOptions::NoCSAction,
                         CodeGenOpts.DebugInfoForProfiling);
@@ -745,13 +739,13 @@ void EmitAssemblyHelper::RunOptimizationPipeline(
              "Cannot run CSProfileGen pass with ProfileGen or SampleUse "
              " pass");
       PGOOpt->CSProfileGenFile = CodeGenOpts.InstrProfileOutput.empty()
-                                     ? getDefaultProfileGenName()
+                                     ? std::string(DefaultProfileGenName)
                                      : CodeGenOpts.InstrProfileOutput;
       PGOOpt->CSAction = PGOOptions::CSIRInstr;
     } else
       PGOOpt = PGOOptions("",
                           CodeGenOpts.InstrProfileOutput.empty()
-                              ? getDefaultProfileGenName()
+                              ? std::string(DefaultProfileGenName)
                               : CodeGenOpts.InstrProfileOutput,
                           "", PGOOptions::NoAction, PGOOptions::CSIRInstr,
                           CodeGenOpts.DebugInfoForProfiling);

clang/lib/CodeGen/CoverageMappingGen.cpp

@@ -1561,17 +1561,9 @@ CoverageMappingModuleGen::CoverageMappingModuleGen(
   CoveragePrefixMap = CGM.getCodeGenOpts().CoveragePrefixMap;
 }
 
-std::string CoverageMappingModuleGen::getCurrentDirname() {
-  if (!CGM.getCodeGenOpts().CoverageCompilationDir.empty())
-    return CGM.getCodeGenOpts().CoverageCompilationDir;
-
-  SmallString<256> CWD;
-  llvm::sys::fs::current_path(CWD);
-  return CWD.str().str();
-}
-
 std::string CoverageMappingModuleGen::normalizeFilename(StringRef Filename) {
   llvm::SmallString<256> Path(Filename);
+  llvm::sys::fs::make_absolute(Path);
   llvm::sys::path::remove_dots(Path, /*remove_dot_dot=*/true);
   for (const auto &Entry : CoveragePrefixMap) {
     if (llvm::sys::path::replace_path_prefix(Path, Entry.first, Entry.second))
@@ -1652,17 +1644,18 @@ void CoverageMappingModuleGen::addFunctionMappingRecord(
     // also processed by the CoverageMappingWriter which performs
     // additional minimization operations such as reducing the number of
     // expressions.
-    llvm::SmallVector<std::string, 16> FilenameStrs;
     std::vector<StringRef> Filenames;
     std::vector<CounterExpression> Expressions;
     std::vector<CounterMappingRegion> Regions;
-    FilenameStrs.resize(FileEntries.size() + 1);
-    FilenameStrs[0] = normalizeFilename(getCurrentDirname());
+    llvm::SmallVector<std::string, 16> FilenameStrs;
+    llvm::SmallVector<StringRef, 16> FilenameRefs;
+    FilenameStrs.resize(FileEntries.size());
+    FilenameRefs.resize(FileEntries.size());
     for (const auto &Entry : FileEntries) {
       auto I = Entry.second;
       FilenameStrs[I] = normalizeFilename(Entry.first->getName());
+      FilenameRefs[I] = FilenameStrs[I];
     }
-    ArrayRef<std::string> FilenameRefs = llvm::makeArrayRef(FilenameStrs);
     RawCoverageMappingReader Reader(CoverageMapping, FilenameRefs, Filenames,
                                     Expressions, Regions);
     if (Reader.read())
@@ -1679,17 +1672,20 @@ void CoverageMappingModuleGen::emit() {
 
   // Create the filenames and merge them with coverage mappings
   llvm::SmallVector<std::string, 16> FilenameStrs;
-  FilenameStrs.resize(FileEntries.size() + 1);
-  // The first filename is the current working directory.
-  FilenameStrs[0] = normalizeFilename(getCurrentDirname());
+  llvm::SmallVector<StringRef, 16> FilenameRefs;
+  FilenameStrs.resize(FileEntries.size());
+  FilenameRefs.resize(FileEntries.size());
   for (const auto &Entry : FileEntries) {
     auto I = Entry.second;
     FilenameStrs[I] = normalizeFilename(Entry.first->getName());
+    FilenameRefs[I] = FilenameStrs[I];
   }
 
   std::string Filenames;
   {
     llvm::raw_string_ostream OS(Filenames);
+    // APPLE-INTERNAL: Using FilenameStrs for compatability with Swift's
+    // GenCoverage.cpp.
     CoverageFilenamesSectionWriter(FilenameStrs).write(OS);
   }
   auto *FilenamesVal =
@@ -1748,7 +1744,7 @@ unsigned CoverageMappingModuleGen::getFileID(const FileEntry *File) {
   auto It = FileEntries.find(File);
   if (It != FileEntries.end())
     return It->second;
-  unsigned FileID = FileEntries.size() + 1;
+  unsigned FileID = FileEntries.size();
   FileEntries.insert(std::make_pair(File, FileID));
   return FileID;
 }

clang/lib/CodeGen/CoverageMappingGen.h

@@ -95,7 +95,6 @@ class CoverageMappingModuleGen {
   std::vector<FunctionInfo> FunctionRecords;
   std::map<std::string, std::string> CoveragePrefixMap;
 
-  std::string getCurrentDirname();
   std::string normalizeFilename(StringRef Filename);
 
   /// Emit a function record.

clang/lib/Driver/ToolChains/Clang.cpp

@@ -900,18 +900,6 @@ static void addPGOAndCoverageFlags(const ToolChain &TC, Compilation &C,
     CmdArgs.push_back("-fcoverage-mapping");
   }
 
-  if (Arg *A = Args.getLastArg(options::OPT_ffile_compilation_dir_EQ,
-                               options::OPT_fcoverage_compilation_dir_EQ)) {
-    if (A->getOption().matches(options::OPT_ffile_compilation_dir_EQ))
-      CmdArgs.push_back(Args.MakeArgString(
-          Twine("-fcoverage-compilation-dir=") + A->getValue()));
-    else
-      A->render(Args, CmdArgs);
-  } else if (llvm::ErrorOr<std::string> CWD =
-                 D.getVFS().getCurrentWorkingDirectory()) {
-    CmdArgs.push_back(Args.MakeArgString("-fcoverage-compilation-dir=" + *CWD));
-  }
-
   if (Args.hasArg(options::OPT_fprofile_exclude_files_EQ)) {
     auto *Arg = Args.getLastArg(options::OPT_fprofile_exclude_files_EQ);
     if (!Args.hasArg(options::OPT_coverage))

clang/lib/Tooling/DependencyScanning/ScanAndUpdateArgs.cpp

@@ -191,7 +191,6 @@ static void updateCompilerInvocation(CompilerInvocation &Invocation,
   // Code generation options.
   auto &CodeGenOpts = Invocation.getCodeGenOpts();
   Mapper.mapInPlaceOrClear(CodeGenOpts.DebugCompilationDir);
-  Mapper.mapInPlaceOrClear(CodeGenOpts.CoverageCompilationDir);
 }
 
 Expected<llvm::cas::CASID> clang::scanAndUpdateCC1InlineWithTool(

clang/test/CodeGen/coverage-compilation-dir.c

@@ -1,3 +1,6 @@
+// XFAIL: *
+// See rdar://82543962.
+//
 // RUN: mkdir -p %t.dir && cd %t.dir
 // RUN: cp %s rel.c
 // RUN: %clang_cc1 -fprofile-instrument=clang -fcoverage-compilation-dir=/nonsense -fcoverage-mapping -emit-llvm -mllvm -enable-name-compression=false rel.c -o - | FileCheck -check-prefix=CHECK-NONSENSE %s

clang/test/CoverageMapping/abspath.cpp

@@ -1,19 +1,15 @@
 // RUN: %clang_cc1 -mllvm -emptyline-comment-coverage=false -fprofile-instrument=clang -fcoverage-mapping -mllvm -enable-name-compression=false -emit-llvm -main-file-name abspath.cpp %S/Inputs/../abspath.cpp -o - | FileCheck -check-prefix=RMDOTS %s
 
-// RMDOTS: @__llvm_coverage_mapping = {{.*}}"\02
+// RMDOTS: @__llvm_coverage_mapping = {{.*}}"\01
 // RMDOTS-NOT: Inputs
 // RMDOTS: "
 
 // RUN: mkdir -p %t/test && cd %t/test
 // RUN: echo "void f1(void) {}" > f1.c
-// RUN: %clang_cc1 -mllvm -emptyline-comment-coverage=false -fprofile-instrument=clang -fcoverage-mapping -mllvm -enable-name-compression=false -emit-llvm -main-file-name abspath.cpp %t/test/f1.c -o - | FileCheck -check-prefix=ABSPATH %s
+// RUN: %clang_cc1 -mllvm -emptyline-comment-coverage=false -fprofile-instrument=clang -fcoverage-mapping -mllvm -enable-name-compression=false -emit-llvm -main-file-name abspath.cpp ../test/f1.c -o - | FileCheck -check-prefix=RELPATH %s
 
-// RELPATH: @__llvm_coverage_mapping = {{.*}}"\02
-// RELPATH: {{..(/|\\\\)test(/|\\\\)f1}}.c
+// RELPATH: @__llvm_coverage_mapping = {{.*}}"\01
+// RELPATH: {{[/\\].*(/|\\\\)test(/|\\\\)f1}}.c
 // RELPATH: "
 
-// ABSPATH: @__llvm_coverage_mapping = {{.*}}"\02
-// ABSPATH: {{[/\\].*(/|\\\\)test(/|\\\\)f1}}.c
-// ABSPATH: "
-
 void f1(void) {}

clang/test/Driver/clang_f_opts.c

@@ -522,10 +522,11 @@
 // CHECK-DEBUG-COMPILATION-DIR: "-fdebug-compilation-dir=."
 // CHECK-DEBUG-COMPILATION-DIR-NOT: "-ffile-compilation-dir=."
 
-// RUN: %clang -### -S -fprofile-instr-generate -fcoverage-compilation-dir=. %s 2>&1 | FileCheck -check-prefix=CHECK-COVERAGE-COMPILATION-DIR %s
-// RUN: %clang -### -S -fprofile-instr-generate -ffile-compilation-dir=. %s 2>&1 | FileCheck -check-prefix=CHECK-COVERAGE-COMPILATION-DIR %s
-// CHECK-COVERAGE-COMPILATION-DIR: "-fcoverage-compilation-dir=."
-// CHECK-COVERAGE-COMPILATION-DIR-NOT: "-ffile-compilation-dir=."
+// See rdar://82543962.
+// XXX: %clang -### -S -fprofile-instr-generate -fcoverage-compilation-dir=. %s 2>&1 | FileCheck -check-prefix=CHECK-COVERAGE-COMPILATION-DIR %s
+// XXX: %clang -### -S -fprofile-instr-generate -ffile-compilation-dir=. %s 2>&1 | FileCheck -check-prefix=CHECK-COVERAGE-COMPILATION-DIR %s
+// XXX-COVERAGE-COMPILATION-DIR: "-fcoverage-compilation-dir=."
+// XXX-COVERAGE-COMPILATION-DIR-NOT: "-ffile-compilation-dir=."
 
 // RUN: %clang -### -S -fdiscard-value-names %s 2>&1 | FileCheck -check-prefix=CHECK-DISCARD-NAMES %s
 // RUN: %clang -### -S -fno-discard-value-names %s 2>&1 | FileCheck -check-prefix=CHECK-NO-DISCARD-NAMES %s