build the toochain failed when doing cross-compilation on M1 MBP

[rayxia]

Describe the bug
I use utils/build-toolchain com.custom command to build the toolchain, and get the error says as following:

Undefined symbols for architecture arm64:
  "llvm::cl::Option::anchor()", referenced from:
      vtable for llvm::cl::list<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > in swift-demangle.cpp.o
  "llvm::cl::Option::addOccurrence(unsigned int, llvm::StringRef, llvm::StringRef, bool)", referenced from:
      vtable for llvm::cl::list<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > in swift-demangle.cpp.o
  "llvm::cl::GenericOptionValue::anchor()", referenced from:
      vtable for llvm::cl::OptionValueCopy<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > in swift-demangle.cpp.o
      vtable for llvm::cl::OptionValue<bool> in swift-demangle.cpp.o
  "vtable for llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >", referenced from:
      __GLOBAL__sub_I_swift_demangle.cpp in swift-demangle.cpp.o
  NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
  "llvm::cl::Option::addArgument()", referenced from:
      __GLOBAL__sub_I_swift_demangle.cpp in swift-demangle.cpp.o
  "vtable for llvm::cl::OptionValue<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >", referenced from:
      __GLOBAL__sub_I_swift_demangle.cpp in swift-demangle.cpp.o
  NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
  "llvm::cl::Option::setArgStr(llvm::StringRef)", referenced from:
      __GLOBAL__sub_I_swift_demangle.cpp in swift-demangle.cpp.o
  "llvm::SmallVectorBase<unsigned int>::grow_pod(void*, unsigned long, unsigned long)", referenced from:
      llvm::cl::Option::Option(llvm::cl::NumOccurrencesFlag, llvm::cl::OptionHidden) in swift-demangle.cpp.o
  "vtable for llvm::cl::opt<bool, false, llvm::cl::parser<bool> >", referenced from:
      llvm::cl::opt<bool, false, llvm::cl::parser<bool> >::~opt() in swift-demangle.cpp.o
      __GLOBAL__sub_I_swift_demangle.cpp in swift-demangle.cpp.o
  NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
  "llvm::cl::basic_parser_impl::getOptionWidth(llvm::cl::Option const&) const", referenced from:
      llvm::cl::list<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >::getOptionWidth() const in swift-demangle.cpp.o
  "llvm::cl::ParseCommandLineOptions(int, char const* const*, llvm::StringRef, llvm::raw_ostream*, char const*, bool)", referenced from:
      _main in swift-demangle.cpp.o
  "llvm::raw_ostream::write(unsigned char)", referenced from:
      _main in swift-demangle.cpp.o
      demangle(llvm::raw_ostream&, llvm::StringRef, swift::Demangle::Context&, swift::Demangle::DemangleOptions const&) in swift-demangle.cpp.o
      llvm::raw_ostream::operator<<(char) in swift-demangle.cpp.o
  "vtable for llvm::cl::parser<bool>", referenced from:
      __GLOBAL__sub_I_swift_demangle.cpp in swift-demangle.cpp.o
  NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
  "llvm::cl::getGeneralCategory()", referenced from:
      llvm::cl::Option::Option(llvm::cl::NumOccurrencesFlag, llvm::cl::OptionHidden) in swift-demangle.cpp.o
  "llvm::cl::basic_parser_impl::printOptionInfo(llvm::cl::Option const&, unsigned long) const", referenced from:
      llvm::cl::list<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >::printOptionInfo(unsigned long) const in swift-demangle.cpp.o
  "llvm::outs()", referenced from:
      _main in swift-demangle.cpp.o
      demangle(llvm::raw_ostream&, llvm::StringRef, swift::Demangle::Context&, swift::Demangle::DemangleOptions const&) in swift-demangle.cpp.o
  "llvm::errs()", referenced from:
      demangle(llvm::raw_ostream&, llvm::StringRef, swift::Demangle::Context&, swift::Demangle::DemangleOptions const&) in swift-demangle.cpp.o
  "llvm::raw_ostream::write(char const*, unsigned long)", referenced from:
      _main in swift-demangle.cpp.o
      demangle(llvm::raw_ostream&, llvm::StringRef, swift::Demangle::Context&, swift::Demangle::DemangleOptions const&) in swift-demangle.cpp.o
      llvm::raw_ostream::operator<<(llvm::StringRef) in swift-demangle.cpp.o
      llvm::raw_ostream::operator<<(char const*) in swift-demangle.cpp.o
      llvm::raw_ostream::operator<<(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in swift-demangle.cpp.o
  "vtable for llvm::cl::opt<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, false, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >", referenced from:
      llvm::cl::opt<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, false, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >::~opt() in swift-demangle.cpp.o
      __GLOBAL__sub_I_swift_demangle.cpp in swift-demangle.cpp.o
  NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
  "llvm::raw_ostream::operator<<(long)", referenced from:
      llvm::raw_ostream::operator<<(int) in swift-demangle.cpp.o
  "llvm::raw_ostream::operator<<(unsigned long)", referenced from:
      llvm::raw_ostream::operator<<(unsigned int) in swift-demangle.cpp.o
  "vtable for llvm::cl::Option", referenced from:
      llvm::cl::opt<bool, false, llvm::cl::parser<bool> >::~opt() in swift-demangle.cpp.o
      llvm::cl::opt<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, false, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >::~opt() in swift-demangle.cpp.o
      llvm::cl::list<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >::~list() in swift-demangle.cpp.o
      llvm::cl::Option::Option(llvm::cl::NumOccurrencesFlag, llvm::cl::OptionHidden) in swift-demangle.cpp.o
  NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

As the error says, the reason is that when linking the swift-demangle.cpp.o, there is no symbols for arm64, and I notice there is another warning:

ld: warning: ignoring file /Users/rayxia/Workspace/opensource/swift-project-youku/build/buildbot_osx/llvm-macosx-x86_64/lib/libLLVMSupport.a, building for macOS-arm64 but attempting to link with file built for macOS-x86_64

But the strenge thing is, I use lipo -info libLLVMSupport.a to check, its arch is x86_64.

Environment

• OS: [macOS 12.4]
• Xcode: 13.4.1 (13F100)
• Swift Branch: swift-5.6.1-RELEASE (2a79777)

[whf881211]

i had the same issue, also on M1

[rayxia]

i had the same issue, also on M1

@whf881211 have you solved this issue?

[whf881211]

nope. I'm going to try to build it on intel mac. That might be less problem.

[rayxia]

nope. I'm going to try to build it on intel mac. That might be less problem.

@whf881211 I have tried to build it on intel, still failed.

[azarovalex]

I didn't face this exact issue while building the toolchain, I even managed to build it today on M1 with the utils/build-toolchain script on commit 983e2f37d35.

However, for a successful build, you still need a lot of hacks in scripts, starting from this hack here because of the line here (recent PR doesn't seem to help). There are other dirty tricks that I won't even show for the sake of sanity.

From other folks, I heard that running this script under rosetta helps as well, but still requires some tweaks.
It's very sad to see that it's been years without a proper arm64 host support for building the toolchain, the situation with the compiler is better but still isn't perfect.

@edymtt I wonder if is it possible to add an arm64 host that builds the toolchain to ci.swift.org, it would make the life of a lot of devs on M1 macs much easier.

[edymtt]

With regard to the original issue, this may be happening because one or more programs used/involved in the compilation are running under Rosetta -- I would suggest to check the python used to run build-script, the shell you are running on and the homebrew you are running (if any)

With regard to hitting issue with Intel as well, I would suggest to file a new issue and providing a log (or at least a snippet of the error message) to understand what may be going on there
Comparing the local log with a successful run from https://ci.swift.org/job/oss-swift-package-macos/ may help as well.

[edymtt]

@azarovalex I will check what we can do here (no promises, since I'm not responsible for the CI infrastructure)