Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cross-compile for Android meets linker error #138488

Closed
huangzhengxiang opened this issue Mar 14, 2025 · 15 comments
Closed

Cross-compile for Android meets linker error #138488

huangzhengxiang opened this issue Mar 14, 2025 · 15 comments
Labels
C-bug Category: This is a bug. needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. O-android Operating system: Android

Comments

@huangzhengxiang
Copy link

When I tried to cross-compile a rust project for android, I met the following linker problem.

cargo automatically add lots of linker flags when cross compiling the dependencies: "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc".
These linker flags are not available for ndk clang compiler I suppose, leading to linker error. Actually these automatically added linker flags are not even required by the dependencies.

= note: ld.lld: error: unable to find library -lgcc_s
          ld.lld: error: unable to find library -lutil
          ld.lld: error: unable to find library -lrt
          ld.lld: error: unable to find library -lpthread
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

How to fix it? Is it a rust issue for android cross-compilation? Is it possible to disable such default linker flags?

To replicate such error, just cross compile any rust project which has a dependency library, and it may show up.

Meta

I've tested all possible version for aarch64-linux-android, including
rustc --version --verbose:

<version> 1.84, 1.82, 1.78, 1.75, 1.70

They all present the same linker error above.

Backtrace

<backtrace>

@huangzhengxiang huangzhengxiang added the C-bug Category: This is a bug. label Mar 14, 2025
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Mar 14, 2025
@bjorn3
Copy link
Member

bjorn3 commented Mar 14, 2025

Did you tell rustc to use the linker from the NDK? What cargo command did you use build?

@bjorn3 bjorn3 added the O-android Operating system: Android label Mar 14, 2025
@huangzhengxiang
Copy link
Author

huangzhengxiang commented Mar 18, 2025

@bjorn3
yes, i have specified the linker and rustc and cargo seems to found the correct linker.

export PATH=${PATH}:${ANDROID_NDK}/toolchains/llvm/prebuilt/linux-x86_64/bin/
export CC_aarch64_linux_android=${ANDROID_NDK}/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android24-clang
export CARGO_TARGET_AARCH64_LINUX_ANDROID_LINKER=${ANDROID_NDK}/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android24-clang

@huangzhengxiang
Copy link
Author

huangzhengxiang commented Mar 18, 2025

The linker is found correct. However, ndk doesn't contain pthread and libc by default unfortunately, and this leads to the problem.

@huangzhengxiang
Copy link
Author

My ndk is the official ndk.

@bjorn3
Copy link
Member

bjorn3 commented Mar 18, 2025

What --target did you pass to rustc or cargo?

However, ndk doesn't contain pthread and libc by default unfortunately

The ndk has to contain libc at the very least. Without it, it would be impossible to link any executable, which is kind of the whole point of the ndk.

@huangzhengxiang
Copy link
Author

--target aarch64-linux-android

@bjorn3
Copy link
Member

bjorn3 commented Mar 18, 2025

I think you need to set the linker using export RUSTFLAGS="-Clinker=${ANDROID_NDK}/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android24-clang".

@huangzhengxiang
Copy link
Author

I tried this line, and it does not help.

I think you need to set the linker using export RUSTFLAGS="-Clinker=${ANDROID_NDK}/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android24-clang".

@huangzhengxiang
Copy link
Author

huangzhengxiang commented Mar 18, 2025

The following error is still there.

= note: ld.lld: error: unable to find library -lgcc_s
          ld.lld: error: unable to find library -lutil
          ld.lld: error: unable to find library -lrt
          ld.lld: error: unable to find library -lpthread
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

For libc, it's my typo. The linker can find it. But for pthread, and the above libraries, they are indeed not in ndk.

@huangzhengxiang
Copy link
Author

The complete error message is below:

Caused by:
  process didn't exit successfully: `CARGO=/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/home/hzx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/proc-macro2-1.0.93 CARGO_PKG_AUTHORS='David Tolnay <dtolnay@gmail.com>:Alex Crichton <alex@alexcrichton.com>' CARGO_PKG_DESCRIPTION='A substitute implementation of the compiler'\''s `proc_macro` API to decouple token-based libraries from the procedural macro use case.' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=proc-macro2 CARGO_PKG_README=README.md CARGO_PKG_REPOSITORY='https://github.com/dtolnay/proc-macro2' CARGO_PKG_RUST_VERSION=1.56 CARGO_PKG_VERSION=1.0.93 CARGO_PKG_VERSION_MAJOR=1 CARGO_PKG_VERSION_MINOR=0 CARGO_PKG_VERSION_PATCH=93 CARGO_PKG_VERSION_PRE='' LD_LIBRARY_PATH='/home/hzx/Desktop/ANL/Project/AIoT-Demo/mlc-llm/3rdparty/tokenizers-cpp/build/release/deps:/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib:/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib' /home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/bin/rustc --crate-name build_script_build --edition=2021 /home/hzx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/proc-macro2-1.0.93/build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=128 --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg 'feature="default"' --cfg 'feature="proc-macro"' -C metadata=352279a8109e00d7 -C extra-filename=-352279a8109e00d7 --out-dir /home/hzx/Desktop/ANL/Project/AIoT-Demo/mlc-llm/3rdparty/tokenizers-cpp/build/release/build/proc-macro2-352279a8109e00d7 -L dependency=/home/hzx/Desktop/ANL/Project/AIoT-Demo/mlc-llm/3rdparty/tokenizers-cpp/build/release/deps --cap-lints warn` (exit status: 1)
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/hzx/miniconda3/envs/llm/bin:/home/hzx/.local/bin:/home/hzx/miniconda3/condabin:/home/hzx/.cargo/bin:/home/hzx/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:~:~/android-studio/bin:/toolchains/llvm/prebuilt/linux-x86_64/bin/:/home/hzx/Android/Sdk/ndk/27.0.12077973//toolchains/llvm/prebuilt/linux-x86_64/bin/" VSLANG="1033" "cc" "-m64" "/tmp/rustcqAYP3Q/symbols.o" "/home/hzx/Desktop/ANL/Project/AIoT-Demo/mlc-llm/3rdparty/tokenizers-cpp/build/release/build/onig_sys-c034aa793ef4d5ae/build_script_build-c034aa793ef4d5ae.build_script_build.4d87e40d698bdf63-cgu.0.rcgu.o" "/home/hzx/Desktop/ANL/Project/AIoT-Demo/mlc-llm/3rdparty/tokenizers-cpp/build/release/build/onig_sys-c034aa793ef4d5ae/build_script_build-c034aa793ef4d5ae.18s6piyq3cwg1zrv.rcgu.o" "-Wl,--as-needed" "-L" "/home/hzx/Desktop/ANL/Project/AIoT-Demo/mlc-llm/3rdparty/tokenizers-cpp/build/release/deps" "-L" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/hzx/Desktop/ANL/Project/AIoT-Demo/mlc-llm/3rdparty/tokenizers-cpp/build/release/deps/libcc-b80cff1ff090c24d.rlib" "/home/hzx/Desktop/ANL/Project/AIoT-Demo/mlc-llm/3rdparty/tokenizers-cpp/build/release/deps/libshlex-343167f769beb915.rlib" "/home/hzx/Desktop/ANL/Project/AIoT-Demo/mlc-llm/3rdparty/tokenizers-cpp/build/release/deps/libpkg_config-25b6a06fb70730f9.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-90f6ddbf82de36ec.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-480c41e9d4e1f677.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-db744c0ca03eed1d.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-8ba652dc7d4b285a.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-d5f4520a9ebc0d58.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-4ad84de58f0cb463.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-0d10aae2e0f38735.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-0c17cff739e6745b.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-b29e17139dde1aa8.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-1f975299829cc7bd.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-86bae3bc3079f89b.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-321b82c30dffdf5f.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-3dd8a6810a0bdfef.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-eb39a61c0c879984.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-aa769569f91c3548.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-510a192a50a983ed.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-0577018320f99037.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-193cf992125ccd4c.rlib" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-8e138eaf26ebb4a8.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/hzx/.rustup/toolchains/1.75-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/hzx/Desktop/ANL/Project/AIoT-Demo/mlc-llm/3rdparty/tokenizers-cpp/build/release/build/onig_sys-c034aa793ef4d5ae/build_script_build-c034aa793ef4d5ae" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
  = note: ld.lld: error: unable to find library -lgcc_s
          ld.lld: error: unable to find library -lutil
          ld.lld: error: unable to find library -lrt
          ld.lld: error: unable to find library -lpthread
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

@bjorn3
Copy link
Member

bjorn3 commented Mar 18, 2025

Wait, the crate that failed to build is getting compiled for the host as it is a build script. Are you able to compile any executable for your host at all? Do you have the build-essentials package installed?

@huangzhengxiang
Copy link
Author

huangzhengxiang commented Mar 18, 2025

Do you mean that my error comes from building process for the host?
Shall I specify linker for the host? As it may find the wrong one?
(My Ubuntu 22.04 host has gcc, make, cmake, and build-essentials installed already)

@bjorn3
Copy link
Member

bjorn3 commented Mar 18, 2025

Do you mean that my error comes from building process for the host?

Yes

Shall I specify linker for the host? As it may find the wrong one?

It looks like it found the right linker. I think you are missing the libc6-dev package (and maybe one of the libgcc-*-dev packages) to be able to link against libc at all. build-essentials should already depend on that package though.

@huangzhengxiang
Copy link
Author

I understood! It's my fault, I've manually set my system linker to the ndk linker.

@huangzhengxiang
Copy link
Author

Thank you very much! Wish you a good day!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: This is a bug. needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. O-android Operating system: Android
Projects
None yet
Development

No branches or pull requests

3 participants