Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial support for dynamically linked crates #134767

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Bryanskiy
Copy link
Contributor

@Bryanskiy Bryanskiy commented Dec 25, 2024

This PR is an initial implementation of rust-lang/rfcs#3435 proposal.

component 1: interface generator

Interface generator - a tool for generating a stripped version of crate source code. The interface is like a C header with only "exported" items included, and function bodies are omitted. For example, initial crate:

#[export]
#[repr(C)]
pub struct S {
   pub x: i32
}
#[export]
pub extern "C" fn foo(x: S) { 
   m1::bar(x);
}

pub fn bar(x: crate::S) { 
    // some computations 
}	

generated interface:

#[export]
#[repr(C)]
pub struct S {
    pub x: i32,
}

#[export]
pub extern "C" fn foo(x: S);

In this example bar function is not a part of the interface as it doesn't have #[export] attribute.
To emit interface use a new sdylib crate type which is basically the same as dylib, but it also produces the interface as a second artifact. The current interface name is lib{crate_name}.rs.

Interface generator was implemented as part of the HIR pretty-printer. In order to filter out unnecessary items, the PpAnn trait was used.

Why was it decided to use a design with an auto-generated interface?

One of the main objectives of this proposal is to allow building the library and the application with different compiler versions. This requires either a metadata format compatible across rustc versions or some form of a source code. The option with a stable metadata format was rejected because it is not a part of the RFC. (discussion)

Regarding the design with interfaces there are 2 possibilities: manually written or auto-generated. I would personally prefer the auto-generated interface for the following reason: we can put it in the dynamic library instead of metadata, which will make it completely invisible to users. (this was my initial plan, but since the PR is already big enough, I decided to postpone it)

But even if we end up with a different design, I believe the interface generator could be a useful tool for testing and experimenting with the entire feature.

component 2: crate loader

When building dynamic dependencies, the crate loader searches for the interface in the file system, builds the interface without codegen and loads it's metadata. For now, it's assumed that interface and dynamic lib are located in the same directory. extern dyn crate annotation serves as a signal for the building of a dynamic dependency.

Here are the code and commands that corresponds to the compilation process:

// simple-lib.rs
#![crate_type = "sdylib"]

#[extern]
pub extern "C" fn foo() -> i32 {
    42
}
// app.rs
extern dyn crate simple_lib;

fn main() {
    assert!(simple_lib::foo(), 42);
}
// Generate interface, build library.
rustc +toolchain1 lib.rs -Csymbol-mangling-version=v0

// Build app. Perhaps with a different compiler version.
rustc +toolchain2 app.rs -L. -Csymbol-mangling-version=v0

P.S. The interface name/format and rules for file system routing can be changed further.

component 3: exportable items collector

Query for collecting exportable items. Which items are exportable is defined here .

component 4: "stable" mangling scheme

The mangling scheme proposed in the RFC consists of two parts: a mangled item path and a hash of the signature.

mangled item path

For the first part of the symbol it has been decided to reuse the v0 mangling scheme as it is mostly independent of the compiler internals.

The first exception is an impl's disambiguation. During symbol mangling rustc uses a special index to distinguish between two impls of the same type in the same module(See DisambiguatedDefPathData). The calculation of this index may depend on private items, but private items should not affect the ABI. Example:

#[export]
#[repr(C)]
pub struct S<T>(pub T);

struct S1;
pub struct S2;

// This impl is not part of the interface.
impl S<S1> {
    extern "C" fn foo() -> i32 {
        1
    }
}

#[export]
impl S<S2> {
    // Different symbol names can be generated for this item
    // when compiling the interface and source code.
    pub extern "C" fn foo() -> i32 {
        2
    }
}

In order to make disambiguation independent of the compiler version we can assign an id to each impl according to their relative order in the source code. However, I have not found the right way to get this order, so I decided to use:

  1. The sequential number of an impl during the intravisit::Visitor traversal.
  2. A new attribute #[rustc_stable_impl_id] that outputs this sequential number as a compiler error. If the visitor's implementation is changed, the corresponding test will fail. Then you will need to rewrite the implementation of stable_order_of_exportable_impls query to preserve order.

P.S. is it possible to compare spans instead?

The second exception is StableCrateId which is used to disambiguate different crates. StableCrateId consists of crate name, -Cmetadata arguments and compiler version. At the moment, I have decided to keep only the crate name, but a more consistent approach to crate disambiguation could be added in the future.

hash of the signature

Second part of the symbol name is 128 bit hash containing relevant type information. For now, it includes:

  • hash of the type name for primitive types
  • for ADT types with public fields the implementation follows this rules

#[export(unsafe_stable_abi = "hash")] syntax for ADT types with private fields is not yet implemented.

@rustbot
Copy link
Collaborator

rustbot commented Dec 25, 2024

r? @oli-obk

rustbot has assigned @oli-obk.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added A-run-make Area: port run-make Makefiles to rmake.rs S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Dec 25, 2024
@Bryanskiy
Copy link
Contributor Author

cc @m-ou-se

@rust-log-analyzer

This comment has been minimized.

@bjorn3
Copy link
Member

bjorn3 commented Dec 26, 2024

However, in that case, we encounter a "disambiguation" issue:

I think that is indicative of a fundamental issue with your implementation. Adding new private impls should not change the ABI. The whole point of the RFC is to allow changing the implementation of the dylib without changing it's ABI. We already have support for dylibs where the ABI depends on the implementation.

@bjorn3
Copy link
Member

bjorn3 commented Dec 26, 2024

Also the interface file is missing either the StableCrateId or it's constituent parts (crate name and all -Cmetadata arguments). Without this it is impossible for the consumer to mangle symbols in the same way as the dylib itself.

Replacing all function bodies in an interface file with loop {} does not work in the presence of RPIT. And serializing it as a rust source file misses the expansion context which is essential for hygiene. I did much rather use a format like JSON of while we don't guarantee a stable ABI across rustc versions a binary format.

let interface_dir = interface.parent().unwrap();
let crate_name = crate_name.unwrap();

let res = std::process::Command::new(compiler)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need to be done either within the current compiler session or as a separate compiler session in the same process. As implemented this will break rustdoc and any other custom driver that has a different cli from rustc itself.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it a blocking issue? I don't see an easy way to do this. (I have left a FIXME comment at the moment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think not, but at the very least make sure to put the produced rmeta file in a temporary directory. As is you could have multiple rustc instances race trying to write an rmeta file and then read a partially written rmeta file.

Maybe an alternative to invoking rustc from within rustc would be to require the end user to compile interface files to rmeta files themself?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe an alternative to invoking rustc from within rustc would be to require the end user to compile interface files to rmeta files themself?

My concern is that in Rust, files generally do not have a special meaning, unlike C++. A translation unit i.e. a crate is not a single file, it consists of modules. Modules, in turn, can be declared either in one file or divided into several. That's why the "interface file" isn't a very coherent concept in Rust. I would like to avoid adding an additional level of complexity for users until it is proven necessary. Therefore, the initial plan was to make the interfaces completely invisible to users. I also planned to put them in the dylib, but this has not been done yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A translation unit i.e. a crate is not a single file, it consists of modules.

There is a single interface file for the entire crate, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but at the very least make sure to put the produced rmeta file in a temporary directory

Done

@bjorn3
Copy link
Member

bjorn3 commented Dec 26, 2024

component 4: stable ABI mangling scheme

A stable ABI across rustc versions is a lot more involved than just mangling all symbols the same. You have to make sure multiple copies of the standard library seamlessly interoperate including using the same TypeId for &str and String (and thus identical layout for these) for panics to work, you have to make sure the same #[global_allocator] is used, all standard library types remain layout compatible across versions (which would be very limiting and eg lock is in forever on which OSes use pthread_mutex and which use futex) and more. Realistically I don't think a stable ABI across rustc versions is possible without severely limiting what can be passed across the ABI boundary (crABI), which is completely orthogonal to making rust dylibs work across crate versions. crABI can work with cdylib's just as easily.

Edit: To put it another way, I did strongly prefer if stable ABI across rustc versions and stable ABI across crate versions with a single rustc version are treated as entirely separate problems implemented entirely separately. For stable ABI across crate versions you don't need to generate interface files. You can just reuse the existing rmeta file format, but only encode the subset of the items corresponding to the stable ABI.

@oli-obk oli-obk removed their assignment Jan 7, 2025
@Bryanskiy
Copy link
Contributor Author

@bjorn3

I think that is indicative of a fundamental issue with your implementation. Adding new private impls should not change the ABI. The whole point of the RFC is to allow changing the implementation of the dylib without changing it's ABI.

I can suggest the following solution for the above problem with impl's:

Split indices (disambiguators) into 2 sets: $S_1 = { 0, 1, ..., k }$, $S_2 = { k + 1, ..., n }$ where $k$ - number of exported impls, $n$ - total number of impls. For the exportable impls we assign indices from $S_1$ based on the their order in the source code. For the other impls we assign indices from $S_2$ in any way. This approach is stable across rustc versions and doesn't depend on private items.

Also the interface file is missing either the StableCrateId or it's constituent parts (crate name and all -Cmetadata arguments).

  1. Crate name is encoded in the interface name: lib{crate_name}.rs.
  2. -Cmetadata arguments are not yet supported. At the moment, dependence on them has been removed for stable "mangled" symbols.

Replacing all function bodies in an interface file with loop {} does not work in the presence of RPIT.

  1. I used loop {} as a temporary solution since fn foo(); syntax is not allowed 😄.
  2. Regarding the RPIT, yes, it is incompatible with the "header file" concept. But anyway, as you have already said, stable ABI should not depend on implementation. However, computing a hidden type is depends on implementation. So, I don't believe it is possible to support RPIT's without imposing strict limitations.

And serializing it as a rust source file misses the expansion context which is essential for hygiene. I did much rather use a format like JSON of while we don't guarantee a stable ABI across rustc versions a binary format.

One of the main objectives of this proposal is to allow building the library and the application with different compiler versions. This requires either a metadata format compatible across rustc versions or some form of source code. Stable metadata format is not a part of the RFC.

component 4: stable ABI mangling scheme
A stable ABI across rustc versions is a lot more involved than just mangling all symbols the same. You have to make sure multiple copies of the standard library seamlessly interoperate including using the same TypeId for &str and String ...

"stable ABI" is a bad wording here. Neither I nor the RFC offers a stable ABI. And all these issues are outside the scope of the proposal.

@bjorn3
Copy link
Member

bjorn3 commented Jan 13, 2025

One of the main objectives of this proposal is to allow building the library and the application with different compiler versions.

"stable ABI" is a bad wording here. Neither I nor the RFC offers a stable ABI. And all these issues are outside the scope of the proposal.

These two statements are conflicting with each other. Being able to build a library and application with a different compiler version requires both to share an ABI, in other words it requires having a stable ABI.

@Bryanskiy
Copy link
Contributor Author

Bryanskiy commented Jan 13, 2025

These two statements are conflicting with each other. Being able to build a library and application with a different compiler version requires both to share an ABI, in other words it requires having a stable ABI.

Only extern "C" functions and types with stable representation are allowed to be "exportable" right now. https://github.com/m-ou-se/rfcs/blob/export/text/0000-export.md#the-export-attribute.

@bors

This comment was marked as resolved.

@rust-log-analyzer

This comment has been minimized.

@bors

This comment was marked as resolved.

@rustbot rustbot added the A-attributes Area: Attributes (`#[…]`, `#![…]`) label Feb 11, 2025
@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rustbot rustbot added the A-meta Area: Issues & PRs about the rust-lang/rust repository itself label Feb 12, 2025
@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@bors

This comment was marked as resolved.

@Bryanskiy Bryanskiy marked this pull request as ready for review February 14, 2025 10:59
@rustbot
Copy link
Collaborator

rustbot commented Feb 14, 2025

Some changes occurred in compiler/rustc_codegen_gcc

cc @antoyo, @GuillaumeGomez

Some changes occurred in compiler/rustc_passes/src/check_attr.rs

cc @jdonszelmann

Some changes occurred in src/tools/rustfmt

cc @rust-lang/rustfmt

This PR modifies tests/run-make/. If this PR is trying to port a Makefile
run-make test to use rmake.rs, please update the
run-make port tracking issue
so we can track our progress. You can either modify the tracking issue
directly, or you can comment on the tracking issue and link this PR.

cc @jieyouxu

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

@jieyouxu
Copy link
Member

r? @bjorn3 (since you already did a bunch fo reviewers, or reroll)

@Bryanskiy
Copy link
Contributor Author

Bryanskiy commented Feb 14, 2025

I can split it into several PR's/commits if necessary.

@@ -1002,7 +1002,9 @@ impl<'a, 'tcx> Builder<'a, 'tcx> {
self.source_scope = source_scope;
}

if self.tcx.intrinsic(self.def_id).is_some_and(|i| i.must_be_overridden) {
if self.tcx.intrinsic(self.def_id).is_some_and(|i| i.must_be_overridden)
|| self.tcx.is_interface_build()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should just skip building MIR entirely rather than adding a ad-hoc hack like this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand how to do this correctly. Many analysis passes depend on mir_built. Adding checks around these passes seems like a bad idea.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe skip it in encode_mir in metadata encoding?
Since we are in --emit=metadata most MIR won't (?) be built unless it's needed for encoding.

let interface_dir = interface.parent().unwrap();
let crate_name = crate_name.unwrap();

let res = std::process::Command::new(compiler)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think not, but at the very least make sure to put the produced rmeta file in a temporary directory. As is you could have multiple rustc instances race trying to write an rmeta file and then read a partially written rmeta file.

Maybe an alternative to invoking rustc from within rustc would be to require the end user to compile interface files to rmeta files themself?

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 21, 2025
@Bryanskiy
Copy link
Contributor Author

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Mar 2, 2025
@bors
Copy link
Contributor

bors commented Mar 4, 2025

☔ The latest upstream changes (presumably #137927) made this pull request unmergeable. Please resolve the merge conflicts.

@Bryanskiy
Copy link
Contributor Author

Ping @bjorn3 , since it has been 2 weeks.

@petrochenkov petrochenkov self-assigned this Mar 18, 2025
@@ -82,6 +82,8 @@ struct AstValidator<'a> {
/// Used to ban explicit safety on foreign items when the extern block is not marked as unsafe.
extern_mod_safety: Option<Safety>,

is_interface: bool,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
is_interface: bool,
is_sdylib_interface: bool,

In other places (pretty-printing, etc) too.

This will be more useful to any unsuspecting reader looking at the code.

s.s.eof()
}

pub fn print_crate_as_interface(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's ok to add a new flag to fn print_crate instead of introducing a new function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer a new function. For the interface dump, PpAnn must always be fixed and comments must always be skipped.

@@ -2872,7 +2876,11 @@ fn add_upstream_rust_crates(
}
Linkage::Dynamic => {
let src = &codegen_results.crate_info.used_crate_source[&cnum];
add_dynamic_crate(cmd, sess, &src.dylib.as_ref().unwrap().0);
let lib = match &src.dylib.as_ref() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let lib = match &src.dylib.as_ref() {
let lib = match &src.dylib.as_ref().unwrap_or_else(...)

@@ -44,7 +44,7 @@ pub fn crates_export_threshold(crate_types: &[CrateType]) -> SymbolExportLevel {
}

fn reachable_non_generics_provider(tcx: TyCtxt<'_>, _: LocalCrate) -> DefIdMap<SymbolExportInfo> {
if !tcx.sess.opts.output_types.should_codegen() {
if !tcx.sess.opts.output_types.should_codegen() && !tcx.is_interface_build() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we even get to this logic in --emit=metadata builds?

Copy link
Contributor Author

@Bryanskiy Bryanskiy Mar 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, see encode_crate_root.

@@ -172,7 +172,7 @@ fn exported_symbols_provider_local(
tcx: TyCtxt<'_>,
_: LocalCrate,
) -> &[(ExportedSymbol<'_>, SymbolExportInfo)] {
if !tcx.sess.opts.output_types.should_codegen() {
if !tcx.sess.opts.output_types.should_codegen() && !tcx.is_interface_build() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question as for fn reachable_non_generics_provider.

)
})?;

let tmp_path = match tempfile::tempdir() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other code uses TempFileBuilder for this.

let tmp_path = match tempfile::tempdir() {
Ok(tmp_path) => tmp_path,
Err(error) => {
return Err(MetadataError::LoadFailure(format!(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rustc_metadata has the FailedCreateTempdir error type for this.

}
};

let crate_name = dyn_crate.unwrap_or(kw::Empty);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let crate_name = dyn_crate.unwrap_or(kw::Empty);
let crate_name = dyn_crate.unwrap();

See the comment about assert above.

@@ -1746,7 +1746,9 @@ impl<'tcx> TyCtxt<'tcx> {
MetadataKind::None
}
CrateType::Rlib => MetadataKind::Uncompressed,
CrateType::Dylib | CrateType::ProcMacro => MetadataKind::Compressed,
CrateType::Dylib | CrateType::ProcMacro | CrateType::Sdylib => {
MetadataKind::Compressed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sdylibs aren't supposed to have metadata (well, in the usual unstable format at least).

Unless the interface libraries are also considered CrateType::Sdylibs during the build.
CrateType::Sdylib with and without is_interface_build behave quite differently.
Maybe they should be split into two crate types, or CrateType::Sdylib should be reset to CrateType::Rlib or something during the interface build? Not sure.

Or maybe crate_type and is_interface_build should just be passed together to all places where the distinction is necessary.

.unwrap_or(self.target.dll_suffix.as_ref());
let dylib = interface.0.with_extension(suffix);
if !dylib.exists() {
return Err(CrateError::ExternLocationNotExist(self.crate_name, dylib));
Copy link
Contributor

@petrochenkov petrochenkov Mar 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, all these errors should be covered by tests (unless they occur due to some external system failures that are hard to reproduce).

Same applies to the new errors in compiler/rustc_passes/src/errors.rs.

tcx: TyCtxt<'tcx>,
exportable_items: FxIndexSet<DefId>,
in_exportable_mod: bool,
catch_exportable_in_mod: bool,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
catch_exportable_in_mod: bool,
seen_exportable_in_mod: bool,

if self.tcx.get_attr(def_id, sym::export).is_some() {
self.in_exportable_mod = true;
}
let old_catch_exportable_in_mod = self.catch_exportable_in_mod;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let old_catch_exportable_in_mod = self.catch_exportable_in_mod;
let old_catch_exportable_in_mod = mem::replace(&mut self.catch_exportable_in_mod, false);

&& impl_.of_trait.is_none()
&& self.tcx.is_exportable(item.owner_id.def_id.to_def_id())
{
self.order.insert(item.owner_id.def_id.to_def_id(), self.id);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.id is always equal to self.order.len() here, no need to track it.

stable_order_of_exportable_impls,
..*providers
};
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't look at the logic in this file in detail, it 100% has issues, but it should be ok approximation for a first prototype.

}
match item.kind {
ItemKind::ExternCrate(Some(orig_name))
| ItemKind::ExternDynCrate(Some(orig_name)) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extern dyn crate self as foo; should produce an error even earlier, so this probably shouldn't be necessary.

Copy link
Contributor Author

@Bryanskiy Bryanskiy Mar 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extern dyn crate self as foo; should produce an error even earlier

I tried it. It is not true.

@@ -1049,6 +1050,10 @@ impl OutputFilenames {
.unwrap_or_else(|| OutFileName::Real(self.output_path(flavor)))
}

pub fn interface_path(&self) -> PathBuf {
self.out_directory.join(format!("lib{}.{}", self.crate_stem, RUST_EXT))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
self.out_directory.join(format!("lib{}.{}", self.crate_stem, RUST_EXT))
self.out_directory.join(format!("lib{}.rs", self.crate_stem))

The constant isn't used anywhere else.

}

impl CrateType {
pub fn has_metadata(self) -> bool {
match self {
CrateType::Rlib | CrateType::Dylib | CrateType::ProcMacro => true,
CrateType::Rlib | CrateType::Dylib | CrateType::ProcMacro | CrateType::Sdylib => true,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use rustc_span::symbol::{Symbol, sym};

trait AbiHashStable<'tcx> {
fn hash(&self, tcx: TyCtxt<'tcx>, hasher: &mut StableHasher);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be a trait? It's not currently used generically.

Also, probably better to rename the method to something like abi_hash to avoid confusing it with regular hash.

v0::mangle(tcx, instance, instantiating_crate)
}),
let mut symbol = match tcx.is_exportable(def_id) {
true => v0::mangle(tcx, instance, instantiating_crate),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
true => v0::mangle(tcx, instance, instantiating_crate),
true => format!("{}_{}", v0::mangle(tcx, instance, instantiating_crate), export::compute_hash_of_export_fn(tcx, instance))

@@ -32,6 +32,7 @@ pub(super) fn mangle<'tcx>(
let mut cx: SymbolMangler<'_> = SymbolMangler {
tcx,
start_offset: prefix.len(),
is_exportable: tcx.is_exportable(def_id),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe pass this as an argument to fn mangle, we already know whether it's exportable when calling this.

@petrochenkov
Copy link
Contributor

There are also several big, but non-blocking, issues here.

  • extern dyn crate probably shouldn't exist. We should determine the crate type when attempting to find and load it in crate loader.
    Most crates are passed through --extern and do not use extern crate items.
    It's possible to add a modifier to --extern too, but probably better to just detect the crate type automatically.
  • Running a new compiler with Command::new.
    It's clear that we cannot call any rustc_interface APIs from rustc_metadata directly due to the crate dependency structure, but maybe it can be done through a function pointer?
    (If possible, using the hook mechanism compiler\rustc_middle\src\hooks\mod.rs)
  • Compiling the interface as pretty-printed source code doesn't use correct macro hygiene (mostly relevant to macros 2.0, stable macros do not affect item hygiene).
    I don't have much hope for encoding hygiene data in any stable way, we should rather support a way for the interface file to be provided manually, instead of being auto-generated, if there are any non-trivial requirements.

@petrochenkov
Copy link
Contributor

The PR description also needs some updates.
@rustbot author

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-attributes Area: Attributes (`#[…]`, `#![…]`) A-meta Area: Issues & PRs about the rust-lang/rust repository itself A-run-make Area: port run-make Makefiles to rmake.rs S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants