Tracking Issue for core::clone::CloneToUninit trait

[dtolnay]

Feature gate: #![feature(clone_to_uninit)]

This is a tracking issue for CloneToUninit and its impls. This trait backs the behavior of Rc::make_mut and Arc::make_mut, and likely in the future also Box::clone.

Public API

// core::clone

pub unsafe trait CloneToUninit {
    unsafe fn clone_to_uninit(&self, dst: *mut Self);
}

unsafe impl<T: Clone> CloneToUninit for T;
unsafe impl<T: Clone> CloneToUninit for [T];

// TODO:
// unsafe impl CloneToUninit for str;
// unsafe impl CloneToUninit for CStr;
// unsafe impl CloneToUninit for OsStr;
// unsafe impl CloneToUninit for Path;

Steps / History

• Implementation: Generalize {Rc,Arc}::make_mut() to unsized types. #116113
• CloneToUninit impls #126877
• Convert Box::clone to be based on CloneToUninit
• Final comment period (FCP)¹
• Stabilization PR

Unresolved Questions

• Consider using *mut () in the signature instead of *mut Self. This makes CloneToUninit dyn-safe. Generalize {Rc,Arc}::make_mut() to unsized types. #116113 (comment)

Footnotes

1. https://std-dev-guide.rust-lang.org/feature-lifecycle/stabilization.html ↩

[dtolnay]

@kpreid #116113 (comment)

I see a big reason to be cautious about stabilizing CloneToUninit: it is a trait both unsafe to implement and unsafe to call, yet it provides something that many slice-like (as opposed to dyn-like) DSTs would want to implement. We wouldn't want to lock in a doubly unsafe interface as a key interop trait, rather than some fully safe “placement clone” (in the same sense as “placement new”) that can solve this problem.

I thought about what a safer primitive could be that still enables what we need in make_mut. I think it might (almost?) exists already:

impl<T, A> Rc<T, A>
where
    T: ?Sized,
    A: Allocator,
    Box<T, RcAllocator<A>>: Clone,
{
    pub fn make_mut(this: &mut Self) -> &mut T {...}
}

where RcAllocator<A: Allocator> is an allocator whose every allocation points to the value part of a MaybeUninit<RcBox<T>>. When you ask it to allocate with the Layout of some type T, it delegates to the underlying allocator to allocate the Layout of RcBox<T> and offsets that pointer.

rust/library/alloc/src/rc.rs

Lines 286 to 291 in fcae626

	#[repr(C)]
	struct RcBox<T: ?Sized> {
	strong: Cell<usize>,
	weak: Cell<usize>,
	value: T,
	}

Then make_mut works by turning the pointer held by the Rc<T, A> into a Box<T, RcAllocator<A>>, cloning it, and turning the Box<T, RcAllocator<A>> back into Rc<T, A>.

Instead of implementing an unsafe trait CloneToUninit, types plug into make_mut support by implementing the safe trait Clone for Box<MyType, A> where A: Allocator. The concrete allocator type RcAllocator itself is not public; your impls are written using a generic A as they already universally are. Notice no unsafe in this impl!

rust/library/alloc/src/boxed.rs

Lines 2218 to 2222 in fcae626

	impl<T: Clone, A: Allocator + Clone> Clone for Box<[T], A> {
	fn clone(&self) -> Self {
	let alloc = Box::allocator(self).clone();
	self.to_vec_in(alloc).into_boxed_slice()
	}

What I don't know is whether Box's use of Unique throws a wrench in the works. As I currently understand it, I think it does not. Casting Rc's NonNull<RcBox<T>> into Box<T, RcAllocator<A>> in order to clone it would not be sound, but I also think that's not necessary for the above to work. If we can rearrange Rc's layout a bit to achieve a guarantee that Rc<T, A> has the same layout as Box<T, RcAllocator<A>>, then we'd only be casting &Rc<T, A> to &Box<T, RcAllocator<A>> to pass into the clone impl, and as such, Unique never comes into play. Obviously I can have as many &Box<T> aliasing one another's allocations as I want. An actual owned Box<T, RcAllocator<A>> would never exist except for the value that becomes the return value of the clone, during which it is unique. I think it all works out as required, but I wouldn't be surprised if I am missing something.

[kpreid]

Box<T, RcAllocator<A>>: Clone,

One thing that this doesn't help with, that the current CloneToUninit does, is cloning into a different container type. I'm not sure if that's useful enough to be worthwhile, but it feels to me like it might be an important piece of a future Rust where there are fewer obstacles to using DSTs, which is one of the things I hoped to do by having (a safer version of) CloneToUninit be public.

[the8472]

Tangentially, a generic way to build a container type and then clone or write into its uninitialized payload slot would fit nicely into a solution for rust-lang/wg-allocators#90

[dtolnay]

https://doc.rust-lang.org/nightly/std/clone/trait.CloneToUninit.html#implementors

I think we should consider adding #[doc(hidden)] to the T: Copy specializations. They are just distracting when it comes to documenting the public API of this trait.

[the8472]

The std-dev-guide says we should use private specialization traits, which we do in most places. The implementation PR promoted a private trait to a public one, so it should have changed the specialization to a subtrait.

[GrigorenkoPV]

The std-dev-guide says we should use private specialization traits, which we do in most places. The implementation PR promoted a private trait to a public one, so it should have changed the specialization to a subtrait.

I have implemented this suggestion in #126877 (scope creep, yay).

Also, this has an additional benefit of no longer needing any #[doc(hidden)] to hide the mess:

I think we should consider adding #[doc(hidden)] to the T: Copy specializations. They are just distracting when it comes to documenting the public API of this trait.

[lolbinarycat]

why does this take *mut Self instead of &MaybeUninit<Self>?

[zachs18]

why does this take *mut Self instead of &MaybeUninit<Self>?

1. It would have to be &mut MaybeUninit<Self>, not &MaybeUninit<Self>, so that it can be written into.
2. MaybeUninit<T> (currently) only supports T: Sized (and AFAIK there is not yet any propopsed RFC to expand MaybeUninit or unions in general to support T: ?Sized fields), which makes this not work for (one of) the main motivating features: Arc/Rc::make_mut on slices.
3. IMO clone_to_uninit should take *mut () instead of *mut Self anyway, so that the trait is object-safe (listed as an unresolved question above).
   a. It could take something like &mut [MaybeUninit<u8>] to be object-safe but still allow size-checking and in some cases safe writing, I suppose, but perhaps just having a separate expected_layout: Layout parameter or similar would be better, though that would only be useful in debug builds anyway, since if they were wrong it would by the contract of the trait already be UB.

[kpreid]

One of the implementation steps listed in this issue is “Convert Box::clone to be based on CloneToUninit”. I thought I’d try implementing it, but that turns out to be tricky, because <Box<T> as Clone>::clone_from() exists. While both clone_from() and clone_to_uninit() allow in-place clones, clone_from() (implicitly) promises that if it panics, the destination is still a valid value, whereas clone_to_uninit() does not promise that, and cannot reuse nested allocations as clone_from() can. This means that <Box<T> as Clone>::clone_from() cannot be implemented in terms of T as CloneToUninit, except in the default no-reuse way.

To solve this problem, I tried writing a specialization trait which used clone_from when T: Clone and clone_to_uninit() into a new box otherwise, but the compiler tells me

error: cannot specialize on trait `Clone`
    --> library/alloc/src/boxed.rs:1825:25
     |
1825 | impl<T: CloneToUninit + Clone> SpecCloneBoxFrom for T {
     |                         ^^^^^

which I read is a restriction to prevent lifetime specialization unsoundness, so this seems like an unsolvable problem, but I’m not particularly familiar with tricks for stdlib specialization. I’m posting this comment in the hopes that someone else can come up with a way that does work without regressing Box::clone_from(). The goal of CloneToUninit isn't having exactly clone_to_uninit(), but being able to clone DSTs, so perhaps someone can come up with an improved CloneToUninit that — also ‘clones to init’, somehow? Perhaps it should be an optional method just like Clone has the optional clone_from() — but how to do that without forcing extra allocations for unwind safety?

[zachs18]

One thing I tried was making CloneToUninit instead be CloneUnsized with an additional clone_from_unsized(&mut self, src: *const u8) method that, on panic, guarantees that *self is still valid, but may have changed (e.g. if cloning the third element of a length-5 slice panics, then the first and second elements may have been successfully cloned into self.

Then, we can have a single impl<T: ?Sized + CloneUnsized, A: Allocator> for Box<T, A>, where Box::clone_from can check if the metadata and layouts are the same, and if so call CloneUnsized::clone_from_unsized without allocating, otherwise just calling *self = Box::new_from_ref(&**src); (which creates a Box<T> from cloning a &T where T: CloneUnsized)

master...zachs18:rust:clone_unsized (ignore the last "stash" commit, that was something additional I was trying)

This seems to work well, but does complicate the API somewhat.

[bal-e]

I copied the CloneToUninit code into my own codebase to try it out, and I noticed an important limitation: it's really hard to use this trait (in stable code). Consider a hypothetical Box::clone_unsized_from:

impl<T: ?Sized + CloneToUninit> Box<T> {
    /// Clone an unsized value into a new [`Box`].
    fn clone_unsized_from(value: &T) -> Self {
        // Allocate the space for the value
        let layout = Layout::for_value(value);
        let ptr: *mut u8 = unsafe { core::alloc::alloc(layout) };
        // Clone into the allocated space
        let ptr: *mut T = todo!(); // How to do this in stable code??
        unsafe { value.clone_to_uninit(ptr) };
        // Even if .clone_to_uninit() took '*mut ()', this call would still need '*mut T'.
        unsafe { Self::from_raw(ptr) }
    }
}

If T is unsized, a pointer/reference to T includes metadata, and copying metadata from value while changing the address to ptr is really hard to do. In nightly, we have pointer::with_metadata_of(), but I'm not aware of the expected stabilization timeline of the pointer metadata features.

I do have a workaround, but it is not dyn-compatible. I added a new method, fn ptr_with_address(&self, addr: *mut ()) -> *mut Self. It's relatively easy for every type implementing CloneToUninit to implement this method, but users of those types can't do it themselves.

unsafe impl<T: Clone> CloneToUninit for [T] {
    unsafe fn clone_to_uninit(...) { ... }

    fn ptr_with_address(&self, addr: *mut ()) -> *mut Self {
        // The metadata is the slice length: add it manually.
        core::ptr::slice_from_raw_parts_mut(addr.cast(), self.len())
    }
}

unsafe impl CloneToUninit for i32 {
    unsafe fn clone_to_uninit(...) { ... }

    fn ptr_with_address(&self, addr: *mut ()) -> *mut Self {
        // Since 'Self' is sized, this is a trivial cast.
        addr as *mut Self
    }
}

struct Foo(i32, [u8]);

unsafe impl CloneToUninit for Foo {
    unsafe fn clone_to_uninit(...) { ... }

    fn ptr_with_address(&self, addr: *mut ()) -> *mut Self {
        // In this method, the compiler knows that `Self` and `[u8]` have
        // the same metadata, so it allows casting pointers between them.
        self.1.ptr_with_address(addr) as *mut Self
    }
}

Now Box::clone_unsized_from can just call value.ptr_with_address(ptr) to get a pointer with the right metadata.

Is the plan to stabilize CloneToUninit after the relevant pointer metadata features are stabilized? Otherwise, is it acceptable that third-party container types (in stable code) can't make use of it until those features are stabilized?