Rust: Provide a trait method to (optionally) control resource allocation

[cpetig]

This enables resource allocation from a user provided arena instead of the heap.

Arena implementation in test case generated by genAI.

[alexcrichton]

I like the idea, thanks! I've got some comments below, but let me know if you'd prefer to not handle anything in particular.

[alexcrichton]

@cpetig ok I tried my hand a this a bit and resulted in the last commit here. Everything looks good to me now but I'd want to run that by you to confirm.

I realized that ideally what I want is:

trait GuestThing {
    type Rep: ResourceRep = Box<Option<T>>;
    // ...
}

however defaults for associated bounds aren't stable in Rust. In lieu of that I did a, now unrelated, refactoring to the Option<T> "rep" which abstracts operations behind a trait. This is sort of like the _FooStorage<T> from prior but that didn't do what I want because it wasn't customizable. It's still not customizable due to the blanket impl, but I think it's a bit closer to being customizable (subjectively).

I also realized that the resource_into_raw_ method needed to be unsafe mostly because otherwise it would be possible to override just resource_into_raw_, nothing else, and you'll have written no unsafe code but that would be theoretical UB. This is best modeled as an unsafe trait but in lieu of that with associated types and such, I'll pass on that.

One possible future option is to basically require all Rust exported resources to manually say type Rep = Box<Option<Self>> as a form of boilerplate. That way it could be fully customizable at the cost of being a bit more verbose.

[cpetig]

however defaults for associated bounds aren't stable in Rust. In lieu of that I did a, now unrelated, refactoring to the Option<T> "rep" which abstracts operations behind a trait. This is sort of like the _FooStorage<T> from prior but that didn't do what I want because it wasn't customizable. It's still not customizable due to the blanket impl, but I think it's a bit closer to being customizable (subjectively).

It amuses me that you took my frustration with the incompleteness of traits in stable Rust to the next level, yesterday I was just sad that defaults for associated types were not possible in this PR, but a bound on an associated type is a concept I didn't miss until today.

I also realized that the resource_into_raw_ method needed to be unsafe mostly because otherwise it would be possible to override just resource_into_raw_, nothing else, and you'll have written no unsafe code but that would be theoretical UB. This is best modeled as an unsafe trait but in lieu of that with associated types and such, I'll pass on that.

Oh, another facet of unsafe which I didn't see before. An unsafe trait would make sense, because you rely on the correctness of the implementation of two particular functions, but you don't want to make the trait implementation unsafe for all people not overriding the allocation method.

One possible future option is to basically require all Rust exported resources to manually say type Rep = Box<Option<Self>> as a form of boilerplate. That way it could be fully customizable at the cost of being a bit more verbose.

That feels excessive, but also missing specialization has made one of my API designs less elegant than I wished for before. 🤷

Thank you for teaching me new aspects of Rust, I will need some hours to read and understand your new design.

[cpetig]

Looks elegant to me. I think documentation always improves when it written by more than a single individual, you pointed out so many things I have forgotten to mention.

[cpetig]

This is the one change which makes me curious, I tried to minimize the number of characters in the unsafe block. Why would you put the let into the unsafe block instead?

[alexcrichton]

Ah mostly just subjective on this. It's all generated code anyway and doesn't really get // SAFETY: ... comments regardless so I basically just took the expedient route here. Otherwise though I'd agree that this would probably best be served by 2 blocks.

[alexcrichton]

Ok I'll go ahead and flag this for merge, thanks for putting up with me here @cpetig and thanks again!