Yeah, and I genuinely want to like Rust and I pick it up a few times every year (and have done for the last decade), but each time I get burned by trying to do something that would be trivial (and safe!) in Go or C.
Most recently, I'm trying to build a non-allocating, minimally copying `Lines` iterator which reads from an internal `Reader` into an internal buffer and then yields slices of the internal buffer on each call to `next` (each slice represents a single line, and it's an error if a line exceeds the capacity of the internal buffer); however, as far as I can tell, this is unworkable without some unsafety because the signature is `fn next(self: &mut Lines<'a>) -> Result<&'a [u8]>` which doesn't work because Rust thinks the mutable reference to self must outlive 'a, and explicitly setting the mutable self reference to 'a violates the trait. If I forego the trait and just make a thing with a `next()` method, then I can't use it in a loop without triggering some multiple mutable borrows error (each loop iteration constitutes a mutable borrow and for some reason these borrows are considered to be concurrent). The only thing I can think to do is have a `scan()` method that finds the next newline and notes its location inside the `Lines` struct and a separate `line()` method that actually fetches the resulting slice from the buffer.
I don't run into this in C or Go, and for all of the difficulty of battling the borrow checker, I'm not getting any extra safety (in this case).
There are downsides to this approach because it uses internal iteration while most things in Rust use external iteration. But shit happens. Go doesn't even have a first class concept of iterators as an abstraction (yet), and its standard library contains patterns for both internal (sync.Map) and external (bufio.Scanner) iteration.
> The only thing I can think to do is have a `scan()` method that finds the next newline and notes its location inside the `Lines` struct and a separate `line()` method that actually fetches the resulting slice from the buffer.
Of course you don't. Neither C or Go even have abstractions called "iteration" at all. They have patterns for them. And Go lets you iterate over a fixed set of built-in types. (Currently. Maybe it's changing: https://research.swtch.com/coro)
Besides, Go has a garbage collector. You should expect all sorts of patterns involving memory/copying to change when you move from a language with a GC to one without.
> Of course you don't. Neither C or Go even have abstractions called "iteration" at all. They have patterns for them.
My goal isn’t “implement an Iterator trait”, I just want to iterate over lines in a file, so I don’t especially care that Go and C don’t have an iterator type.
> Besides, Go has a garbage collector. You should expect all sorts of patterns involving memory/copying to change when you move from a language with a GC to one without.
I’m not allocating, so the GC doesn’t matter. The Go and C versions look essentially the same—it’s only the Rust version that I had a hard time with because of the borrow checker.
Great, then the closure approach should work perfectly!
> I just want to iterate over lines in a file
No... you don't. You specifically said you wanted to do this without allocating and minimal copying. That's a different problem than "just iterate over lines in a file."
> I’m not allocating, so the GC doesn’t matter.
Of course it does. The GC manages the lifetimes for you. In your C code, you manage the lifetimes yourself and likely rely on the caller to not fuck things up.
> No... you don't. You specifically said you wanted to do this without allocating and minimal copying. That's a different problem than "just iterate over lines in a file."
Yes, of course. The point is iterating over the lines of a file given the aforementioned constraints and not “implementing Rust’s Iterator abstraction”. Whether or not Go or C have iterator abstractions is immaterial.
> Of course it does. The GC manages the lifetimes for you. In your C code, you manage the lifetimes yourself and likely rely on the caller to not fuck things up.
GC only manages allocations. There are no allocations here to manage.
> GC only manages allocations. There are no allocations here to manage.
You're missing the forest for the trees. The GC is integrally tied to lifetimes. If you have a `*Foo` in Go, that may or may not be on the stack. It might be on the stack, thus no allocation, if the Go compiler can prove something about its lifetime and usage. Same deal with `[]byte`. Maybe that's tied to an array on a stack somewhere. Maybe not. AFAIK, in order to get guarantees about this in Go you need to drop down into `unsafe`.
> Yes, of course. The point is iterating over the lines of a file given the aforementioned constraints and not “implementing Rust’s Iterator abstraction”. Whether or not Go or C have iterator abstractions is immaterial.
That's why the very first thing I said to you was to point out the closure approach. I even linked you to real code (that I've written) that does it. Yet 'round and 'round we go.
TBF you could do it in most languages which do have iteration abstraction.
However assuming trying to keep to 0-alloc odds are good you'd probably just be telling the user to git gud as the slices would become nonsensical on every refill of the buffer.
I could easily imagine trying to do it because I can rather than because I actually need to.
Also because of the missing middle: in Go (or Java, or C#, or even python) you could hand out slices to a large buffer, but discard the buffer rather than refill it in place, relying on the GC to clean things up (and possibly give you the same buffer on the next allocation). So you get an efficiency middle ground where you allocate more than strictly necessary but not for every line, and maintain correctness.
In Rust that’d require some sort of Rc projection which I’m not sure even exists?
Most recently, I'm trying to build a non-allocating, minimally copying `Lines` iterator which reads from an internal `Reader` into an internal buffer and then yields slices of the internal buffer on each call to `next` (each slice represents a single line, and it's an error if a line exceeds the capacity of the internal buffer); however, as far as I can tell, this is unworkable without some unsafety because the signature is `fn next(self: &mut Lines<'a>) -> Result<&'a [u8]>` which doesn't work because Rust thinks the mutable reference to self must outlive 'a, and explicitly setting the mutable self reference to 'a violates the trait. If I forego the trait and just make a thing with a `next()` method, then I can't use it in a loop without triggering some multiple mutable borrows error (each loop iteration constitutes a mutable borrow and for some reason these borrows are considered to be concurrent). The only thing I can think to do is have a `scan()` method that finds the next newline and notes its location inside the `Lines` struct and a separate `line()` method that actually fetches the resulting slice from the buffer.
I don't run into this in C or Go, and for all of the difficulty of battling the borrow checker, I'm not getting any extra safety (in this case).