Let's say I have a bunch of objects (e.g. parquet) in R2, can the agent mount them? Or how do I best give the agent access to the objects? HTTP w/ signed urls? Injecting the credentials?
Dynamic Workers don't have a built-in filesystem, but you can give them access to one.
What you would do is give the Worker a TypeScript RPC interface that lets it read the files -- which you implement in your own Worker. To give it fast access, you might consider using a Durable Object. Download the data into the Durable Object's local SQLite database, then create an RPC interface to that, and pass it off to the Dynamic Worker running on the same machine.
See also this experimental package from Sunil that's exploring what the Dynamic Worker equivalent of a shell and a filesystem might be:
At first I was trying to figure out why the parent comment was getting downvoted, then I read the last line. Yeesh, ya, you don't need to "learn" to sharpen, just get one of those pull-throughs. They is a minuscule learning curve with with it. It doesn't do the best sharpening job but as a particularly YouTuber once said: "The best sharpener is the one you will use."
People don't want to do it and they don't want to learn to do it. It's easier for them to buy a new knife. They're not expensive. Maybe keep the old one for garage stuff and gardening.
A new knife might not be expensive, but it's a new thing that has to be produced, and packaged, and shipped, and stored, and so on. Just keep your old stuff in shape, people.
Indeed, it would have been interesting but I really wanted to get the blog post out on the launch day of the MacBook Neo and did not have the bandwidth to run additional cloud experiments.
I ran TPC-DS SF300 now on the c6a.4xlarge. It turns out that it's still quite limited by the EBS disk's IO: while 32 GB memory is much more than 8 GB, DuckDB needs to spill to disk a lot and this shows on the runtimes. Running all 99 queries took 37 minutes, so about half of the MacBook's 79 minutes.
> Command being timed: "duckdb tpcds-sf300.db -f bench.sql"
> Percent of CPU this job got: 250%
> Elapsed (wall clock) time (h:mm:ss or m:ss): 37:00.96
reply