Absolutely love temporal, but pretty shocked to see they have a $12.55B valuation.... feels nuts?
We're using temporal as the foundation for agents running on our platform doing accounting work for clients, it's proven extremely effective and I love how durable workflows are within temporal.
+100 on the technology, but realizing they are also VC funded makes me a little more cautious when it comes to vendor lock in.
Sounds like you're contributing to your own narrative here. The only self-centered douche canoe of a comment I see in this thread is from you.
HN has its share of good & bad, but I've generally enjoyed talking to some of the people on this site; much more than sites like Reddit which imo fit your description much much better.
Yeah this article is quite shallow after having read it; delegating to a remote sub-agent sounds novel but how does it _actually_ perform?
I'm personally skeptical of optimizing for minimal token consumption, the closer to a vanilla setup I am the more confident I feel I'm always getting the best performance out of my models.
Just look at how JetBrains measured rtk and found that while an individual call saves tokens, agents on average perform *more* turns and use *more* tokens to accomplish one task[1]
I've almost entirely stopped using Opus 5, at least with Sonnet I know what i'm getting -- Fable for the complex stuff and Sonnet for the precision changes.
The verbosity of Opus 5 isn't even my issue, it's consistency. For every 10 tasks Opus 5 accomplishes, there's at least one task that Opus 5 does just an atrocious job of, or a debugging investigation that it just completely goes off the rails on.
Great article and super relevant advice. As building becomes easier and easier in our career those with the agency to find problems worth solving become x10 more productive.
Like other people in this thread are also saying, the more you master this skill the more often you'll have people finding *you* to give them their problems, which only makes this easier.
I’ve been working on a harness for accounting agents at my job recently and it’s been a pretty interesting experience.
We originally started with building a CLI tool so our LLMs could more easily interact with our platform. I cannot recommend enough the value of having an internal CLI. It’s both fun to build and extremely useful for agents.
We paired this with skills initially, but found that the way folks built skills was often too prescriptive and limited to the authors own specific function in the company. A 2k line long skill suffers from the same gaps as we do, if an agent is just following a laundry list it’s less likely to reason about the request it’s doing.
So we instead asked ourselves: what if we just _let_ the agent reason about the work to be done and only provided the tools + guardrails to gather context and perform accounting work?
Turns out frontier models are GOOD at what they do, they outperformed our highly prescriptive skills and were able to work across a larger set of tasks even without instruction on how to do those tasks.
It’s a breath of fresh air from the decade of CRUD I’ve worked on, harness engineering is very neat.
I think you've really hit the mark on how the harness should be structured:
1. Guardrails - deterministic, social intelligence, team alignment & accountability
2. Learn by doing
3. make it stupid easy for the agent to research and access data
4. DRY
Hi. This is very interesting, could you link to the research? There is a dearth of proper research studies that A/B test what approach is best in terms of harness structure based on repeatable benchmark data with relevant sample uses-cases.
This is the same "tension" I keep seeing in my day job. Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask.
As you say frontier models are very good at figuring things out. Being too prescriptive is counterproductive, it over-constrains the model, it fills the context with conflicting instructions, it reduces the ability of the agent to respond to novel situations (and really in real life most situations are going to be novel). If you want to follow a process or a checklist you probably shouldn't use an LLM, or you should use it for some sub-tasks in the checklist/process but something more deterministic to work through the list.
> If you want to follow a process or a checklist you probably shouldn't use an LLM
I like to externalize tasks as markdown files with checklists, they are still planned by agents but I can pass the plan around to judge agents and fix some errors before implementing.
I also have the coding agents comment on each closed checklist item, so the same file becomes a log of what happened. This goes to the implementation judge. I can also switch agents anytime, or resume a task days later no problem.
I am avoiding internally provided tools for todo lists and planning because they do not leave the same artifact trail which makes judging with separate agents easy.
> Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask.
Hm, but thats ok right? I mean some people like to code with LLM and other people like to let LLM code for them.. no?
That works for well trod paths, e.g “fix ci” works exceedingly well. “why app slow” obviously doesn’t work because the task is underspecified. But in order to properly specify you either need an experienced engineer who knows how to narrow the problem domain, or you have to provide some template instructions/output formats (e.g, skills) which will invariably never fit the problem perfectly
I wouldn't agree. Sota models can do self-directed sampling, profiling, benchmarking, read call trees, etc. to give you a report of the app's bottlenecks and then recommend solutions that can be vetted.
I do this constantly.
As the upstream comment points you, you don't need to specify. Sota models are that good. And by being overprescriptive you can accidentally shut off branches that they would've taken, downgrading the quality of their work.
In my experience if you’re at the point where you have something to sample then the hard part is already done.
In a perfect world everything is covered by distributed tracing and the problems are only in your application code and the agent just needs to find the data
In reality the data is often missing or misleading. “Your observability sucks”? Yeah, but that’s life
I use skills. The skills are not typically "how to perform a task in detail" they are more about what relevant tools and knowledge are required to work in a domain. That is I give the LLM the information it needs about the system but not a sequence of how to accomplish a task. I treat it more like a human and less like a computer.
It really doesn’t need to be that much more specified, give it context to the tools and level of analysis you expect then “why app slow” is a reasonable prompt
> “why app slow” obviously doesn’t work because the task is underspecified
Definitely not true and like everyone else is saying, shows how people still underestimate these models.
I have been working on a simple vite + react app lately and commonly ask Gemini/Antigravity to just "improve speeds", "x is running slow, check it out" and have no complaints.
disturbingly, when I was using antigravity with gemini pro it was actually quite good at working out 'why app slow' types of problems. Maybe I've been lucky but it seems really good at determining why something might be wrong. It may ask for more logging or diagnostics and run for a long time but it was really digging in and making changes or suggestions to solve the problems.
The problem is that even Fable still make trivial yet high impact mistake when let on their own, and then you'd need to read the whole code to catch them…
Meanwhile they are very good at implementating an explicit algorithm that you feed it to them.
The trick is to set up the harness so that the solution is easy to verify - you’ve profited as long as verification is cheaper than building, but ideally verification is close to automatic (not always achievable of course).
Generally you want to include objective/repeatable outputs as citations.
An example would be, if you invest in an awesome layered test rig (browser test, fuzz/property tests, very well reviewed unit/integration tests, etc.) then you should be able to add features by just reading the acceptance test and scanning unit tests.
If you want outlier performance from these models it is best to just ask in the most high level prompt of the most minimal harness and let them loose.
Any extra information reduces their performance.
However, as often as these models output masterpieces, they also produce utter garbage so our current choice is for them to have a process to follow that can be reviewed by humans and LLMs.
I've been building a harness (on top of Pi for that matter) and have had similar experiences. Pi itself helps a lot with it being extensible by design but it's definitely been a challenge to make certain things work in an expected way.
The native app I'm building on top, which I hope people who are less technical (or not technical at all) will use, is even more interesting because it's not just supposed to shell out to the CLI for everything and needs its own state.
I’ve noticed it’s the performance that suffers when agents are paired with more than a single CLI and non-prescriptive skills. Since it seems to be out of its training data, anything non-trivial and the model just tries to brute force its way to a solution. Maybe it’s also about building them as self-improving, though I’ve been doing it manually for a CLI we don’t own.
I’ve also found that Claude and friends are eerily good at using classic Unix CLI tools so I build mine in the same style, not unlike the `gh` CLI from GitHub, though with an agent-first design shape.
Usually I’m returning TSV as a default format and I add a `help-all` subcommand to list every available command at once when needed. Another thing that helps is adding just-in-time context-sensitive hints, such as: user has just run a list query with at least one result. Add a one-liner to the response explaining the command shape for getting the detail view of the first response.
In terms of skill files, I like to have my CLI generate them dynamically at runtime by walking their own current command tree and then feeding that through a text template.
Yeah the CLI can provide schema for commands via the usual ‘—help’ syntax, so agents are able to discover + explore commands on their own.
As for an example: if our agent wants to book a journal entry to cash accounts for a client, it MUST provide receipt and directly link the transaction from the clients bank feed, if it attempts to do so without the requisite information we deny the tool call and ask the agent to escalate back to the client for proof of receipt.
Often times this results in the agent not doing the work and instead sending a message back to the client asking for proof of the transaction.
For humans on our platform there may be valid situations where we’d want to allow this, but for our agent this is a hard guardrail thus why it’s not just standard validation for any JE posting on our platform.
if our agent wants to book a journal entry to cash accounts for a client, it MUST provide receipt and directly link the transaction from the clients bank feed
It’s actually encoded in an abstraction that we call “gates” which run before any tool call an agent makes, this allows us to prevent the tool call from happening and return a cited code + explanation on why their tool call was not executed
This couldn't have been said a year ago. It's amazing to watch. I have been building harnesses and applying networked agents to various domains since the GPT-3 API came out, and even two years ago, frontier models were just not at acceptable quality to make these harnesses useful. Everything changed overnight near the end of 2025. What will next year hold?
This doesn’t seem to work when the harness feeds images and asks the agent to do things in the real world. It fails to devise ways to keep track of its progress and fails to utilize its tools effectively.
Sorry I can't share what we're doing here directly!
I will however say that this page alone does a pretty good job of illustrating what an agent harness might look like: https://docs.agno.com/tools/overview
* System prompt
* Tool calls
* Model definition
Everything else (guards / etc) can just exist as code abstractions between the agent layer and the tool layer.
I don't have a strong argument for or against using MCP, it honestly comes down to familiarity.
In my own opinion, a CLI tool is going to be much more familiar ground for engineers -- I wouldn't expect the 200+ engineers at my company to all have read and understood the paradigms of the MCP protocol but I _would_ expect all of us to have a strong understanding of CLI tools and what a good/bad tool is.
With a whole separate LLM? FWIW you can also consider using something deterministic like vale.sh [1] , it's pretty easy to write prose violation plugins [2]
Thanks! Author here, I'll have to take a look. I am all for programmed, deterministic solutions. I hate praying to the rocks we created, begging for rain and not vomit.
TL;DR `hop claude/57680` will cd you into the worktree for the branch.
I've wrote this tool so I can do things like `hop claude/57680 && vim index.ts` and quickly edit a file on a branch without having to ask claude to make small adjustments.
Found this pretty nice to work with when you have a lot of worktrees going and otherwise would need to be tracking down where to cd into a given worktree to find and edit a file on a given branch.
We're using temporal as the foundation for agents running on our platform doing accounting work for clients, it's proven extremely effective and I love how durable workflows are within temporal.
+100 on the technology, but realizing they are also VC funded makes me a little more cautious when it comes to vendor lock in.
reply