By what definition is this local first? You use firebase and the client is not open source or at least downloadable. The definition is that i can keep using the app if your company goes out of business, which is not the case.
It's local-first in the sense that all your financial data is local (synched across your devices), and backed up to your dropbox. Firebase is for auth (e.g. for us to know who you are, manage subscription etc). None of your financial data is kept there. We don't have a decent way to set up and track subscription without that central piece (firebase).
In other words, when the company goes down, there are two concerns:
1st whether you will have access to your data. Yes, and no one else has access to it.
2nd whether you will be able to use the app. We plan to open-source the app when/before that happens. This part, you have to trust us. We don't see an easy way out of this, yet.
I understand your hesitation. But there are already user using it on a regular basis, who care about privacy but trust us to have the auth centralized. It's not for everyone, but it's what we plan to do, at least for near future.
I am confused this article does not talk about taint tracking. If state was mutated by an agent with untrustworthy input the taint would transfer to the state, making it untrustworthy input too, so the reasoning of the original trifecta with taint tracking is more general and practical. I am also also investigating the direction of tracking taints as scores rather than binary as most use cases would otherwise be impossible to do at all autonomous. Eg. with sensitivity scores to data, trust scores to inputs (that can be improved by eg. human review). One important limit that needs way more research is how to transfer the minimal needed information from a tainted context into an untainted fresh context without transferring all the taints. The only solution i currently have is by compaction and human review, if possible aided with schema enforcement and optimised UI for the use case. This unfortunately cannot solve encoded information that humans cannot see, but it seems that issue will never be solvable outside alignment research.
PS: An example how scores are helpful: Using browser tab titles in the context would by definition have the worst trust score possible. But truncating titles to only the user-visible parts could lower this to acceptable for autonomous execution if the data was just mildly sensitive.
Have you seen the DeepMind CaMeL paper? It describes a taint tracking system that works by generating executable code that can have the source of data tracked as it moves through the program: https://simonwillison.net/2025/Apr/11/camel/
Of course. CaMel was a breakthrough and especially promising as similar execution architectures were discovered from the reliability angle too (eg. cloudflare code-mode)
I would consider the runtime and capabilities part of CaMel an implementation exploration on top of the trifecta + taint tracking as general reasoning abstraction.
My hope was that there would be an evolution of the the more general reasoning abstraction that would either simplify or empower implementation architectures, but instead I do not see how metas rule of two adds much here over what we already had in April. Would have loved for you to add one sentence why you thought this was a step forward over taint tracking, maybe i am just missing something.
Totally. I think the original "Lethal trifecta" post by OP only pertained to data exfiltration and never included changing state (maybe was implied by sensitive data access).
Can you explain what you mean? How is Chesterton's fence applied to AI security helpful here? Are you just talking about not removing the "Non-AI" security architecture of the software itself? I think no one ever proposed that?
Right, what got me going is the reduction of plenty cyber security concepts into a simple "safe" label in the diagram.
So what I meant is that before you discard all of the current security practices, it's better to learn about the current approach.
From another angle, maybe the diagram could be fixed with changing "safe" to "danger" and "danger" to "OMG stop". But that also discards the business perspective and the nature of the protected asset.
I am also happy to see the edit in the article, props to the author for that!
And to address the last question, no one proposed that right now, yes. But I was in plenty of discussions about security approaches. And let me tell you, sometimes it only takes one sentence that the leadership likes to hear to detail the whole approach (especially if it results in cost savings). So I might be extra sensitive to such ideas and I try to uproot them before they bloom fully.
Hmm, what do you mean by current approach? This is new territory and agent safety is an unsolved problem, there is no current approach, except you mean not doing agent systems and using humans. The trifecta is just a tool on the level of physics saying "ignore friction", we assume the model itself is trustworthy and not poisoned most of the time too, but of course when designing a real world system you need to factor that in too.
Yes, by current approach I mean security best practices for non-LLM apps. Plenty of those are directly applicable.
And yes, LLMs have some challenges. But discarding all of the lessons and principles we've discovered over the years is not the way. And if we need to discard some of them, we should understand exactly why they are no longer applicable.
EDIT: I know that models need to omit stuff to be useful. But this model omits too much - claiming that something is "safe" should be a red flag to all security workers.
Hot module replacement masks a lot of annoyances for end users. Yes its more instantaneous than reloading a page and relying on urls for all of the state and I am not advocating hard for abolishing HMR anymore, but it would be nice if we still used way more url state than currently the case. Browsers will also hibernate tabs to varying degrees, server sessions expire all the time, things are not shareable. The only thing that works as users expect is url state. One thing i absolutely hate about ios apps is how every state is lost if i just have the app in the background for a few seconds, this even applies to major apps like youtube, google maps, many email clients etc. Why do we live in this stupid world were things are not getting better, just because someone made things more convenient for developers?
PS: and i curse the day the social media brainwashed marketing freak coined the term "deep link" to mean just a normal link as its supposed to work.
Its kind of naive satire that looks silly on second thought. Recall was not bad because of the concept but implementation details, rollout communication and of course the microsoft part. Recall by an open entity with data ownership, security and transparency would have none of those issues and its just a new take on the universal desktop search that is enabled by ai being able to utilise pixels. I refuse to be shamed by e2e encryption freaks that i want to be able search anything I encountered and having universal data control and ownership vs locked in app silos.
All of that data sent to a third party server is going to be public on the Internet at some point. Security? Don't make me laugh. Countries that required government IDs to participate online have already made this mistake and those IDs have been leaked. Just because it's open source or run by $NOT_MICROSOFT won't make it any safer.
The problem with other people consenting to it is that it makes every one else less safe. People get compromised and scammers can use that compromised data to work against people who didn't share their data with the, "Benevolent Open Source Recall Service."
This is correct - it was all on-device, with security guarantees that were instantly proven incorrect. Microsoft withdrew Recall, then brought it back with a newer, more secure implementation that was also proven insecure.
It also claimed that it wasn't going to record sensitive information but it did, to the point where some apps, like Signal, used available Windows APIs to set DRM flags on their windows so that Windows wouldn't capture those regions at all.
What Microsoft could have offered is an easy-to-implement API for application developers to opt into (but users can opt out of), and a blanket recall-esque toggle that users can apply to applications without explicit support. Applications like Firefox or Chrome could hook into the API to provide page content to the API along with more metadata than a simple screenshot could provide, while at the same time not providing that data when sensitive fields/data is on the page (and possibly providing ways for the HTML to define a 'secure' area that shouldn't be indexed or captured, useful in lots of other circumstances).
But, as with everything AI, they don't want users to want it; they want users to use it regardless of whether or not they want it. This is the same reason they forced Copilot into everyone's Office 365 plans and then upped the price unless you tried to cancel; they have to justify the billions they're spending and forcing the numbers to go up is the only way to do that.
I have to wonder what edge AI would look like on a laptop. Little super mini Nvidia Jetson? How much added cost? How much more weight for the second and third batteries? And the fourth and fifth batteries to be able to unplug for more than a few minutes?
They're called NPUs and all recent CPUs from Intel, AMD, or Apple have them. They're actually reasonably power efficient. All flagship smartphones have them, as well as several models down the line as well.
IIRC linux drivers are pretty far behind, because no one who works on linux stuff is particularly interested in running personal info like screenshots or mic captures through a model and uploading the telemetry. While in general I get annoyed when my drivers suck, in this particular case I don't care.
Conceptually a feature similar to Recall doesn't have to involve sending any data to third parties. It should not need to be a service just a piece of software running locally, doing OCR and full text search indexing using local compute.
Incidentally I often tell my friends I run an app on my phone that captures my location 24/7 and they would initially sound horrified. But then I tell them all my location data is not sent to anywhere on the Internet, and ask them specifically what is horrifying about it. There is none.
> I often tell my friends I run an app on my phone that captures my location 24/7 [...] But then I tell them all my location data is not sent to anywhere on the Internet
Your phone is on the Internet.
It takes only one attack (for instance, someone sends you an image which exploits an RCE on the image decoder and then chains into a privilege escalation exploit), or a careless mistake (like marking the wrong folder to be synchronized), or even an automatic update of the app (which adds a helpful "sync across your devices through the cloud" feature or similar), to have all that saved location data copied elsewhere.
You can't leak what you don't have; if you never saved your location history, there's no risk of it being leaked after the fact.
>if you never saved your location history, there's no risk of it being leaked after the fact
Very Buddhist in principle. I still prefer having my GPX tracks though, because they're useful to me, as well as notes, journals, logs... Local security is a separate question, and it's light years apart from stuff like Recall.
You wouldn't rather have only some of your location recorded? I don't understand the appeal of saving all data all the time.
It's akin to going to a concert and recording the whole thing, versus recording a small bit that feels memorable, so you can enjoy the rest of the experience fully present.
as a total aside, how do you know what they're going to pay at the concert before they start playing and you know it's your favorite song? Wouldn't you miss the beginning of the song?
It's a good total aside, my analogy was not great.
I went on Sunday, and she announced what she was playing. Otherwise from the initial notes it's easy to spot what's coming. Of course you end up with an imperfect recording, but it's good enough for the memories, I guess.
(I actually wanted to record the 10-minute jam session via Apple's Voice Memos but didn't notice it wasn't recording, because there's no feedback to when you press the button, and red-on-dark is easy to miss.)
Wasn't there a HN post a few weeks ago, describing how your phone's location can be tracked without anything installed and without leaving any trace on your phone? I think it was an exploit of CSS7 protocol used by networks?
The problem is that the data has to go somewhere. If you don't have the compute power locally, you have to send it to a server you control. At a point, this starts to break down because your attention to detail isn't sufficient to protect other operators. I think there are some happier mediums, but I wouldn't be as strident as saying there is no risk even if this is stored locally.
“I store all my location data and I see no problem because it's stored locally” is the new “I store all my passwords on a post-it and I see no problem about it”.
The more you store, the higher the risk, simple as that.
You have a convincing argument for not taking photos and not writing notes down. In fact, why write anything down? Remember everything like Socrates asked people to.
I use Arc which I've recommended on HN a few times. https://news.ycombinator.com/item?id=38662095 As a power user I find that it could be buggy but the developer behind it responds quickly on the forum. The developer also made another app with fewer features but FOSS: https://github.com/sobri909/ArcMini and it's been on my TODO list to use that as the basis to create my own location tracking app (I have some UI ideas for such an app in my mind).
The battery life impact is quite large. The iOS battery tool reports on the order of 5% to 10% but I think subjectively it feels much more than that. Getting GPS signals itself is IMO a bigger power draw than the app writing some time series data into a SQLite database (it defers expensive processing until the app enters the foreground).
> All of that data sent to a third party server is going to be public on the Internet at some point.
Windows Recall is on-device only (for now). The captures stay on device in a local sqlite database, and all the processing is done on device on the NPU.
I don't get the deal with requiring govt Ids. Back home the government has an OpenId provider and you could link your governmental account if you wanted without leaking your Id/DL/Passport which has data that's considered more private than your Id number.
To me it felt like just shoving more AI for the sake of it, pretending that there is a problem to solve while making no case for it, and boiling oceans along the way as an acceptable externality.
Taking screenshots of your whole desktop every other second and sending it to a third part, just in case their OCR and cheap search built on top of it might come up with something useful? I haven't found a single situation where I would rather take that above more conventional and established approaches (browser history search, bookmarking, file management hygiene, etc)
The problem of "where did I see that" is something I suspect most people have encountered before. How that's actually done, though, is the devil. The vision -- semantic search of human experience -- is cool. The implementation -- always recording cameras piping every minute of your life to TotallyTrustworthyPeople's servers -- not so.
General chatbots are great for things they have general data about. "What was that movie where..." type things. They don't help with individualized information, unless you feed the same type of information as Recall type solutions gather anyways. Perhaps you don't have much individualized information, or perhaps you just remember it all very aptly - it shouldn't be hard to imagine differently though.
My main usage problem with Recall type solutions is less with lack of something to promise and more with lack of ability to deliver. Especially for local-only solutions. The concept can be great as can be, but it needs to be damn near foolproof to beat out how much we already remember.
>I haven't found a single situation where I would rather take that above more conventional and established approaches (browser history search, bookmarking, file management hygiene, etc)
But all of those are terrible for the use case at hand. It's like searching for the book that contains a passage without being able to search passages, rather searching by title, author or subject.
Aren't screenshots just one implementation specific variant of the possible solutions? Recall immediately made me think about a universal API that all kinds of applications write history to. That data can then be searched, or you could possibly even have global undo/redo.
Screenshots are just "easier" to use, because you don't need to implement anything for individual apps. "Easier" only if you have data centers full of compute and the capital to very inefficiently throw a bunch of silicon and electricity at the problem.
In the end, age comes for us all. I pay for the Mac version of this software. I bought concert tickets to something, but couldn't find the confirmation email, but did find the credit card charge. So where did the tickets go? Did I hallucinate buying them? In the end, I rewound to the late night buying session to find that I'd misspelled the email address and the platform didn't confirm my email before sending.
Of course, that would never happen to you, but it saved my bacon.
For what's worth, I have a "form history save" add-on for Firefox that retains this kind of info for a time, that doubles with a password manager (bitwarden) which I suspect would have caught it as well.
ostensibly - and I'm in your camp, so this is just devilish advocacy - the idea is that you will be wondering, "who said X to me the other day?" or "what was that article I saw about Y?" and you'll be able to just ask one unified search interface instead of searching in 5 different messengers or poring through your browser history.
I don't know that I need that. Certainly not at the privacy cost involved.
>and boiling oceans along the way as an acceptable externality
Can we stop this? It's like saying it takes 2500 litres of water to make a hamburger. Distorting reality to make it seem worse than it is implicitly justifies it.
I refuse to be shamed by AI and surveillance freaks who can't be bothered to take notes of important things and instead demand their and by extension "my" daily computing habits are recorded "just in case."
you can turn off the features, no one forces you. but locked data is not available to me so you influence my fundamental freedom and right of data ownership for my whole life when i only force you to once click “i don't want ai” in onboarding of an app. everything else is just implementation details. im not talking about specific implementations.
I still recall (duh) a post here from a guy who literally made a recall feature on his mac way before them. I would love to find that post...
It was screenshotting all the time, storing that locally, and then you could ask it any questions and roll back to that moment. Processing was also fully local.
I cannot even tell Windows that I manage updates myself, how can Recall not be an insane paternalism fail?
In my opinion the product owners of Windows lack the maturity do implement anything like Recall responsibly. Perhaps there is pressure in the background, but as the consumer that isn't my problem.
I could see something like Recall be helpful for a lot of users, but the politics of Windows would need to be changed considerably.
e2e encryption freaks should know about the limits of encryption for that matter.
Also I still don't have a Microsoft account. A private one at least.
I think you hit the nail on the head. Recall was problematic because of the lack of information provided when they announced it. How would it handle sensitive details like banking account pages. Can developers opt out for best security practices? How is the managed in an enterprise environment?
Once they delayed and got all their ducks in a row it's a much more solid feature. Not for everyone, but a good way to leverage your PC as a source of information that you can search without having to save everything.
The post reads a little childish to me. Very "Micro$hit are greedy and want to steal your data" level of criticism.
What irritates me most about all these unwanted "innovations" is that they always make them opt-out. "If it's opt-in, nobody's gonna use it - and it's such a flawless feature, we should be shoving it down everyone's throats"
Recall IS opt-in. Personally I don't get the big deal. Opt out stuff is just a simple toggle. If they were shoving down everyone's throat you wouldn't be able to opt out of it. Like ads on Google Discover feed.
Recall is absolutely horrible in concept irregardless of implementation. Would you accept a big brother security camera over your shoulder recording every minute of your life? No? That’s the equivalent of what Recall is.
i have security cameras in all my rooms recording all the time, so yeah. you mix up the features with the saas and corporate hellscape that currently provides us these products. i want all of this but with secure and local software and hardware that i truly own and control.
Recall would be fine, I guess, on Linux, because we could be fairly certain it wouldn’t be slipped in under the radar. But… actually installing it seems like a pretty bad idea. Too big a target.
Its a pure unhidden giveaway: ai is not about anything you want as a user, its about busting the last shreds of privacy and security for the vast majority of computer users
You're criticizing a joke. Worse, you seem to be aware that you're criticizing a joke, and still went through with it.
I also disagree with your premise: Recall by an open-source entity would have many of the same problems. The threat model for most people isn't that Microsoft might tailor ads to their interests. The threat model is that you're giving that ransomware gang, or an abusive spouse, a new tool with devastating capabilities.
Even for people legitimately worried about law enforcement / the government, the same applies. You're gifting your adversary a database of everything you've ever done that understands context and can literally be queried for "just show me the bad things". It's slightly better if it lives locally rather than in the cloud, but it can be used to nab you just the same.
You say it as if criticizing a joke is obviously something not to do for some reason? It is not like the joke does not imply something the author tries to say, otherwise it would not be on the front page. That aside, you overstate the difference to what we have today dramatically. Your browser history is already that and look at what controls firefox built on top to let you manage it. You can pause it, limit it, exclude pages etc. An adversary can root your pc and enable key logger and screen recording if they can break into an encrypted database, the delta to what adversaries already have is not that big, but the unlock for users and agents is quite big.
if operating systems had just put a bit more time into the clients and not stopped any work in 2010 or so, webdav could have been much more, covering many usecases of fuse. unfortunately especially the mac webdav and finders outdated architecture make this just too painful
Every time i try to switch to a libre android i encounter the same blocker of not being able to do a full backup and restore with all app data and full control without hacky, weird third party apps that don't work, just as i can do on any linux in the world. I don't understand how the android ecosystem and everyone working on this is just ignoring the data.
Same here. For me the biggest bummer with GrapheneOS is that the promised new back up system is still not even on the horizon and was promised a gazillion years ago.
I use a self-hosted Nextcloud and sync all contacts, photos and calendar with it. Having full native support for all Android apps would be pretty cool though.
googling for seedvault result: “Seedvault's app-specific restoration capabilities are limited, and it does not directly handle WhatsApp's chat backups, which must be handled by WhatsApp its” I am looking for filesystem level data control that can backup everything without relying on something in the control of an app developer.
I think it depends on the way applications store data. Local WhatsApp backups may be backed up by Seedvault, but not the database. I'm not sure to be honest, though. I'm so paranoid I also make app-level backups at regular intervals so that nothing breaks or is lost.
netflix falcor. the graphql hype killed a much better alternative for many usecases. there were only a few missing pieces and improvements such as a proxy based adapter layer for popular frontend frameworks. Im now the lonely last user hoping to find a way to reboot development
Surprised a basic article without much deep insight gets to the front page. nango is one of the hundreds of integration unification attempts i saw come and go, they all get enshittified, bought and closed or severely limited by the new evil daddy. They all have licenses that do not really allow the community owning the platform mainly to try pretending they are not giant lock-in machines that pretend to save you time until you inevitably start fighting the system because their abstractions don't make sense for whatever your product develops into.
while work on pure algorithms is invaluable i always feel work on knowledge augmented algorithms has lots of untapped potential. two examples: recording key events like move and delete on a more fine grained timescale or directly from editors and then storing those as mutable metadata in commits that is only allowed to be used for diff generation. as its provable if diffs are technically correct these do not weaken the consistency guarantees while adding helpful context. they are also highly compressable and pruneable. another one is optimizing diffs for consumption by llms and let those generate for optimal human readability.
Do you have examples of any of these ideas being implemented? In general I agree, there’s so much opportunity for these “knowledge augmented” algorithms
reply