My biggest worry is tying the HTML and JS engines too deeply into the desktop and end up with drive-by exploits that can do more than a browser would ordinarily be able to do.
I think this is a valid concern, but as JS engines start implementing more and more of the rich web client working drafts like FileAPI (http://www.w3.org/TR/FileAPI/), those same concerns will be valid for your browser too. I can only imagine that developers of desktop platforms like this are aware of these drafts too and are planning to utilize as much as they can of the functionality provided.
As with most security, our best hope lies in open implementations which can be audited for such problems, with consideration being drawn in during design and implementation stages from security experts. Hopefully some lessons were learned from the ActiveX desktop days!
I like the idea of assigning permissions to applications. Then if browser B runs under user U, its effective permissions would be the intersection of what both B and U are allowed to do. I don't see why Firefox should be able to write anything outside ~/.mozilla/firefox and ~/Downloads
These type of restrictions are exactly what Linux's AppArmor (as well as SELinux, etc) do.
I'm not sure what profile Firefox runs under, but what you suggested would be quite reasonable, though maybe not as default -- You probably want to be able to "save as" to an arbitrary directory, and open files for upload from anywhere too. Though since both of those involve a user dialog, that could easily be a secondary application with its own profile that uses IPC/shared memory/something to pass data to the browser. Smaller target attack area.
