My understanding is Signed Exchanges only solve "I want to allow another server to handle requests for my HTTPS server" and not "I want to guarantee the integrity of content coming from my HTTPS server". Did they expand the spec to somehow address the latter?

You could obtain an HTTPS certificate and then only use it for offline signing. That would fulfill the same purpose.

But the platform would not know that this is a code signing certificate - unless something specific is designed.

HTTPS certificates are already treated as a kind of code signing certificate by browsers.