You need the app to download the pass (AFAIK) but it goes into your Google/Apple wallet after that. I have the app disabled (so it can't run in the background, as much as possible), and I only use it to get the pass.

I wish they'd just let me download a pkpass file, but what can you do.