Spoiler alert, more than you think you do. I was involved in refactoring a product detail page for an e-commerce app. It was a several month long project as the view is pretty complicated and the only goal was to improve UI performance. Our efforts led to a direct, measurable result in user engagement and an increase in items being added to the cart.
General user engagement is a measurable benefit, regardless if it's for e-commerce purposes or not. Other apps that provide a blazing fast experience set the standard for what users expect out of an app. If your app performs below that threshold, users are less likely to use it. Loading performance is the biggest reason I reach for my Harmony physical remote rather than ever using the app.