Most physicists don't believe that infinity can actually exist in the universe.
Put another way, the program which searches those works of art in the digits of pi will never finish (for a sufficiently complex work of art). And if it never finishes, does it actually exist?
That's a completely different issue. Using math to solve physics problems deals with physical models. Models are imperfect and what kinds of math they use is completely separate from asking "does infinity exist in our actual universe".
To answer that question, you would have to dismiss with experimental evidence all models people can come up with that try to explain the universe without "infinities". It's neither completely clear what that would mean, nor whether it's even in principle possible to determine experimentally (it's also most likely completely irrelevant to any practical purpose).
Put another way, the program which searches those works of art in the digits of pi will never finish (for a sufficiently complex work of art). And if it never finishes, does it actually exist?