Here in Poland, I've already had several banks and at least one insurer send me CD-ROMs. Never heard of anyone sending USB sticks before, but I'm not surprised. The problem is, approximately no one owns a CD/DVD reader anymore, and there are no modern read-only physical media. With SD cards also going the way of the floppy, USB stick is just about the only medium you can hope most customers have means to read.
AFAIK notch is just declaration of intent, like with floppies and magnetic tapes - it's politely asking the reading device to not write to the medium, and it's up to the device to respect it (or up to user to not bridge the notch with a piece of tape).
Still, actual write-once (or read/write until hardware fuse is triggered, read-only afterwards) SD cards should be possible to make.
Since SD cards and USB sticks are both just computers you plug in to a network port on your computer, they could definitely make write-once SD card controllers.
Aside from the local tax collector, which insists on snailmailing me a copy of all correspondence even though they also sent everything to me digitally, I can't even remember the last time I received any documents on paper, and I'm in the EU.
Neither of those sources back up your claim that paper (or a USB drive, for that matter) is required in certain cases. The court case cited in your second link even lays out the conditions under which a website can be considered to satisfy the requirements.
Why the bank decided to use USB for this purpose, instead of paper, is very strange.