I vaguely remember fixing this in gpedit.msc but apparently it's even easier to fix now:

If you’d rather have File Explorer open to This PC, on the ribbon, select See more > Options > Open File Explorer to:. Then select This PC > Apply.