You have to nip these habits in the bud. Might require changing company if it's a well established thing there though.

I know when my meetings start in the morning. If someone books a Monday morning meeting with me after I go home for the day Friday they will learn really quickly that I won't see their meeting invite until its too late. Same for any other days.