I'm glad to know I'm not alone in this. Early mornings and late nights are both good... but during the typical work day I find it nigh on impossible to really be productive at the office.
The problem is the people who tap you on the shoulder, or worse... those who know they shouldn't disturb you so hang quietly around your desk until you give up and make eye contact or otherwise acknowledge that they're there.
They succeed in disturbing you without actually disturbing you, it's great (for them).
It reminds me of this talk by Jason Fried, which is so obvious but no conventional workplaces solve these issues: http://www.ted.com/talks/jason_fried_why_work_doesn_t_happen...