I think the obvious limitations of Python is a big reason, but probably not the main reason why Pandas isn't orthogonal. The reason why Pandas is such an ungodly mess is because it must be, in order to be even halfway efficient. When you do try to compose things, or even have the audacity to use a python lambda or an if statement or whatever, you suddenly suffer a 100x slowdown in performance.

Julia doesn't have these problems, and I've found it so much nicer to use for data analysis. You can even call Python libs, if you really have to.