Got it, thanks! The whole rectangular felt like a single object, and I always clicked in the center, so missed the other parts.