Much of the differential narcosis effect is actually due to carbon dioxide rather than nitrogen. CO2 will interfere with your ability to think clearly even worse than nitrogen. If you're relaxed and breathing deeply then you generally won't accumulate much CO2, but then if you have to exert yourself it suddenly hits you and can have severe effects.

The solution to both N2 and CO2 narcosis is the same: put more helium in the mix.