What actually distract people are boilerplate/composability, lack of fancy types, package publishing openness and slow-paced community. Don't get me wrong, I like Elm, this is just my observation.
The boilerplate issue is frustrating. It sometimes seems like the Elm resources out there all stop at toy examples; SPAs with only three or four different screens without components that need to manage internal view state.
Really I think it's just the stage of adoption that the language is in. It really is cutting-edge in that way and everyone is figuring out the best way to do things. It's early, early days.
In my current project, I've used templating to solve this. I ended up with a huge Elm main program that was 90% case-statements to call the appropriate update, view or other functions (eg- passing JSON data from the back-end server to the appropriate places). I got cranky enough with it that I made a templates version of it using the doT.js library and a script.
Now I can specify the subscreens I want in my application in a JSON file and have a script take that and my Elm code template and spit out a perfectly readable main.elm file which then gets compiled as normal. It's actually turned out really well for my purposes.