You guys suggest using Mixpanel to measure user engagement via split tests, but Mixpanel doesn't have a good way to split test!
I've resorted to hacking something together using properties. I use the property name to represent a segment/test name and the values to represent the splits. This leads to having a ton of properties, which the UI isn't designed for.
An A/B testing tool is on the roadmap, but we just haven't had time yet.
The best way to do tests right now is by utilizing funnel properties. You can test the whole length of your signup funnel and just add a 'button-color' property.
This lets you see the difference in conversion rates between your two treatments pretty easily.
I would also like to do this for one off events. The important thing for us is the ability to view the event metric across property values (intersecting segments):
I've resorted to hacking something together using properties. I use the property name to represent a segment/test name and the values to represent the splits. This leads to having a ton of properties, which the UI isn't designed for.
ex: