… you’ve HAVE A TESTING SOLUTION, this helps complete TEST HYPOTHESIS WITH MINIMAL CHANGE
❖ ❖ ❖
What are the alternatives and what do we measure?
Therefore:
Build the test – include the current as an alternative, use business metrics and specific user actions to drive your choices on what to measure. Don’t be afraid of measuring more than one thing.
Include the current alternative
- alternative a: current meal plan
- alternative b: current meal plan with popup
Use the business metrics that the problem, i.e. the one(s) that you want to improve (KNOW YOUR IMPORTANT BUSINESS METRICS), and metrics that reflect the specific user actions the hypothesis depends on (IMPROVE THE BIGGEST OBSTACLE)
- how many people return a 2nd time
- how many people get to the grocery list page
Often we need to track two more more metrics. Why? Because there’s a dependent relationship and we want to understand how we’re changing both elements (i.e. use of grocery list page, short-term retention)
Because of this and since vanity only measures 1 metric and we often need to track 2 or more, we will:
- measure the smallest feedback loop in vanity
- measure the larger feedback loop using a separate views
Steve Sanderson 