what's next?

Archive for the ‘Lean Startup’ Category

Ad-Hoc Usability Testing With Craiglist Users #leanstartup

In Lean Startup on March 20, 2013 at 4:24 am

At Food on the Table, feedback from real users is the life-blood of how they do lean startup. When we needed qualatiative feedback from new users, one way was to hire people off of craigslist – a form of ad-hoc usability testing.

This question has come up so often, I promised to share the answer here:

Hi Steve:

I’ve got a question for you on doing usability testing via random folks on Craigslist.  Do you have an example of the ad that you guys use?  We’re going to shamelessly steal this concept, but we want to make sure we don’t write something that only attracts uber geeks.

The answer, abstracted a bit:

  1. First, we need to pull together some information for the ad. Much of this should be derived from the experiment you’re working on (i.e. as part of the problem + hypothesis + experiment you’ve got – right?)
    1. what is the context of your experiment, e.g. “website” or “iphone app”
    2. what does your hypothesis + experiment say about how to filter or categorize the participants, e.g. for Food on the Table, we may have wanted to filter / categorize whether they cooked, whether they shopped, whether they used coupons. Turn these into individual questions about the participant.
  2. More broadly, given your product, what is a few words associated with your product, e.g. for Food on the Table it may have been “recipes, meal planning and grocery shopping”
  3. Now compose the ad:

title: <context> Usability Testers For Hire

body:

We are conducting a usability study, which means we would like your opinion about a new <context> we are developing that has to do with <few words associated with your product>.

The sessions last approximately an hour and pay $30.

Please answer the following questions with your response.

1. Name:

2. <questions about the participants>

12. Would you be available to do a research session at a downtown Austin office? Y/N

13. What day and time work best for you? (most sessions will take from 1-1.5 hours)

TIPS

  • Often we had some additional questions in the email exchange – one of the big questions is “are you a developer” or similar; in our experience, developers make horrible usability testers – they want to critique the implementation, etc.
  • We don’t include the name of our product or company because, as you may have guessed, we would like to guarantee that this is truly a new user, i.e. they don’t look ahead.
  • Conducting these usability interviews is a distinct skill – without that skill the value of the learning is at risk. I hope to offer some specific tips on this later, but key points are:
    • don’t lead the witness – ask open-ended questions and wait… wait… wait…
    • watching what they do is as important as hearing what they say, i.e. body language communicates tremendous information – such as hesitation with a mouse, while verbally they’re assuring you it’s all fine.
    • try to get at least 2 people in the interview with the participant
    • get interviewers to sit around the participant so they can get different views
    • get all the interviewers to capture their thoughts during the process
    • don’t waste effort on trying to formalize results, instead debrief immediately afterwards with all the interviewers and the relevant other people in the company
    • everyone (yes, everyone) should participate as an interviewer regularly – developers, customers support, marketing, bus. dev, CEO – everyone.

Here’s a version of the ad we posted on craigslist:

Website Usability Testers For Hire

We are conducting a usability study, which means we would like your opinion about a new website we are developing that has to do with recipes, meal planning and grocery shopping

The sessions last approximately an hour and pay $30.

Please answer the following questions with your response.

 1. Name:

2. Age:

3. Do you do all/most of the cooking in the house? Y/N

4. Do you do all/most of the grocery shopping? Y/N

6. How many children under the age of 18 are in the house?

7. What grocery store do you go to buy your groceries?

8. How often do you use the Internet?

9. Would you be available to do a research session at a downtown Austin office? Y/N

10. What day and time work best for you? (most sessions will take from 1-1.5 hours)

Moving on from Food on the Table & What’s Next!

In Business, Lean Startup, Personal, Product Development on March 14, 2013 at 9:05 pm

It’s been a great 3 1/2 years at Food on the Table, however it’s time for me to move on.  As you’d expect with a team and investors of this caliber, everyone has been very supportive of my decision. While this next phase will be without me, I remain confident that the work at Food on the Table will lead to great results.

Working closely with Manuel Rosso and our team for the last 3 1/2 years has been a tremendous experience. Out of that I’ve become deeply passionate about my Lean Startup experience & knowledge as well as having radically evolved how I do product development.

Now, the fun part! I’m using this time to collect evidence for two alternatives:
  • I’ll take on one or two consulting projects (Austin or remote) while validating some startup ideas; or
  • I’ll sign on in a new leadership role with an Austin team that is creating new products in conditions of extreme uncertainty (i.e. typically an early stage startup, but possibly a team in a larger org).

Regardless of which alternative, I’m looking forward to new challenges in building products that will continue to evolve my product development approach and give me the right dose of hands-on technical work.

As always, you can reach me at by sending email to any address at stevesanderson.com, e.g. harvestthisemail@

How would you benchmark the productivity of a lean-startup product development team?

In Lean Startup, Product Development on February 9, 2012 at 3:59 am

How would you benchmark the productivity of a product-development team in lean-startup?

Why?

In my experience at Food on the Table, we’ve been much more productive than any of the other early-stage startups I’ve been involved with. I believe that one of the causes is our pervasive use of lean-startup & feel as if we’re on to something important here. So, how can I (dis)prove this?

Uh, really?

I recognize the some of the inherent issues in my question:

  • no accepted standard to compare productivity
  • small sample size
  • what’s the impact of environment & how can it be separated
  • what does lean-startup have to do with this at all
  • etc.

… despite these issues, I’m still asking the question.

Ok, you might…

So if you’ve got some ideas how you’d benchmark product development productivity, and maybe how’d you’d take that to the next step, how to compare, then leave a comment below.

Thanks!