Pilots are pilots because the product is not complete yet. It’s not been fully tested. It’s been conceived (hopefully well) and prototyped (hopefully well) and been through several rounds of feedback (which hopefully yielded actionable insight). However, it has not been sent out into the wild to see how it will behave. That is the purpose of your beta/early adopter test just prior to launch. The pilot, however, shouldn’t be the wild. It should be more like Lion Country Safari.
It is tempting to want to use your pilot sites as beta tests. There is always the pressure of time and budget. But in reality, think of your pilots as a controlled experience; Lion Country Safari. The experience for your pilot testers should be new and exotic (you’ll see giraffe babies/this solves a problem in a new/better way). Expectations about the pilot experience need to be articulated up front (animals don’t always come up to the cars/the collaboration feature is not active). Participants’ safety needs to be ensured (don’t roll down your windows/your data is backed up here), and there needs to be a guarantee that if something should happen you will be standing by (our rangers are never more than 500 yards away/Your support person is Jill. Here is her contact info.). As you build out, or participate in, pilots this spring, here are a few ways to increase your knowledge about your product and the experience of using it.
Your pilot participants just joined your product team.
Context is important. Let your pilot folks know where you started and where you are in your product design. Let them know what you expect to learn with this pilot. Ask them to comment on the plans to date and if they want to learn other things. In a world where resources are dear, your pilot testers are your product development arm in the world. They’re not guinea pigs; guinea pigs can’t tell you how they feel. They are trusted advisors, and they are your best candidates for early adoption.
Be specific about implementation.
Take this product. Use it for two months. Let me know what you think. This is the standard pilot plan for many, many products, but it is typically too general to yield meaningful insight. Instead, try letting your pilot participants show you where they think your product will work best for them. Ask them what success with your product looks like to them. Gain a deeper understanding about how they want to experience your product.
For instance, an easy way to do this is to have your participant take her syllabus/lesson plan and have them insert two activities/modules/games/assessments/whatever into that syllabus/plan. Ask them what success would look like (early intervention? More students seeking help? Fewer students seeking help?). Understand what success looks like to them, not to your business plan. Sometimes, it’s not just the grade. Ask them to implement that strategy--their strategy--with their students. You will not just learn how your product performs but you will understand how you customer wants to experience your product and how it performs in that circumstance. You might just find out that they have a better implementation model.
Make participants detectives, not judges
Let’s face it, the pilot experience is shaky at best. A major piece of onboarding instruction is missing. There is a bug in the feedback loop. It doesn’t work on Safari. We need our pilot participants poking and prodding and asking why. We want them to sleuth in the product and around the experience and we want to know what they think. We need their insight.
The reality, however, is that we too often ask them to judge. Was this good or bad? Was this helpful or not? Was this too hard or too easy? The answers to these questions are not actionable. They don’t require deep thinking. They are too easily answered with the same generalized answers, “Some was good. Some was bad. I didn’t like …..”
Pilots don’t have to be huge, but they do have to be smart. Understanding how your customer wants to experience your product and how your customer understands the success of your product is more important that understanding that pilot testers would like more selections in the test bank, additional videos or better Power Point presentations. Because the first will gain you market share.
EyeLevel Educational Consulting specializes in customer-centric product testing and go-to-market strategy. Please give us a call. We are happy to help you discover and hone the real value in your products.