The Technical Preview fever
Using Technical Previews as a Swiss army knife
Although this topic may resonate more with the B2B folks, I’m sure there are some nuggets that may be also useful for customer facing products.
Short story
The last two years at Nexthink have been an intense roller coaster. Not only because of the company growth and organisational changes, but specially for the pace of innovation we have achieved.
It was a time where we had 12 teams delivering functionalities in private and public technical preview at the same time, meaning to a small portion of our customer base, than available to the entire customer-base.
As you can imagine, this is not sustainable, neither for teams, nor for GTM folks, and let alone customers.
The question is, how do you handle great technical previews?
What it is
Just to clarify the concept. A technical preview, is a particular state within the delivery lifecycle of the product you are building, that is only available for a subset of the entire customer base.
The goal should be simple, for the product team to learn and iterate on the product before is launched into the rest of the world, and for customers to solve a clear problem/need/desire. Normally for customers is a leap of faith, since they now the product might not be as polished as the ultimate version (normally called GA - General Availability).
In our case, customers can opt-in to be enrolled into the technical preview.
Downsides of Over-Indexing
The main problems you may run into when relying too much on technical previews are:
Sometimes as a PM you keep aggravating the problem: although PMs want to reduce risk and deliver something valuable, we tend to overlook what is going on in other parts of the product and forget that there may be other PMs thinking the same as we do. System thinking is a great way to ask deeper questions to really assess the need of a technical preview.
Customers are finite: customers’ time is finite, they will only be able to test as many feature as they have time. This also creates a huge FOMO, because they have a huge cool stuff to try out.
Are you getting quality feedback: this is more related to whom you let to participate in the technical preview. More doesn’t mean better, because you may end up getting customers who either don’t have time to test your feature or the problem you feature solves is not as important for them now. Collaborate with people in the GTM to really bring customers that have struggling moment your feature addresses.
Difficult coordination: when you have so many moving pieces is quite challenging for leadership and people in the field to convey the message of “this is going to be ready for this date”, or this use case cannot be fully achieved because we need the other feature is in technical preview. The message starts to become unclear.
Signals of Over-Indexing
You attend calls with customers where the message is “I haven’t had time to test this feature”. Meaning that that customer shouldn’t have been part of the technical preview in the first place.
We view everything as a technical preview, because we want to avoid risks. Being on a technical preview provides an umbrella for excusing yourself is something goes wrong.
Number of customers joining the technical preview. If you have have feature and your goal is to learn whether you are addressing the problem in the right way, you should have a clear target of who, should be part of a TP. If you have 50 customers on a TP, it’s likely that you are spending too much time on TP and not paying attention to your other duties.
Vanity metrics: PMs like to brag about numbers, especially when numbers are growing. You’re not bringing more customers into your technical preview to increase usage and eventually monetize them. Your goal is to learn whether you’re addressing the problem at a certain scale in the right way, period.
Conclusion
Define the stages of your product and the main principles / guidelines for each of them.
Adjust step 1, after five or six release cycles.
Collaborate with other departments (marketing, sales, engagement) to get qualified customers in your technical previews.
Each technical preview is different. Some may take more, other less. Make sure both, you and your customer reach your goal before moving into the next stage.
Set up the kill criteria for your technical preview. Define clearly what are the conditions that will stop the tech preview to become general availability.