Building a culture of experimentation

In my experience, one of the most important changes you can make to improve the culture within your business is to create a focus on learning.

When I say “learning”, I don’t mean doing a course or reading the latest developments in a particular field. Instead, it’s a change of attitude and approach.

Change the solution mindset

Think about the last meeting you had where you talked about what everyone was going to “do”. Were the people in that meeting, including you, able to extract their key assumptions and hypotheses from what they were planning?

Did anyone set out what they wanted to learn from the next piece of work, or were you all busy deciding what needed to be done and when it was going to be delivered?

Let me tell you, thinking about these things instead of just deliverables and due dates is one of the most important changes you can make to the way your team works. And when you achieve this change, you will see a dramatic difference in what people do and what they learn as a result.

Here’s a simple example…

How product design usually works

Imagine this scenario: you’ve decided that you want to design a new registration experience for your product. 

After some research, you’ve decided that it doesn’t look as good as the experience from other products. At the same time, you also want to improve the conversion rate of visits to registrations because Google Analytics shows you a dropout that feels low.

Your amazing Product Design team has had a good look at what is best-in-class and put together some great scamps. You all like what you see, so you agree on a course of action. 

The plan is to:

1. Do some internal testing on the options at higher fidelity.
2. Build some variants to do some multivariate (MVT) testing.

Does this sound familiar so far?

How product design should work

In the all-too-common process of ideate, design, test, does anyone stop to think about what they want to learn from the exercise?

• Have you documented your learning points and how you intend to measure them?
• Will you be able to refer to your initial questions in a week, a month, or 6 months, when you have a question about your registration experience? 
• How are you centralising what you have learnt so that others can benefit from the new knowledge you have acquired?

Using the registration experience scenario above, I have created some example questions that I expect my teams to have documented upfront before doing anything.

• What device type has the highest dropout?
• What is the site behaviour of a successful registrant compared to that of a dropout?
• What does a user do after registering?
• What do people feel when they land on the registration page? Is there something qualitative that we are missing?
• What type of user are we targeting to register?

Alex Sherratt (one of the best Product Managers I have worked with) kindly reviewed this blog before I published it and, as he quite rightly points out, these questions are fundamental to rigorous problem definition. He also expects that – with high-performing teams – the answers should already exist. 

The key is to ask the right questions so that you can bring all of the relevant answers together into one place to get the full picture.

Discover and document

To create a culture of learning, there needs to be a shift from a focus on activity to a focus on being rigorously inquisitive. Then, the key to turning that focus into positive change for your business is to create a very high expectation of documentation and insight capture.

If you click this link, you can download a very simple spreadsheet that I have used repeatedly throughout my career. It always helps me remain very focused on why I’m doing something and what I need to get out of the work. I have pre-populated the first row based on the hypothetical example above. I hope that helps!

I’ve borrowed the principle of Learn/Measure/Build out of the pages of Lean Start-Up, but given how much I hear “Lean” being talked about, I’m always surprised by how few systems and behaviours are implemented to support it. Worse still, there is often very little rigour in its day-to-day implementation.

Put it into practice

Using the spreadsheet I’ve provided, I would encourage you to try an exercise with a project you’re currently working on.

Re-examine the project and capture all of the insights and learning that you want to take away from it. Then, try to figure out how you will get this information. 

Remember, the answer isn’t always to ship code. In some cases, it could be as simple as picking up the phone.

In my last role, one of the PMs recognised that they needed to speak to a real customer. They called a few customers that matched the profile and quickly got an answer without wasting an expensive engineering resource. Simple!

If this blog has piqued your interest then please keep an eye out for the 2nd one coming out shortly which will focus on identifying the pillars of a successful experimentation culture and giving you some ways to assess your own organisation’s readiness to adopt it.