In this series of blogs, we will be sharing some of the challenges posed by our clients and lending our perspective on best practices and how to navigate common concerns.
Client challenge – What is the right sample size for accurate insights?
In recent months it’s a question that is being asked of us more and more: ‘Is this a good enough sample size for the insights to be accurate?’ – or questions to that affect. When conducting a piece of consumer research or customer insight, brands are developing a preoccupation with size, but is it justified?
Wind the clock back several years and it’s probably not something that was asked very often. The evolution of large data sets has meant businesses and brands can now expect large amounts of customer data, and not just in depth but in breadth too. Many data businesses (Starcount included) talk about the size of their panel or the volume of consumers they have at their finger tips (1.3billion in case you were wondering). This then sets an expectation.
Less can mean more
‘You do need to ensure that you have enough of a sample size in order to make an informed customer led decision’
For any brand embarking on customer research, a more is more philosophy exists and we see many clients wanting to achieve absolute sample sizes running in to the millions of customers, perhaps without considering their relevance. One common mistake is kicking off a project by stating that you want to analyse one million customers for example. This is the wrong approach: don’t work backwards, work forwards! On several occasions we have encouraged customers not to ‘loosen’ criteria in order to achieve another X thousand customers.
Firstly, sample size is important. Regardless of what your objective or how you plan to the use the data, you need to ensure that you have enough of a sample size in order to make an informed customer led decision. There’s no argument to the premise that a larger sample size means more data to analyse, and let’s be honest, if you’re looking at, for example, an acquisition campaign, a larger audience size is always going to preferable.
As previously mentioned, the good news is that these large-scale customer sets do exist – even post GDPR we still have access to millions of consumers with whom you can profile, segment and identify. You can be sure your competitors are making the most out of it and if they are making crucial customer-based decisions based on data points from thousands, if not millions, of customers then you should be too. Otherwise, you risk waving goodbye to the next market share report.
So, how can you achieve a large sample size? Simple, social data. Social data offers one of the largest data-sets out there and you can be fairly certain your customers or next group of potential customers are represented. By matching, you could increase your sample of customers tenfold – now you can start to make some key decisions off a substantial panel. Just note sample size is important but it’s not the everything!
Stage two to this challenge is ensuring that by increasing your sample size you’re not just diluting the effectiveness of your end goal and sabotaging your own results. You need to stay true to your original brief, don’t start loosening your selection criteria simply because a sample of 1 million sounds better than a sample of 500,000. It could be that your 1 million consumers are now no longer relevant.
Quality over quantity
‘But a richness of data from a smaller pot is often more valuable’
One common misconception is that people often confuse robustness with quantity, a larger data set is not always a more robust one. In the right context, a sample of 1,000 customers could be extremely robust if the depth of data is good and the customers are representative. Think quality over quantity in this scenario, a richness of data from a smaller pot is often more valuable. This is how we regularly advise our clients when building an audience for projects: ensure that you maintain a representative and relevant customer group. If you want to expand the size, ask yourself, is the bigger sample still representative of what you were aiming for when you set out?
So, what is the best sample size? 1000 customers, 10,000, 100,000…
In short, there is no right answer, ultimately it comes down to how you apply the insight and translate it in to data led decision making. The more important question should be ‘is the sample of customers I have identified representative and therefore robust?’ If you can satisfy this then you’re on the right track and the outputs are likely to provide far more actionable information.
This means that identifying the right sample size is a balancing act: reach for the largest group possible and relevance goes out the window but dig deep in to a small group and your base may not be representative. Provided you have a good spread of customers for your specified segments and have identified consumers based on specific tangible criteria, you can be confident in the outputs. Next stop activation.
Remember, more is always better, assuming you can maintain relevance and robustness.