Client Challenge: The way we see it

In this series of blogs, we will be sharing some of the challenges posed by our clients and lending our perspective on best practices and how to navigate common concerns.

Client challenge – What is the right sample size for accurate insights?

In recent months it’s a question that is being asked of us more and more: ‘Is this a good enough sample size for the insights to be accurate?’ – or questions to that affect. When conducting a piece of consumer research or customer insight, brands are developing a preoccupation with size, but is it justified?

Wind the clock back several years and it’s probably not something that was asked very often. The evolution of large data sets has meant businesses and brands can now expect large amounts of customer data, and not just in depth but in breadth too. Many data businesses (Starcount included) talk about the size of their panel or the volume of consumers they have at their finger tips (1.3billion in case you were wondering). This then sets an expectation.

Less can mean more

‘You do need to ensure that you have enough of a sample size in order to make an informed customer led decision’

For any brand embarking on customer research, a more is more philosophy exists and we see many clients wanting to achieve absolute sample sizes running in to the millions of customers, perhaps without considering their relevance. One common mistake is kicking off a project by stating that you want to analyse one million customers for example. This is the wrong approach: don’t work backwards, work forwards! On several occasions we have encouraged customers not to ‘loosen’ criteria in order to achieve another X thousand customers.

Firstly, sample size is important. Regardless of what your objective or how you plan to the use the data, you need to ensure that you have enough of a sample size in order to make an informed customer led decision. There’s no argument to the premise that a larger sample size means more data to analyse, and let’s be honest, if you’re looking at, for example, an acquisition campaign, a larger audience size is always going to preferable.

As previously mentioned, the good news is that these large-scale customer sets do exist – even post GDPR we still have access to millions of consumers with whom you can profile, segment and identify. You can be sure your competitors are making the most out of it and if they are making crucial customer-based decisions based on data points from thousands, if not millions, of customers then you should be too. Otherwise, you risk waving goodbye to the next market share report.

Social data

So, how can you achieve a large sample size? Simple, social data. Social data offers one of the largest data-sets out there and you can be fairly certain your customers or next group of potential customers are represented. By matching, you could increase your sample of customers tenfold – now you can start to make some key decisions off a substantial panel. Just note sample size is important but it’s not the everything!

Stage two to this challenge is ensuring that by increasing your sample size you’re not just diluting the effectiveness of your end goal and sabotaging your own results. You need to stay true to your original brief, don’t start loosening your selection criteria simply because a sample of 1 million sounds better than a sample of 500,000. It could be that your 1 million consumers are now no longer relevant.

Quality over quantity

‘But a richness of data from a smaller pot is often more valuable’

One common misconception is that people often confuse robustness with quantity, a larger data set is not always a more robust one. In the right context, a sample of 1,000 customers could be extremely robust if the depth of data is good and the customers are representative. Think quality over quantity in this scenario, a richness of data from a smaller pot is often more valuable. This is how we regularly advise our clients when building an audience for projects: ensure that you maintain a representative and relevant customer group. If you want to expand the size, ask yourself, is the bigger sample still representative of what you were aiming for when you set out?

So, what is the best sample size? 1000 customers, 10,000, 100,000…

In short, there is no right answer, ultimately it comes down to how you apply the insight and translate it in to data led decision making. The more important question should be ‘is the sample of customers I have identified representative and therefore robust?’ If you can satisfy this then you’re on the right track and the outputs are likely to provide far more actionable information.

This means that identifying the right sample size is a balancing act: reach for the largest group possible and relevance goes out the window but dig deep in to a small group and your base may not be representative. Provided you have a good spread of customers for your specified segments and have identified consumers based on specific tangible criteria, you can be confident in the outputs. Next stop activation.

Remember, more is always better, assuming you can maintain relevance and robustness.


Leading high street bank onboarding SMEs

Starcount mined a High Street bank’s business data, third party data and Starcount Observatory data to better understand the SME market, their needs and wants.

Read More

7 Steps to Data Transformation

Data transformation demands clearly pronounced goals, strong engagement strategies and a willingness to move quickly and risk failure. These require solid, visionary leadership in order to inspire buy-in across an organisation. This Whitepaper, authored by data pioneer Edwina Dunn details the 7 success factors needed for data transformation.

Read More

Unravelling the urban myths of the pensions industry

Starcount has delved into the state of the UK pensions industry, examining how disruption in the space is causing many providers to reassess how they communicate with their customers.

Read More

Retirement & Investments business

Starcount created actionable insight and a data-focused strategy for a leading retirement & investment business, developing a deep understanding of their customers and collating data to identify significant savings in their business.

Read More

Your Life: A revolution in approaching education

Starcount worked with community interest company Your Life to create deep insight into how schools are performing based on how highly they perform in STEM subjects.

Read More

The key to understanding emotional loyalty

This Whitepaper examines our revolutionary new approach to data science using emotional analytics, revealing how peoples' passions predict purchase.

Read More

Customer centricity and the future of banking

Starcount has delved into the future of the banking industry, looking at how different forms of disruption, such as the Open Banking Initiative, will affect the industry.

Read More

Boosting test drives by personalising communication

Starcount worked with this automotive brand to maximise engagement and increase sales of their new model.

Read More