by Matt Dusig, Co-Founder & CEO, uSamp

If you’ve worked in the industry long enough, you’ve experienced a five-alarm fire drill. You’ve carefully plotted out what you believe will be the perfectly executed project. Everything is running as scheduled when suddenly, incidence drops without warning, response rates tank, and the client unexpectedly adds that last minute “little” change that screens out potential respondents. Your best-laid plans are no longer suitable and more resources are needed—fast. What do you do to keep your project alive and your client satisfied? Did you say...aggregation?

The ‘A’ word is frequently thrown around the industry, and often with negative connotations. But what exactly is it, who’s doing it, and when is it used? In practice, aggregation is when a research company combines multiple panels or traffic sources in order to fulfill the sample needs of a planned or in-field project.

It is standard practice for research companies to “blend panels and traffic” on some or even all projects. Sample providers may combine their resources with a third party or “outside” supplier to offer full feasibility for a project. Any company that aggregates is providing research companies with sample from multiple sources; they’re simply being a ‘one-stop shop’ to source a project’s needs, and provide a complete sample plan for a research project to their clientele.

Research companies tend to turn to aggregation in a few instances. When a client demands a cushion or safety margin for sample needed, in other words “overlapping feasibility,” providers use aggregation to ensure respondent quotas can be met.

When the end client’s (the company commissioning the research) demands are particularly targeted (i.e. needs Spanish-speaking Hispanics in Tampa, Albuquerque, El Paso, Tucson and Fresno. At low incidence, and for two waves of a pre/post advertising test. De-duplicated, of course.), “filling in the edges” becomes difficult, and this is where a combination of more than one company’s resources becomes necessary to get the numbers needed.

Finally, aggregation is applied for operational efficiency’ssake. For research companies buying sample, working with fewer suppliers on a given project saves time just by virtue of fewer partners to coordinate, communicate with, and watch over during the projects. Fewer invoices and P.O.’s too, of course!

From these scenarios, you can see that the practice of sample aggregation is common and necessary at times. So, how should sample providers be assessed?

Here are two key things clients should be asking of their sample providers regarding aggregation:

Transparency

If you come across a large sample company who claims they don’t aggregate, you should be wary. A savvy client who wants to understand the methodology behind who actually completed the survey and where they came from deserves a thorough answer. After all, the data and results of any research project are meant to drive actions and decisions in the marketplace. Any sample provider needs to be open about what resources were used for a project so that clients can be confident of how their audience was sourced. Ask your supplier to show you their Google Analytics report that reveals how much daily, weekly and monthly web traffic their panel web sites receive. Ask us and we’ll be glad to show you.

Frequency

Determining how frequently aggregation is used is important. Every sample supplier will know how their panel performs, and what nuances influence and drive their delivery. How much they then rely on combining the sample resources of others with their own will affect how they estimate  feasibility, time-in-field to complete projects, incentives necessary to motivate proper responses, and so many other fundamental, practical aspects of a project. While important for ad hoc projects, this is absolutely crucial for wave projects and trackers so that weekly/month/quarterly sample frames can be kept consistent. Otherwise, the sources alone can change the data for a project, and you become mired in that particular circle of project hell where you don’t know what caused your results to shift—an actual, real-market reaction, or just a hidden methodology change.

At uSamp, we invest a lot of time, money and resources to build, manage and grow our panels and traffic sources effectively, domestically and internationally. When called on to fill a project, we buy very little from other suppliers like us to get jobs done, and meet client needs with our own resources almost completely. In 2011 we aggregated on less than five percentof our total project volume by revenue.

So, bottom line: On your next project, do you really know where your sample is coming from? You have the right to that answer and a responsibility to your clients to find out.