How We Test Products at SPY.com
SPY.com is a discovery and reviews platform, connecting you with the latest in the world of products. We focus on men’s grooming and lifestyle, technology and gadgets, and health and fitness. We bring you under-the-radar releases and top-rated picks from across the globe. Over the past few years, the SPY team has tested thousands of products, and we have a simple goal: to help you shop smart. We will never recommend a product that we wouldn’t buy ourselves, and we love finding awesome new gear to share with SPY’s readers. So how do we do that?
The SPY editorial team includes product testers, commerce journalists, and product reviewers with decades of combined experience testing products. From our office in New York and Los Angeles, or from our own homes, we’re always trying out new products for SPY readers.
To find the best products, we scour the web, carefully read customer reviews, compare product specifications, and talk to experts. When writing about the best face wash for men, we’ll consult dermatologists in addition to relying on our grooming writers’ expertise in this subject area. Most importantly, we have implemented a rigorous product testing system to ensure SPY readers are getting the best possible product recommendations.
To be as transparent as possible, we wanted to outline our product testing process in detail. You can also refer to our About Us page for more information on our editorial process.
The Testing Process
Our editors and writers hand-select every product we feature on SPY. Our coverage includes new product releases, trends and news that’s relevant to our readers, in addition to product reviews based on in-depth hands-on testing. Brands cannot pay SPY for positive reviews, and every review and shopping guide on our site is based purely on our editorial recommendations.
Though we can’t try every product we cover, the SPY team has rapidly ramped up our product testing efforts. That includes bringing on a reviews editor and new team members to expand our hands-on testing capacity.
If you’ve ever been curious to take a peek behind the shopping guide and see how we chose a specific product, these are the steps we take when writing our reviews and review roundups.
Step 1: Research
We start by conducting market research to learn more about the overall product category, what the top-performing brands are, which products people have questions about and which are the biggest players in the space.
Depending on the product category, the number of products we test can vary wildly. For a product with thousands of choices, like men’s underwear, we narrowed down our list of test products to about 100. For adjustable dumbbells, that number was closer to 15. After we have our list of products we need to test to determine rankings, we start calling in samples.
Part 2: Calling in Sample Units
In order to conduct reviews, brands will sometimes send out sample units. These products are typically free for publishers like SPY. In return, brands typically request that we publish a review, but we make no guarantees about coverage. We have tested many products that we ultimately decided not to feature on the site. In some cases, sample units are not available. In those cases, we will purchase products for our reviewers to evaluate.
Part 3: Creating Judgement Criteria
As the sample units are arriving, we create a list of review criteria. This is a detailed list of specifications and features we think are relevant to overall product quality. The criteria will vary widely depending on the specific product category. We create internal spreadsheets to centralize our criteria, helping to reduce bias when products are evaluated by different testers.
Part 4: Testing
Now we get to the fun part: testing. SPY has about a dozen experienced product testers producing reviews at present. We spend as much time as possible with each product, using it in a variety of contexts and taking note of everything.
When testing a product, our reviewers evaluate and rate how it performs according to the criteria we’ve set. This can be a time-consuming process, and sometimes it can take three months or longer to fully produce a review like this. Reviewing products in this way allows us to fairly compare how similar products perform. We log detailed notes about how well the product performs in the given criteria.
Using the criteria we’ve established, we then evaluate the products and award them a numeric score. This is then used to help finalize the rankings in a review. This allows us to create rankings that are as objective as reasonably possible. Many of our reviews will feature a section that details how we tested that particular product category.
Part 5: Re-Testing and Updating
The last step in our testing process happens weeks, or even months, after our initial testing round. We regularly update our reviews and shopping guides with the latest products and new thoughts on products we’ve already tested. Sometimes our experience of a product changes after spending more time using it, and so we need to adjust our rankings accordingly. Likewise, if there’s a new product that’s garnered a lot of attention, we’ll test it to see how it fits into our existing ranking.