Thursday, August 2, 2012

Review Methodology

Introduction


Being The Outdoor Nerd, I feel it's important for my reviews to be as uniform as possible, not only with each other, but with other reviewers as well. This gives you, the reader, a more consistent experience, and makes it easy to compare similar products you are interested in. What I am attempting to do is take the best of what I like about other reviewers' work, and add my own unique spin, hopefully with a little humor. What I am not attempting to do is to duplicate the technical lab results of some of the more well known reviewers. As a consumer, what I want to know from an expert review is what the expert feels about that product based on his or her expertise, and this is the kind of reviews I am interested in providing. However, I will make references to lots of technical details where I feel they are relevant to a potential owner of the product. Does a product have better or worse performance than it should have? Are the stated specs wrong? If something about the specs are different than what I expected, then I will try point those differences out.

Product


This is where I provide very general information about the product: Who makes it. What it costs. Where to get it. I generally prefer to provide my own review samples, but in some cases I may accept a review sample for a product that I'm not sure I would purchase on my own. In these cases I will provide the name of the manufacturer or retailer who provided the review sample, in compliance with Federal law.

Official Specs


This section describes the official product specifications, usually taken either from the manufacturer or retailer. Sometimes it takes several different sources to piece together a cohesive spec. And sometimes the data isn't reported anywhere but easily obtainable, such as product measurements and dimensions. And in some rare cases the data isn't reported and I may decide to run my own tests if I cannot find the information available anywhere else.

First Impressions


As the old saying goes, you only get one chance to make a first impression. This is important to me as a consumer, and I give it importance in my reviews. This is where I talk about the experience of opening the product packaging, taking the product out and sizing it up. Is there something, good or bad, which stands out? This section might duplicate a little bit of the review with other sections. I may note some obvious flaw in this section and then go into more depth for example in the Fit And Finish section.

Design


This is where I talk about the design of the product, adding my own thoughts. I may also compare the design to similar products I have previously reviewed. What are the benefits of this design? What are the drawbacks? How can the product be improved? As someone who has designed large software systems for the last 25 years, I'd like to think that I know a little something about design. It's also a pet peeve of mine, as I don't think that most manufacturers pay enough attention to design.

Usability


I've told you about the product: it's specs, how it works and who made it. This information is generally invaluable, and part of any good review. Different people look for different qualities, and it's important to put it all out there. But where I try to make my reviews different is in giving my thoughts about the product in use over time in my own life, and with my own experiences. Has it held up well, and is it useful? Is there something annoying about the product? Occasionally I'll see something of concern in the specs which isn't an issue in practice. And the reverse can be true also. The specs and design of something might look great on paper, but be lousy in actual use. This is where the rubber meets the road, so to speak.

Most people purchase a product with the intention of using it. Maybe that statement sounds obvious, but to me, this is where the magic is. I will also use this section to describe any interesting stories surrounding the product. Does my wife hate it? Does my use of the product somehow involve a Racoon? These are the stories that I personally look for when I'm looking to buy a product. These kinds of stories help me picture this product as part of my daily experience. And in some cases, they are the deciding factor in my decision not to purchase the product.

Conclusions


I've taken the product out of the box, looked at its specs and analyzed its design. I've carried it with me and given my general thoughts. This is the section where I try to tie everything together and give any final thoughts on the product and how it fits into my life. Do I like the product? Am I using it? Is the product now shelf-candy?

Generally I try to avoid a simple numeric score. And for the most part, you can already find that anywhere on most products. The sites with the 1-5 star rating are the places where I start my journey with a product, and I look to independent reviews to give me knowledge not contained within them. My goal is to augment those user ratings and reviews, not replace them.

Head-to-Head Comparison


Every once in a while a product I am reviewing will be very close in price/features/whatever to another product. And in some cases I might own that product as well. So I will do a mini review-within-review of the competing product, with some or all of the sections of a normal review. How is the product I am reviewing better or worse than the competing product? I will try to highlight difference and similarities in design, usability and fit-and-finish.

Other Thoughts


Sometimes I will hear about or discover something cool or unusual about a product. In these cases I will create a separate section just for those thoughts. Can the product be used as a can-opener or desert topping? Are these products catching fire and/or outlawed in 37 states? Sometimes even a rigorous and disciplined methodology breaks down under the weight of something that it wasn't designed to fit. In these cases, I may change my methods slightly to fit the product that I am reviewing, as opposed to the other way around.

Gallery


I try to show as many pictures of the product as I think makes sense. Obviously pictures of the product itself. For small gadgets, I like to put a common object like a quarter in the picture to give the user a sense of scale. Nothing more frustrating than buying a product and finding out that it's drastically too large or too small for your needs. Sometimes I like to take some artsy pictures of the product in an interesting setting. Or with a similar or complimentary product. Some products are closely associated with other products.


No comments:

Post a Comment