Sniff, scratch and score - how you can evaluate new technologies fast and efficiently, and why it's easier than ever.
When seeking technology solutions to business problems you quickly find there is a phenomenal number of products and alternatives available. Pick any one of the technology sets in your organisation and you will find there are dozens of possible alternatives, with many of them being really viable solutions.
I thought I would share with you how your approach to product evaluation can be streamlined to make the best use of your time and resources, and how the ability to evaluate products has also evolved along with the technologies.
Fully integrating a new solution into a production environment is not usually trivial, but luckily vendors have made it easier than ever to try their products out. These days you benefit from:
- Simpler installation and uninstallation processes, with no or low impact installs.
- More software and appliance based products, reducing the reliance on hardware.
- Web and cloud based labs and demos.
- More accessible product information with improved quality.
- The ability to share experiences with peers via social media.
- More open standards meaning less vendor lock-in, better inter-operability and increased portability of data.
- Easier creation and disposal of scratch and test environments.
- Test environments that are easily isolated logically.
- Easily produced copies of production data for testing.
At ViFX we seek out and evaluate dozens of products every year. To perform long and rigorous evaluations of every single one would be overly resource intensive. However, by applying the three step approach described below, and knowing our customers and their needs well, we distil all the possibilities down very quickly to just a few that need further attention.
I have a three stage approach I like to use when evaluating new technologies, which allows me to make the best use of my time. Each stage is a gate that the product must pass before moving to the next stage.
The Sniff stage is my initial attempt at understanding the product. This typically includes online research to develop a simple understanding of what it is, how it works and what the benefits may be. This research consists of downloading product literature, watching (short) videos, and reading case studies. The questions I ask here are: does it seem to do what I need and does it provide the benefits I seek? If so, then it passes the Sniff test and it's onto some scratching...
The Scratch stage usually involves getting hands-on with the product if possible. Most partners and vendors can arrange demos or lab access. Better yet, see if the product can be evaluated in your own environment. Obtain or arrange a demo or evaluation copy and run it up. This is a quick and dirty test run of the product. You want to find out if it actually does what it says on the tin? Compare it with products you may have already seen and see how it stacks up. Often it's here that I start to discover a product's warts or limitations. As this should be a very quick evaluation you are only scratching the surface of the product and may encounter issues that could have been avoided with proper planning, but remember this is about getting a feel for the product.
Start working your partner or vendor hard to provide you with exactly what you need to properly assess a product. Get them to help cut through their own marketing. This is also an opportunity to begin an internal dialogue amongst your peers to share findings and gather opinions.
For those who work more in software development, scratch testing is very much like smoke testing, basic - so don't over complicate it.
After passing the Sniff and Scratch tests it's then time to more formally assess the product. There's a bit more planning around the testing to ensure the validity of testing and the success criteria (scoring it). Here you need to start thinking about more detailed configuration information, possibly formal costings, and more involvement and feedback from interested internal parties. Put the product through its paces as you would expect to use it. Outputs from this stage would typically feed into your business case development. This phase is essentially a proof of concept.
As with Scratch testing it should be relatively easy to get the product up and running. By now your partner or vendor should know you are more serious and should be very accommodating with support and advice.
You should now be well armed to make a decision about whether a product is right for your business.
Risk vs. Rewards
What lead me to think more in depth about our technology evaluation process was some recent research we did on a product called PernixData FVP. To grossly simplify the product I would call it storage acceleration for virtualised environments.
It was easy for me to understand what this product did (that message was driven home on every page of their website) but what I couldn't figure out was how the product did it. Why should I care? Because by its nature it was going to integrate at several points of our infrastructure and I wanted to understand what the impacts were likely to be.
It passed the sniff test easily - it looked good. When we moved into Scratch testing I was amazed at the simplicity of trying the product out. For a product that appears to be quite invasive, in that it is installed on the hosts and is in the data path, we found that it can be installed, activated, tested, deactivated and uninstalled with no interruption to VM's running on our hosts. There was copious information on exactly how the product worked, which left me with few questions. We also had the vendor engaged at this point who supplied a lot of useful information specific to our environment.
To me this really shows how highly evolved products have become, how easily they can be trialled, and how we'd be remiss not to at least try them out. It's quite a powerful experience to get one's hands a little dirty and really feel what a product is like and capable of. It makes the marketing promises that much more tangible.
What are the benefits of trying things out? Building confidence that a technology will actually solve your business problems, such as improving application responsiveness, increase agility or performance, or reducing costs.
SummaryKey points to remember are:
- Cast your initial net wide (sniff) across many vendors' products.
- Open your mind beyond the traditional big vendors you are used to hearing about. New products are usually fairly well evolved before they make it to our shores.
- Remember that, technically, product testing is easier than ever.
- Work your partners and vendors hard to provide you with what you need to make an informed decision.
- Scratch, Sniff and Score for efficient product evaluation.
"People will try to tell you that all the great opportunities have been snapped up. In reality, the world changes every second, blowing new opportunities in all directions, including yours". Ken Hakuta.
As always we would love to hear your feedback. What methodology do you use? How brave is your organisation when trying new technologies? How easily do you find new products? How independent is the advice you get?