Turn a Ranked List Into a Three-Product Shortlist
Buyer guides · 10 min read ·
Turn a long ranked list of SaaS tools into three real candidates: requirements first, a simple matrix, real-task trials and a decision rule set in advance.
A ranked list of software tools can contain fifty names. A buyer cannot trial fifty products, and does not need to. What a buyer needs is three good candidates, tested properly, and a clear way to choose between them. This guide is a method for getting from a long list to that shortlist and then to a decision, in a few hours of focused work spread over a week or two.
Step 1: Write the requirements before you read the list
Ranked lists are persuasive, and if you read one first, it will shape what you think you want. So begin with a blank page and answer five questions.
- What job must this tool do? Describe it as a task: "send invoices and chase payments", not "financial management".
- Who will use it? How many people, with what skill?
- What must it connect to? The systems it has to exchange data with.
- What are the non-negotiables? Security needs, data location, accessibility, support hours.
- What is the budget model? Not the number, but the shape: per user, per usage, flat. Where figures are needed, use the vendor's live pricing page.
Split the answers into must-haves, should-haves and nice-to-haves. A good rule is no more than five must-haves; if you have fifteen, you are describing an ideal, not a requirement.
Step 2: Skim the list as a map
Now open the list. Read it quickly, ignoring positions for the moment. Note:
- Which products are named in several places?
- Which have descriptions that match your job?
- Which categories appear nearby that you had not considered?
The Pareto principle, often summarised as "eighty percent of effects come from twenty percent of causes", is a useful reminder: in most categories a few products meet most buyers' needs. Your job is to find which few meet yours.
Browse the neighbouring categories. A product that does what you need may be classified differently from how you imagined. The search page can help you look by task rather than by label.
Step 3: Pick up to eight candidates
Choose a long shortlist of up to eight products by applying a first filter:
- The description matches your job.
- No must-have is clearly missing, as far as you can tell from the listing.
- The product is actively maintained: a recent update, a live changelog or a working site.
Include the top few ranked products, because they are popular for reasons that may apply to you. Include at least two lower-ranked ones that match your needs closely, because a good fit often hides further down.
Step 4: Build a decision matrix
A decision matrix is a simple table that compares options against criteria. Put your candidates in rows and your requirements in columns. For each cell, record yes, partly or no, and note the source: a page, a document, a line from support.
Add a column for each of these, which most buyers forget:
- Exit: can you export your data in a usable form?
- Evidence: do independent sources, such as audit reports or customer references, back the claims?
- Fit with your team: would the people who will use it be comfortable?
If the matrix uses weights, keep them simple: must-haves count as pass or fail, should-haves score one to three. Over-engineering the scoring is a common way to disguise a gut decision as a calculation.
Step 5: Cut to three
Remove any candidate that fails a must-have. Among those left, keep three:
- The safe choice: typically the most established, with the strongest evidence.
- The best fit: the one that scores highest against your requirements.
- The challenger: a smaller or newer product that scores well and offers something different, such as simpler pricing or a better experience for your use case.
If one product is both the safe choice and the best fit, take your two best alternatives. The point of having three is to compare, not to tick a box.
Step 6: Trial with a real task
A trial with fake data tells you little. Choose one real, representative task and carry it out in each of the three tools.
- Set up: how long did it take to get started? What did you have to ask for?
- Do the task: was it clear? Did you get stuck?
- Connect it: if integration matters, try one.
- Export: can you get your data out?
- Ask a question: contact support with a genuine query and note the response time and quality.
Time each step. Ask a colleague to try the same task and compare notes. People often discover that a tool that looked lovely is hard to use, or that a plain one is surprisingly pleasant.
Step 7: Read the small print
Before deciding, read what the matrix cannot capture.
- The terms. Notice termination, notice periods, renewal and limits.
- The privacy and security pages. Check scope and dates.
- The pricing page. Understand what triggers an increase.
- The changelog. See how often and how carefully the product changes.
The detail of pricing and terms belongs on the vendor's own live pages; do not rely on a list's summary.
Step 8: Decide with a rule you set earlier
Before you look at the trial results, write the decision rule. For example: "Choose the tool with the highest matrix score, unless it fails the export test, in which case choose the next." Having the rule in advance protects against the pull of a persuasive demo.
If two products tie, use a tiebreaker you chose beforehand. Candidates include the quality of the support response, the ease of leaving or the preference of the people who will use the tool daily.
Step 9: Record and review
Record the decision, the reasons and the date. Set a review date, perhaps after a quarter, to ask whether the choice is working. Keep the matrix: when you next need to choose, you will have a head start. If you later find that the list's ranking helped or misled you, note that too; it will improve how you read lists next time.
What the process saves you
The method takes a few hours and avoids the two commonest buying mistakes: choosing the top-ranked product without checking fit, and drifting into a long evaluation that never ends. It also leaves a paper trail that explains the choice to anyone who asks.
A worked example
A six-person marketing agency needs a tool to manage client approvals on creative work. Their must-haves: share a draft with a client by link, record the client's approval with a date, work on a phone and keep data in the UK or EU. Their should-haves: integration with their file storage and a free guest access for clients.
They skim a ranked list in the proofing category and note eleven products. Applying the first filter leaves seven. The matrix shows that two lack guest access, one stores data outside the region and one has not been updated for a year. That leaves three: a long-established product with every feature, a mid-ranked product that matches the requirements closely, and a newer product with a clean interface and a simple pricing model.
The trial task is the same for each: upload a draft, send a link to a colleague pretending to be a client, collect an approval and export the record. The established product takes forty minutes to set up and offers far more than they need. The mid-ranked product takes ten minutes and does the task smoothly. The newer product takes fifteen minutes but its export is a screenshot rather than a file.
Their pre-written rule says that a failed export test disqualifies a tool, so the newer product is out. The decision between the first two comes down to the support test: the mid-ranked vendor answers a question within the hour, with a clear reply; the established vendor takes a day and sends a link to a general help page. They choose the mid-ranked product, write down why and set a three-month review.
Questions buyers ask
Is it wasteful to ignore the top of the list? Not ignoring, just not deferring. The top positions earn their place on your shortlist the same way as any other product: by matching your requirements.
What if I do not know my requirements yet? Run a short discovery first. Talk to the people who will use the tool, list the tasks they perform and rank them by how often and how painfully they occur. Requirements emerge from that list.
Should I ask for a demonstration? A demo is useful for understanding scope, but it shows the best path. A trial with your own task shows the real one.
How do I avoid being swayed by a sales call? Bring your matrix. Ask the vendor to fill in the cells they claim to satisfy and to show you where each answer is documented.
Summary
Write requirements first, skim the list as a map, pick up to eight candidates, build a simple matrix, cut to three, test each with a real task, read the small print, apply a decision rule you set in advance and record the result. A ranking is a good guide to who is out there. The decision belongs to your requirements, your trial and your judgement.
Questions and answers
- Why three products?
- Three gives you a safe choice, a best-fit choice and a challenger to compare, without the time cost of evaluating a long list.
- Should I start with the top-ranked product?
- Start with your requirements, then use the ranking to find candidates. The top-ranked product is a candidate, not a default.
- How long should a trial take?
- Long enough to complete one real task from start to finish, plus the setup. For many tools that is a few days of light use.
- What if two products tie in my matrix?
- Use a tiebreaker you chose in advance, such as ease of leaving or quality of support, or run a second trial focused on the difference.