technology-evaluation

Write weighted evaluation criteria before touching any candidate, shortlist to 2-3 finalists plus the incumbent, then check maintenance vitals, price the exit cost, and run a timeboxed spike against your real use case. Score candidates against the original criteria and record the decision in an ADR.

Check release recency, contributor concentration (bus factor), median time-to-response on issues and PRs, CVE response speed, and whether maintainers are paid to work on it. Star counts measure past marketing, not present health.

Yes, the incumbent must compete as a candidate on equal footing, and it starts ahead because migration and retraining costs are real. A challenger that is only marginally better usually loses once switching costs are counted.

A spike is a timeboxed (days, not weeks) vertical slice of your actual use case that deliberately probes your data shape, deployment reality, integration seams, and one failure drill. A spike where everything worked perfectly probably probed nothing.

Do not use it for surveying a research field or literature; that calls for a literature-review approach instead. This process ends in an adopt/reject decision for a specific tool with switching costs, not a broad field survey.

Exit cost determines how expensive a wrong decision is in 18 months; open data formats and adapter seams can cut it dramatically. A mediocre tool with a cheap exit routinely beats a brilliant one with a moat, and vendors design against your exit.