How the ranking is scored
Updated
By Ronald Renaud, Analyst writing on QA vendors and test automation
Editorial ranking published by Ronald Renaud. Personal project of the publisher. No advertising, no vendor payments.
The ranking scores each agency on eight criteria. The weights were fixed before any vendor data was collected, and the file that holds them is locked: the site build fails if a weight changes after vendor records exist.
What the weights reflect
The heaviest weight goes to public Cypress work (22%), because a buyer searching for a Cypress agency needs evidence that the firm has shipped Cypress suites, not just listed the tool. Ownership of the test code and the handover that goes with it comes second (16%): buyers repeatedly ask who owns the Cypress code after the project ends. CI integration and maintenance (14%) covers the recurring cost of flaky tests and slow pipelines. Migration experience (12%) matters to teams moving from Selenium, Protractor or WebdriverIO. Client reviews (12%), named clients with published outcomes (9%), engagement terms (8%) and pricing transparency (7%) complete the set.
How a score is set
Each criterion is scored from 0 to 100 against the scale below. The anchors 25, 50, 75 and 100 describe the public evidence required; a score between two anchors is allowed when the reasoning names both. A score of 0 means no public evidence was found. Scores are set from the collected data only, and each one has written reasoning that cites the source.
How the total is calculated
Total = the sum of each score multiplied by its weight, divided by 100. It is computed when the site is built. Equal totals are separated by the score on the highest-weight criterion, then the next.
What the ranking does not measure
It does not measure engineering quality directly. It measures what a buyer can verify before a first call. A capable firm that publishes little will score lower than its work may deserve; the sources page lists what was found for each firm.
Scoring scales
No public evidence found for the dimension. Tie-break: Higher score on the highest-weight dimension, then the next dimension in weight order.
Cypress depth and public work · 22%
Documented Cypress cases, public repositories, technical articles, Cypress Cloud and component testing experience
| Score | What it takes |
|---|---|
| 25 | Cypress appears only in a list of supported tools; no Cypress-specific page, case or article. |
| 50 | A Cypress services page or at least one Cypress case or technical article, without code or measurable detail. |
| 75 | Several Cypress cases or articles with technical detail (framework structure, CI runs, metrics), or a public repository by the vendor. |
| 100 | Multiple Cypress cases with metrics plus public repositories, plugins or talks, and documented Cypress Cloud or component testing work. |
Test code ownership and handover · 16%
Documented policy that the client owns the code, standard Cypress project layout, documented handover
| Score | What it takes |
|---|---|
| 25 | No public statement on code ownership or handover; the framework is described as proprietary or not described. |
| 50 | A general statement that the client owns deliverables or IP, without detail on test code, repository or handover. |
| 75 | An explicit statement that tests live in the client repository and belong to the client, or a described handover or knowledge-transfer step. |
| 100 | Explicit ownership of test code in the client repository plus a documented handover process (documentation, training, transition plan). |
CI/CD integration and maintenance · 14%
Documented CI setup with parallelisation, flaky-test handling, maintenance model
| Score | What it takes |
|---|---|
| 25 | CI/CD mentioned only as a keyword. |
| 50 | Named CI tools or pipelines in a service page or case, without parallelisation, flake handling or maintenance detail. |
| 75 | A case or article describing test runs in CI with parallelisation or flaky-test handling, or a stated ongoing maintenance model. |
| 100 | Documented CI integration with parallelisation and flaky-test handling, plus a stated maintenance model with measurable results. |
Migration experience · 12%
Documented migrations from Selenium, Protractor or WebdriverIO to Cypress, or Cypress to Playwright when asked
| Score | What it takes |
|---|---|
| 25 | Migration listed as a generic service without framework names. |
| 50 | A page or article on migrating between named frameworks, without a client case. |
| 75 | At least one public case of a migration to or from Cypress with named frameworks. |
| 100 | Several public migration cases with named frameworks and measurable results (run time, stability, coverage). |
Client review quality and volume · 12%
Verified reviews on Clutch, G2, GoodFirms weighted to automation engagements
| Score | What it takes |
|---|---|
| 25 | Fewer than 5 verified reviews across Clutch, G2 and GoodFirms, or an average rating below 4.5. |
| 50 | 5-19 verified reviews with an average of at least 4.5. |
| 75 | 20-49 verified reviews with an average of at least 4.7, several describing automation work. |
| 100 | 50 or more verified reviews with an average of at least 4.8, many describing test automation work. |
Named clients and published outcomes · 9%
Named clients with measurable automation outcomes in public cases
| Score | What it takes |
|---|---|
| 25 | Client logos or names without cases. |
| 50 | Named clients in cases without measurable outcomes, or anonymised cases with metrics. |
| 75 | At least three named clients with measurable automation outcomes. |
| 100 | Six or more named clients with measurable automation outcomes, at least one involving Cypress. |
Engagement flexibility and onboarding · 8%
Models offered, documented time to start, minimum contract
| Score | What it takes |
|---|---|
| 25 | One engagement model described. |
| 50 | Two or more models (dedicated team, project, staff augmentation) described, no start time. |
| 75 | Several models plus a stated time to start or a stated pilot or trial option. |
| 100 | Several models, a stated time to start, and a stated minimum contract or pilot terms. |
Pricing transparency · 7%
Published rates or ranges and minimum project size
| Score | What it takes |
|---|---|
| 25 | No rates on the vendor site; only a range on a third-party directory. |
| 50 | An hourly range and minimum project size on a third-party directory profile. |
| 75 | Rates or package prices published on the vendor site. |
| 100 | Rates by role or package on the vendor site plus a stated minimum project size. |
Scores by dimension
| # | Company | Cypress 22 | Ownership 16 | CI/CD 14 | Migration 12 | Reviews 12 | Clients 9 | Engagement 8 | Pricing 7 | Total |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ThinkSys | ● 75 | ● 100 | ● 100 | ◐ 50 | ● 75 | ● 75 | ● 75 | ● 75 | 79.5 |
| 2 | DeviQA | ● 85 | ● 75 | ● 85 | ◐ 50 | ● 90 | ● 85 | ● 85 | ◐ 50 | 77.4 |
| 3 | BetterQA | ● 75 | ● 80 | ● 85 | ○ 0 | ◐ 65 | ◐ 65 | ● 100 | ● 75 | 68.1 |
| 4 | Codoid | ◐ 65 | ● 75 | ● 75 | ◐ 50 | ◐ 50 | ◐ 50 | ◐ 50 | ◐ 50 | 60.8 |
| 5 | QASource | ◐ 50 | ● 75 | ● 75 | ○ 40 | ◐ 65 | ○ 40 | ● 85 | ◐ 50 | 60 |
| 6 | TestFort | ◐ 60 | ○ 0 | ● 75 | ● 75 | ● 75 | ◐ 65 | ● 75 | ◐ 50 | 57.1 |
| 7 | TestDevLab | ◐ 65 | ◐ 50 | ● 75 | ○ 0 | ● 75 | ◐ 50 | ● 85 | ◐ 50 | 56.6 |
| 8 | a1qa | ◐ 50 | ● 75 | ● 85 | ○ 0 | ◐ 65 | ◐ 50 | ◐ 50 | ◐ 50 | 54.7 |
| 9 | QA Madness | ◐ 50 | ◐ 60 | ● 75 | ○ 0 | ◐ 65 | ◐ 50 | ● 75 | ◐ 60 | 53.6 |
| 10 | Testlio | ○ 35 | ◐ 50 | ● 75 | ○ 25 | ◐ 65 | ◐ 60 | ● 85 | ◐ 50 | 52.7 |
| 11 | Abstracta | ◐ 65 | ○ 0 | ◐ 60 | ○ 40 | ● 75 | ◐ 60 | ◐ 60 | ◐ 50 | 50.2 |
| 12 | Testrig Technologies | ● 75 | ○ 0 | ● 75 | ● 75 | ◐ 50 | ○ 25 | ○ 25 | ◐ 50 | 49.8 |