Skip to content
cypressagencies.com

How the ranking is scored

Updated

By Ronald Renaud, Analyst writing on QA vendors and test automation

Editorial ranking published by Ronald Renaud. Personal project of the publisher. No advertising, no vendor payments.

The ranking scores each agency on eight criteria. The weights were fixed before any vendor data was collected, and the file that holds them is locked: the site build fails if a weight changes after vendor records exist.

What the weights reflect

The heaviest weight goes to public Cypress work (22%), because a buyer searching for a Cypress agency needs evidence that the firm has shipped Cypress suites, not just listed the tool. Ownership of the test code and the handover that goes with it comes second (16%): buyers repeatedly ask who owns the Cypress code after the project ends. CI integration and maintenance (14%) covers the recurring cost of flaky tests and slow pipelines. Migration experience (12%) matters to teams moving from Selenium, Protractor or WebdriverIO. Client reviews (12%), named clients with published outcomes (9%), engagement terms (8%) and pricing transparency (7%) complete the set.

How a score is set

Each criterion is scored from 0 to 100 against the scale below. The anchors 25, 50, 75 and 100 describe the public evidence required; a score between two anchors is allowed when the reasoning names both. A score of 0 means no public evidence was found. Scores are set from the collected data only, and each one has written reasoning that cites the source.

How the total is calculated

Total = the sum of each score multiplied by its weight, divided by 100. It is computed when the site is built. Equal totals are separated by the score on the highest-weight criterion, then the next.

What the ranking does not measure

It does not measure engineering quality directly. It measures what a buyer can verify before a first call. A capable firm that publishes little will score lower than its work may deserve; the sources page lists what was found for each firm.

Scoring scales

No public evidence found for the dimension. Tie-break: Higher score on the highest-weight dimension, then the next dimension in weight order.

Cypress depth and public work · 22%

Documented Cypress cases, public repositories, technical articles, Cypress Cloud and component testing experience

ScoreWhat it takes
25Cypress appears only in a list of supported tools; no Cypress-specific page, case or article.
50A Cypress services page or at least one Cypress case or technical article, without code or measurable detail.
75Several Cypress cases or articles with technical detail (framework structure, CI runs, metrics), or a public repository by the vendor.
100Multiple Cypress cases with metrics plus public repositories, plugins or talks, and documented Cypress Cloud or component testing work.

Test code ownership and handover · 16%

Documented policy that the client owns the code, standard Cypress project layout, documented handover

ScoreWhat it takes
25No public statement on code ownership or handover; the framework is described as proprietary or not described.
50A general statement that the client owns deliverables or IP, without detail on test code, repository or handover.
75An explicit statement that tests live in the client repository and belong to the client, or a described handover or knowledge-transfer step.
100Explicit ownership of test code in the client repository plus a documented handover process (documentation, training, transition plan).

CI/CD integration and maintenance · 14%

Documented CI setup with parallelisation, flaky-test handling, maintenance model

ScoreWhat it takes
25CI/CD mentioned only as a keyword.
50Named CI tools or pipelines in a service page or case, without parallelisation, flake handling or maintenance detail.
75A case or article describing test runs in CI with parallelisation or flaky-test handling, or a stated ongoing maintenance model.
100Documented CI integration with parallelisation and flaky-test handling, plus a stated maintenance model with measurable results.

Migration experience · 12%

Documented migrations from Selenium, Protractor or WebdriverIO to Cypress, or Cypress to Playwright when asked

ScoreWhat it takes
25Migration listed as a generic service without framework names.
50A page or article on migrating between named frameworks, without a client case.
75At least one public case of a migration to or from Cypress with named frameworks.
100Several public migration cases with named frameworks and measurable results (run time, stability, coverage).

Client review quality and volume · 12%

Verified reviews on Clutch, G2, GoodFirms weighted to automation engagements

ScoreWhat it takes
25Fewer than 5 verified reviews across Clutch, G2 and GoodFirms, or an average rating below 4.5.
505-19 verified reviews with an average of at least 4.5.
7520-49 verified reviews with an average of at least 4.7, several describing automation work.
10050 or more verified reviews with an average of at least 4.8, many describing test automation work.

Named clients and published outcomes · 9%

Named clients with measurable automation outcomes in public cases

ScoreWhat it takes
25Client logos or names without cases.
50Named clients in cases without measurable outcomes, or anonymised cases with metrics.
75At least three named clients with measurable automation outcomes.
100Six or more named clients with measurable automation outcomes, at least one involving Cypress.

Engagement flexibility and onboarding · 8%

Models offered, documented time to start, minimum contract

ScoreWhat it takes
25One engagement model described.
50Two or more models (dedicated team, project, staff augmentation) described, no start time.
75Several models plus a stated time to start or a stated pilot or trial option.
100Several models, a stated time to start, and a stated minimum contract or pilot terms.

Pricing transparency · 7%

Published rates or ranges and minimum project size

ScoreWhat it takes
25No rates on the vendor site; only a range on a third-party directory.
50An hourly range and minimum project size on a third-party directory profile.
75Rates or package prices published on the vendor site.
100Rates by role or package on the vendor site plus a stated minimum project size.

Scores by dimension

#CompanyCypress 22Ownership 16CI/CD 14Migration 12Reviews 12Clients 9Engagement 8Pricing 7Total
1ThinkSys● 75● 100● 100◐ 50● 75● 75● 75● 7579.5
2DeviQA● 85● 75● 85◐ 50● 90● 85● 85◐ 5077.4
3BetterQA● 75● 80● 85○ 0◐ 65◐ 65● 100● 7568.1
4Codoid◐ 65● 75● 75◐ 50◐ 50◐ 50◐ 50◐ 5060.8
5QASource◐ 50● 75● 75○ 40◐ 65○ 40● 85◐ 5060
6TestFort◐ 60○ 0● 75● 75● 75◐ 65● 75◐ 5057.1
7TestDevLab◐ 65◐ 50● 75○ 0● 75◐ 50● 85◐ 5056.6
8a1qa◐ 50● 75● 85○ 0◐ 65◐ 50◐ 50◐ 5054.7
9QA Madness◐ 50◐ 60● 75○ 0◐ 65◐ 50● 75◐ 6053.6
10Testlio○ 35◐ 50● 75○ 25◐ 65◐ 60● 85◐ 5052.7
11Abstracta◐ 65○ 0◐ 60○ 40● 75◐ 60◐ 60◐ 5050.2
12Testrig Technologies● 75○ 0● 75● 75◐ 50○ 25○ 25◐ 5049.8