Choosing a technical SEO agency is a procurement problem disguised as a marketing decision. The vocabulary is unfamiliar, the deliverables are hard to inspect before purchase, and the people who will judge the work — usually developers — are rarely in the sales meeting. So the decision gets made on rapport and price, and the failure shows up nine months later as a folder of unread PDFs.
The framework below is a structured way to make that decision defensible. It is an editorial procurement model, not a measured industry benchmark: ten weighted categories totalling 100 points, with the evidence to request in each. Adjust the weights to your situation, but score every shortlisted agency against the same version of the sheet.
How to run the evaluation
- Write one brief and send the identical version to every shortlisted agency.
- Shortlist three. Fewer gives no comparison; more collapses into noise.
- Score independently — ideally one marketing scorer and one technical scorer — then reconcile.
- Score only what is evidenced. An unanswered question scores zero, not "probably fine".
- Record the decision and the reasoning at the end, before anyone signs anything.
Category 1 — Diagnostic capability (15 points)
The core competence. Can they find the real cause, not the visible symptom?
- Explains crawl, render and index as three separate stages, and can say which one a given problem lives in. (4)
- Compares source HTML against rendered DOM as standard practice, not on request. (3)
- Uses log files or CDN logs where available, and says plainly what they cannot conclude without them. (3)
- Reconciles crawl data with Search Console page indexing categories rather than treating them as separate reports. (3)
- Distinguishes correlation from cause when discussing past traffic changes. (2)
Evidence to request: a redacted example of a non-obvious issue they diagnosed, including how they proved it.
Category 2 — Evidence standards (12 points)
Every finding should be reproducible by a sceptic.
- Findings include a specific reproducible example URL and method, not just a category name. (4)
- Severity is justified by expected commercial impact, not by tool defaults. (3)
- Screenshots and exports are dated and sourced. (2)
- States confidence levels and known unknowns explicitly. (3)
Evidence to request: a full redacted issue register from a past engagement. Read three random rows end to end.
Category 3 — Developer handoff quality (12 points)
This is where most engagements quietly die. A recommendation that a developer cannot action is not a recommendation.
- Writes tickets in your team's format, with a title, context, change specification and acceptance criteria. (4)
- Names the affected template or component, not just affected URLs. (3)
- Provides a test plan or QA steps per ticket. (3)
- Willing to join sprint planning or refinement sessions. (2)
Evidence to request: two redacted tickets. Show them to your own lead developer and ask whether they could pick them up without a meeting. That single test is more predictive than any reference call.
Category 4 — Prioritisation and impact reasoning (10 points)
- Ranks by expected impact against commercial pages, not by issue count. (4)
- Accounts for implementation effort and platform constraints in the ranking. (3)
- Explicitly names what they recommend not doing, and why. (3)
An agency that never tells you to skip something is not prioritising; it is listing.
Category 5 — Measurement discipline (10 points)
- Defines leading indicators — index coverage, rendered parity, Core Web Vitals pass rate — separately from lagging revenue metrics. (4)
- Establishes a baseline before work begins, with dates recorded. (3)
- Reports on segments and templates rather than a single sitewide line. (3)
Ask how they would report a month where technical health improved and traffic did not. A good answer describes both, in that order, without embarrassment.
Category 6 — Migration and release QA (10 points)
Weight this higher if any replatform, redesign or domain change is on your roadmap. Migrations are where technical SEO engagements either prove their value or produce their largest disasters.
- Requires staging access and pre-launch crawl comparison. (3)
- Maintains a URL mapping with single-hop redirects and a validation pass. (3)
- Runs a launch-day checklist and a post-launch recrawl within 48 hours. (2)
- Has a documented rollback trigger and knows who can pull it. (2)
Category 7 — Platform and stack fit (8 points)
- Demonstrated work on your CMS or framework, including its specific limitations. (3)
- Comfortable with client-side rendered applications and the trade-offs of each rendering strategy. (3)
- Knows what your platform will not allow, and proposes achievable workarounds. (2)
The strongest signal here is an agency telling you early that something you want is not possible on your current platform.
Category 8 — Communication and governance (8 points)
- Named day-to-day contact who is also the person doing the work, or an explicit escalation path if not. (3)
- Fixed cadence with written summaries, not only calls. (2)
- Decisions logged with rationale so a new team member can catch up. (2)
- Timezone overlap with your development team. (1)
Category 9 — Transparency and independence (8 points)
- Explains what is uncertain and what data they do not have. (3)
- Makes no ranking guarantees and does not imply special relationships with search engines. (3)
- Discloses tooling and any commercial relationships that could shape recommendations. (2)
Category 10 — Commercial terms (7 points)
- Scope defined by templates and workstreams, with a stated process if scope changes. (3)
- Implementation quoted alongside diagnosis, so the report has a route to production. (2)
- Reasonable exit terms and full handover of data, documents and access. (2)
Interpreting the total
Treat the score as a structured conversation, not a verdict — but the bands are useful.
- 85–100: strong fit. Proceed, and hold them to the standard they described.
- 70–84: workable with named gaps. Write the gaps into the contract as requirements.
- 55–69: risky. Usually means good analysis with weak handoff, or the reverse. You will be supplying the missing half.
- Below 55: do not proceed on price alone. The gap between a document and a shipped outcome is the whole cost.
One override rule worth keeping: any agency scoring zero on developer handoff should be disqualified regardless of total. Nothing else in the sheet survives a deliverable your team will not use.
Interview questions that produce real answers
Generic questions get rehearsed answers. These do not:
- Walk me through a technical issue you diagnosed that the client's own team had missed. How did you prove it?
- Tell me about an engagement that did not work. What would you do differently?
- Our developers have limited capacity. Which three findings would you insist on, and which would you park?
- How do you verify that a fix actually shipped correctly?
- What would you need from us in the first two weeks, and what happens if you do not get it?
- If our organic traffic dropped 30% overnight, what are your first four checks, in order?
- Which of your recommendations do clients most often ignore, and why?
The last two are the most revealing. Triage order exposes how someone actually thinks under pressure, and the "ignored recommendations" question separates agencies that have learned from delivery reality from those that have only written reports.
Evidence pack to request from every finalist
Ask for the same five artefacts from each shortlisted agency, redacted as needed:
- A sample issue register with at least twenty rows.
- Two developer tickets, complete with acceptance criteria.
- A migration QA checklist from a real launch.
- One monthly report, so you can see how they discuss a flat month.
- A short case narrative with baseline, intervention, verification and outcome.
Agencies that cannot produce these are usually not hiding them. They usually do not have them, which is the answer you were looking for.
Do a small technical check yourself first
Walking into evaluation meetings with an independent read on your own site changes the dynamic considerably. You do not need to diagnose anything — you just need enough baseline that you can tell whether an agency is describing your site or reciting a template.
Record the decision
Before signing, write a short decision record: who was evaluated, the scores, the deciding factors, the known gaps, and what success looks like at 30, 90 and 180 days. It takes twenty minutes and it does three useful things — it exposes weak reasoning while you can still change your mind, it gives the incoming agency an unambiguous definition of success, and it makes the next review an evidence-based conversation instead of a mood.
Where this fits
We publish this scorecard because we are comfortable being scored against it. If you are evaluating technical partners, the same sheet should be used on us and on everyone else on your list. Our own technical work sits within our practice as an SEO company in Vancouver, the verification loop runs through marketing reporting, and past engagements are documented in our portfolio. If you want to talk through your brief before sending it out, contact us.
The agencies worth hiring are the ones that welcome the questions. That, more than any number on a scorecard, is the signal.
