1. A lost booking can still be real demand
Take three plumbing calls. One is a sales pitch from a vendor. One is a homeowner in your area with a leaking heater, and nobody in the office calls them back. One is a homeowner in your area who books, then cancels because a part is unavailable. If you call all three "bad leads," you mix up a junk call, a missed callback, and a job that stopped because a part was unavailable.
That distinction matters when a campaign manager wants to cut spend or a dispatcher feels blamed for low conversion. Use the AI to organize the evidence. One conversation is a weak basis for deciding who is at fault. Build separate fields for what the caller wanted, what the staff did and what later happened to the job.
Treat a score as a workflow input. A questionable call can enter a review queue, a coaching example or a targeting investigation. It should not silently trigger an employment decision or erase an otherwise valid lead from reporting. Record the transcript excerpt supporting each label so a human can challenge it.
2. Use a rubric the office can explain without AI
Start with what you can hear on the call. For demand, write down the service they asked for, whether the address is in your area, and whether they want to hire. For handling, note whether the person on the phone asked the right intake questions, explained the next step clearly, and handed the job to the right person. Booking, completion, and payment usually come from another system.
Skip a mysterious one-hundred-point score that mixes every dimension into one number. A qualified panel-upgrade caller may need a site assessment, so an installation appointment is the wrong next step on that call. A restoration safety escalation can be the right handling even when the job never becomes ordinary dispatch. The rubric should count both of those as good handling.
Write definitions with examples and exclusions. "Qualified" might mean an in-area person requesting work the company provides, with a reasonable next step. Wealth, fluent English, and taking the first price are the wrong tests, and those shortcuts bias the system and hide legitimate demand.
| Dimension | Permitted labels | Evidence source |
|---|---|---|
| Demand | In scope / out of scope / unclear | Requested service and location |
| Handling | Correct next step / missed step / review | Conversation and approved script |
| Outcome | Task / appointment / completed / unknown | CRM event and job identifier |
| Follow-up | Owned / overdue / not needed | Assigned task and due time |
3. Build a small reference set before trusting the classifier
Select calls across trades, time windows, languages, lead sources and outcomes. Include difficult cases along with the clean recordings. Two reviewers should label them independently, discuss disagreements and document the final reason. If people cannot agree using the rubric, buying a more expensive model will not solve the ambiguity.
Keep a separate holdout group that you did not use to tune the prompts. Score that group again after you change the instructions. Otherwise a tool can look accurate because the instructions memorized the original examples. Use lawful recordings and transcripts, remove identifiers you do not need, and limit access to the staff who need the material.
For a small operator, begin with a manageable set and expand when new failure patterns appear. There is no magic sample count that establishes reliability. Cover the decisions the system will influence, including the rare cases with expensive consequences.
- ☐ Define labels before collecting test calls
- ☐ Include short, noisy and interrupted conversations
- ☐ Include multilingual calls if served by the business
- ☐ Separate prompt-development examples from holdout evaluation
- ☐ Keep human disagreements and their resolution
- ☐ Retest after prompt, model or transcription changes
4. Worked example: why overall accuracy can mislead
Illustrative audit: a holdout set contains twenty calls humans consider qualified and twenty they consider unqualified. The model marks eighteen qualified calls correctly, misses two, and incorrectly promotes five unqualified calls. It correctly rejects the remaining fifteen. Overall agreement is thirty-three of forty, or 82.5%.
That sounds respectable until the consequences are separated. The five false positives could teach bidding to buy more junk. The two false negatives could hide real opportunities from a busy office. Decide the tolerance for each action instead of celebrating the single accuracy percentage. A coaching queue can accept more uncertainty than an automated budget change.
Go through the mistakes. Were price shoppers labeled unqualified even though they later booked? Did background noise scramble an address? Did a CSR say "you are booked" without saving an appointment? Those are findings the office can act on. A vendor's accuracy claim, with nothing behind it, will not answer those questions for your shop.
5. Translate labels into real conversion events
Google's current documentation distinguishes qualified leads, which have been further qualified offline, from converted leads, which completed a business-defined conversion step. A closed sale is a common converted event, but the definition must be explicit. Do not rename every model-positive call a converted lead.
Join the call record to the CRM with identifiers you trust, and combine repeated contacts about the same job. Keep the source and the timestamps. An estimate set today and a sale completed next month should stay connected, and that job should count once. When a salesperson corrects a status, give them a controlled way to correct the report too.
For enhanced conversions for leads, Google's 2026 implementation checklist emphasizes setup validation, daily uploads, appropriate match keys and diagnostics. It recommends starting a new action as secondary while validating it. Follow the current supported import route for your account; do not copy a stale API recipe into production.
Protect customer data and follow applicable consent and platform terms. Hashing a field is not permission to collect it, and a transcript does not belong in an advertising upload simply because the scoring system processed it. Send only supported event and matching fields through an approved integration.
6. Turn scores into specific coaching
A weekly review can focus on one behavior: asking for location early, setting an honest arrival window or assigning follow-up ownership. Play a lawful, appropriately access-controlled example and compare it to the approved intake process. Avoid turning a dashboard into a public ranking of employees based on imperfect labels.
If staff consistently lose bookings because no appointment slots exist, the action is to address capacity or advertising coverage. If many calls request an excluded service, inspect targeting. If a qualified caller waits for a callback that nobody owns, repair task routing. The useful result is a diagnosis you can act on. Another scorecard on the wall does not change the calls.
Give dispatchers and technicians a way to question a score. They often know facts the transcript never captured. A warranty caller may be a returning customer, so score that call apart from a new lead. When someone corrects a score, update the definitions and the evaluation set, and keep the correction after the disagreement is settled.
7. Pilot read-only, then expand carefully
Start by tagging records and producing an exception queue without changing bids, sending messages or overwriting official outcomes. Compare labels with completed work for a defined pilot period. Record review time as a cost. If the classifier saves five minutes and creates thirty minutes of cleanup, the office is slower.
Your scorecard should show label coverage, unknown rate, false positives, false negatives, time to correct errors and verified lead-to-job conversion. Also track whether an intervention improved the broken step. Also track whether the fix improved the broken step. If bookings rise after you add capacity, that capacity change can explain the result on its own.
Before any expansion, nominate an owner for rubric changes and access permissions. Lock the version used for each report. A quiet prompt change can otherwise make last month's lead-quality trend incomparable with this month's and send the team hunting for a marketing problem that is really a measurement change.
Questions to ask a scoring vendor
Ask whether the product stores recordings, transcripts, extracted fields or all three; who can access them; and how long each is retained. Ask how a customer deletion request propagates to backups and subcontractors. Avoid uploading historical calls until those answers and recording permissions have been reviewed.
Request an export of labels and the evidence behind them. If you cannot retrieve corrections or change your rubric without a sales call, the system may be hard to leave, and you may not be able to change the rubric yourself. Ask for a live demonstration on your holdout examples. Ask for a live demonstration on your holdout examples. A polished demo of another contractor's clean calls says little about your noisy intake.