Lead scoring decay in seasonal markets: how to fix it

By KYN AI Advisory Team — AI implementation specialists, Singapore

Lead scores decay in seasonal markets because they are trained on a trailing window that lags the market. A lead that scored 40 in the off-season may be a 90 in the new cycle, but a model anchored to last year's weights still ranks it low. The fix has four parts: monitor conversion per segment, refresh the window on a schedule, require a second signal before loosening a threshold, and let a person override a score before it drives action.

Scope: this is a reasoned diagnostic, not a case study. We have not measured seasonal decay on a client's scoring model. The design patterns below come from reconciliation and monitoring systems we have built, applied to scoring.

Why does a trailing window fail at the turn of a season?

Most scoring trains on the last 12 months of outcomes. That works when demand is flat. It fails at the inflection, when demand shifts and the window has not caught up.

Typical consequences:

• fewer qualified leads convert;
• sales says "good" leads are dead ends;
• marketing spend chases a segment that peaked months ago.

Travel, retail, tax-linked services and any business with a demand curve are exposed.

How do marketing agents anchor to last year?

A marketing agent scores leads against historical weights: this channel converts at X, this segment at Y, this month at Z. The weights are fixed at build time or the last retrain. The agent applies them on every lead and never asks whether they still hold.

So the design question comes before the marketing question. Does the system treat its baseline as fixed, or as something it keeps testing against live signal?

What should a scoring system watch?

Three patterns from other systems carry over.

• Track per segment, not blended. In AI-visibility tracking, engines disagree, and averaging them hides the gap. A national conversion rate can look stable while one region, channel or cohort has already shifted.
• Reconcile to the live pipeline. In an outreach system synced to a CRM, every reply and step is logged in real time. A model compared against that live record can see its own predictions drift. A model checked only against its training set cannot.
• Keep the baseline alive. The signal set and the window need a scheduled refresh, not a one-off setup.

What are the warning signs of score decay?

Any one is a sign of drift. In a market with a real demand curve, season is the likely cause once others are ruled out.

• Segment conversion moves away from the blended average, but the dashboard shows only the blend.
• The training window has not changed since launch.
• Sales reports that high-scored leads convert worse, or low-scored leads close anyway.
• No path exists for a person to flag a disagreement before a lead is deprioritised.
• The model has never been tested on current pipeline data.

How should recalibration work?

Use two rules from reconciliation.

1. Ask over assume. A confident, stale score that buries a high-value lead costs more than an extra manual review. When a score diverges from live pipeline signal, escalate. Do not auto-resolve.
2. Loosen only with a second signal. When an exact match fails, the second pass widens tolerance but needs an independent signal such as a matching reference. For scoring: do not relax a threshold without cause when patterns shift. Require a second signal first, such as an engagement spike or faster stage progression.

Add a bounded review step: a person or second agent can override the primary score before action, with a capped number of rounds so scoring does not become a manual bottleneck.

The sequence:

1. Score diverges from live pipeline signal.
2. The system does not auto-resolve.
3. It asks, and checks a second signal.
4. A person clears or overrides.
5. The rolling window updates.

Which questions should you put to a vendor?

• How often is the training window refreshed: on a schedule, or when someone complains?
• Is drift monitored per segment, or as one accuracy figure?
• Is there a human override between a score and the action?
• When the model loosens a threshold, does it require a second signal?
• Was it validated on current pipeline data, and can the vendor show the test?

A model validated only on last year's labelled set has passed a sandbox. It passes the cases it was built for and misses the shift it was never shown.

What should you do before the next season?

1. Pull conversion by segment for the same weeks over the last two years.
2. Compare the shape to your score distribution.
3. Add a monthly per-segment drift report.
4. Name the person who can override a score.

Related: pipeline velocity illusions and marketing.

What does a monthly drift report contain?

Keep it to a page:

• conversion rate by segment this month against the same month last year;
• the score distribution of leads that converted, by segment;
• the share of high-scored leads that stalled;
• the share of low-scored leads that closed;
• the number of overrides by people and the reason given.

If low-scored leads close at a rate close to the high-scored ones, the scores no longer separate good from poor leads. That is the signal to refresh the window or adjust the weights.

Which approach suits a small team?

A small team does not need a complex model. A rules-based score with a few weights per segment, reviewed each month by a named person, is easier to inspect than a trained model. When weights are visible, a seasonal shift is a one-line change. When they are hidden inside a model, the shift stays hidden too.

What does a seasonal shift look like in numbers?

Illustrative example. The figures are made up to show how a blend hides a shift.

A tax-linked services firm scores leads from 0 to 100. The model was trained on the last 12 months. Conversion by segment, for the first three weeks of the new filing season:

| Segment | Average score | Wins per 100 leads last season, same weeks | Wins per 100 leads this season | Blended view | |---|---|---|---|---| | Sole proprietors | 38 | 4 | 11 | Hidden | | Small companies | 61 | 12 | 13 | Hidden | | Mid-size companies | 72 | 15 | 9 | Hidden | | All segments | 57 | 10 | 11 | Looks stable |

The blended figure moved from 10 to 11 wins per 100 leads. A dashboard that shows only that line reports no problem. Underneath, the lowest-scored segment nearly tripled its conversion and the highest-scored segment fell. The model keeps ranking sole proprietors low, so sales calls them last, and the season's strongest leads wait.

Nothing here required a model failure. The weights described last year accurately. The market moved.

How do you choose a refresh window?

A longer window is steadier. A shorter window reacts faster. Choose by the demand cycle, not by habit.

| Business pattern | Suggested window logic | Trade-off | |---|---|---| | Flat demand | 12 months, refreshed quarterly | Steady, slow to see change | | One annual peak | Same weeks in prior years plus a recent 6 to 8 week window | Needs at least two prior cycles | | Several short peaks | Per-peak windows, each compared with its own history | More windows to maintain | | Event-driven demand | Short window with an override path | Noisy, needs people in the loop |

A common design blends the two: a seasonal baseline from prior years and a recent window from live pipeline data. When the two disagree beyond a set gap, the system flags the segment and asks a person, instead of trusting either one.

What data do you need to start?

You do not need a large data science project. You need four things:

1. Lead records with the source, segment and created date.
2. Outcomes with dates: won, lost, stalled.
3. Stage history, so you can see how fast leads progress.
4. Overrides: when and why a person changed the order.

If stage history is missing, start recording it now. Without it, the system sees only the end result and cannot spot early drift. Settling that record is the same job described in what to build first.

How does the override path work?

An override is a person changing a score, or the order of work, before action. Design it with four properties.

• Quick. One action in the queue. If it takes five minutes, nobody uses it.
• Reasoned. A required reason from a short list, plus free text.
• Logged. Every override records the actor, the old score and the new one.
• Reviewed. The monthly drift report counts overrides by segment. A segment with many overrides tells you where the model is wrong.

Overrides are data. A model that never learns from them repeats the same mistake each season.

Who should own the score?

Name one person. Marketing builds the lead sources, sales works the leads and operations runs the system. Without an owner, each blames the other when the numbers disagree. The owner:

• reviews the monthly drift report;
• approves any change to weights or the window;
• decides what the agent may do alone, such as ranking, and what waits for a person, such as sending an outbound message.

Outbound messages wait for a named person. A score changes the order of work, not the right to send.

What does this cost to run?

Scoring a lead is a small task. In our live systems, similar narrow tasks cost cents: about S$0.01 per reply sorted and about S$0.02 per outreach email. A scoring pass has a similar shape, but the cost depends on the model and the amount of text per lead. Treat these figures as a guide, not a quote. The larger cost is the data work: recording stage history and outcomes consistently so the score has something sound to learn from.

When should you retire a score?

Retire or rebuild a score when:

• low-scored leads close at a rate close to high-scored ones for two periods in a row;
• the segments it was built on no longer match how you sell;
• reps stop looking at it, which the override and queue logs will show;
• the product or price changes enough that last year's outcomes no longer describe this year's buyer.

A score that people ignore is worse than none, because it still shapes reports.

What is the first step this month?

Pull last season's conversion by segment for the same calendar weeks and put it next to this season's. It takes an afternoon with the CRM export. If any segment moved more than the blend, you have found the decay. Then name the score owner, add the monthly drift report and agree the override path. Those three steps cost little and show quickly whether a larger change is needed.

Where KYN fits

KYN builds marketing and sales agents inside a system that runs in your own cloud account. Every action is logged with the actor. Outbound messages wait for a named person. Agents have their own restricted accounts. The first 14 days are a review of your stack and interviews with your team. A full build ships in under 90 days. Rights in the custom code and configuration pass to you on full payment.

Frequently asked questions

How often should a lead score be retrained?

On a schedule tied to your demand cycle, such as monthly or at each season boundary, plus a trigger when segment conversion departs from the score. A single yearly retrain lags the market.

Is a blended conversion rate misleading?

It can be. A stable blend can hide a segment that has moved. Track conversion by channel, region and cohort.

Should a person override scores?

Yes, at a defined checkpoint. The override also generates data on where the model is wrong.

Checked 30 October 2026.

Curious whether Marketing agents fits your business? Talk to KYN on WhatsApp — no forms, just a conversation.

Start Building →