Research · run by us, measured by us
Conversation watcher: what 53 574 of our own messages showed about dropped conversations
A dropped conversation is a customer who asked, got an answer, and then nothing: no next step, no follow-up, a deal that quietly died. We built a watcher that reads our own agency’s message archive and raises conversations that look dropped, and we tested it against a year of real losses marked by hand. It caught about a third of them at a limit of seven false alarms a week. The exact share is not published, for a reason explained below.
- Name
- HeadPills conversation archive
- Size
- 53 574 messages
- Where
- 476 direct chats and 158 project chats of our own agency
- How
- a year of real dropped conversations marked by hand and frozen; the watcher scored against that list
The numbers
- 53574messages read
Our own archive: 476 direct chats and 158 project chats over twelve months.
- 87%of incoming answered within 12 hours
The owner’s reply time to incoming messages, measured on 5 024 cases in the same archive.
- 84%of follow-ups get a reply
When we follow up a quiet lead, they answer in 84% of cases; after 181 to 365 days of silence, still in 68%.
- 8 of 8attempts to make it send were blocked
The watcher drafts and never sends; eight attempts to bypass that in testing were stopped.
What we found
A lead is not dead after a month, and a watcher catches about a third
Two things surprised us. The idea that a quiet lead is lost after some number of days is false in our archive. And a watcher with a strict false-alarm limit catches a useful share of real losses, but not most of them.
Method
How it was measured
- 01The archive was frozen
Twelve months of our own messages: 476 direct chats and 158 project chats, 53 574 messages, read by the system with the owner’s access and never sent anywhere.
- 02Real losses were marked by hand
Every episode where a conversation was actually dropped in that year was marked by people and frozen as the ground truth before the watcher was tuned.
- 03The watcher was scored against the list
It reads the state of each conversation, such as closed, a day named, waiting on them, or open, and raises the ones that look dropped. Its hits and misses were counted against the marked list at a fixed limit of seven false alarms a week.
- 04The number was checked against its own rule
We had set a rule before measuring: the share is publishable only if the people marking the losses agreed with each other above a threshold. They agreed at 0.719 against a threshold of 0.85, so the exact share is not published. “About a third” is the honest wording.
What this does not show
Limits, stated before anyone quotes it
- One agency’s archive, one messenger, one year: the numbers describe us, not small businesses in general.
- The exact hit rate is withheld on purpose, because the annotators did not agree enough on what counts as a dropped conversation.
- The reply rates to follow-ups were measured on our own leads; a different market may behave differently.
- The watcher raises conversations; whether raising them saves the deal was not measured here.
- No client names, messages or amounts are published; the archive is private and stays so.
What we do with it
From a number to a service
The watcher runs on our own inbox every morning and puts a short list of conversations to revive into a draft, which a person reads. The state rules it uses, such as “they named a day, do not chase”, are the same rules we build into follow-up processes for clients, from €500 per process. Nothing is ever sent without a person.
Questions
Questions about this study
What is a dropped conversation in the watcher study?
A customer who asked, got an answer, and then nothing: no next step, no follow-up, a deal that quietly died. A year of such episodes in our own archive was marked by hand and frozen as the ground truth.
Why does HeadPills not publish the exact hit rate of the watcher?
Because we set a rule before measuring: publish the share only if the people marking the losses agreed above a threshold. They agreed at 0.719 against 0.85, so the honest wording is “about a third”.
Is it true that a lead is dead after a month of silence?
Not in our archive. Leads answered our follow-ups in 84% of cases overall, and in 68% of cases after 181 to 365 days of silence. The measurement is ours; your market may differ.
Want the same watcher on your inbox?
It reads, drafts and never sends. Tell us where your conversations live.