Varun V
Partner
Date
Read time
Read time
Bringing it down from four months to five weeks.
A commercial insurance business took 17.4 weeks to onboard a new hire , against a written onboarding programme that spanned 6 weeks.
Our 8 week engagement made the company’s documentation answerable, and removed 12 weeks from the ramp. This produced $228,000 to $300,000 in new recurring operating savings.
At a glance
Measure | Before | After |
Time to independent productivity | 17.4 weeks | 5.4 weeks |
Median time to an answer | 4.2 hours | 2 minutes |
Average senior specialist hours per week on repeat questions | 16.8 | 8.3 |
Questions resolved without a person | Not measured | 50% |
Gross operating cost removed, annualised - $293K to $330K
Cost to Operate Annualised - $ 45K - 65K
Net Recurring Operating saving - $228K - $288K
The situation
The client was a programme insurance business with approximately $210 million in annual revenue, writing commercial lines through agents.
The engagement was scoped to one function: the 69 people who sell and service the commercial book. That meant we focused on 34 producers, 22 client-service managers, nine operations specialists and four senior underwriting specialists.
The company also writes several hundred programme and class combinations spread across 14 states, where appetite, rate and eligibility change on filing cycles.
The stated written onboarding programme ran six weeks, but the on-ramp time, drawn from HR and CRM records for around 14 hires over 24 months, was 17.4 weeks.
To uncover the constraint better, we decided to start with live logging of working days and reconstructed 14 months from Slack and the service desk. This enabled us to capture 2,180 question instances that reached a senior specialist.
341 distinct questions accounted for nearly 72% of the volume. And almost 95 percent of those questions were answerable from the material the company had already written.
While the median time to answer was 4.2 hours, at the ninetieth percentile it was 31 hours.
In addition, our findings revealed that the docs had 147 live contradictions, where two or more current documents answered the same question differently, with nothing to indicate which took precedence.
On top of this, another ~2,000 retired pieces of content were still being returned by search: superseded rate pages, training decks and similar, most of which the company no longer promoted.
Our approach
About three weeks of diagnosis preceded any development or building. We had a consultant who liaised with the engineering team sit inside the function, log questions as they were asked, with the goal of re-constructing the ramp of most hires from the past two years.
After which we started building for around six weeks. We had connectors that read Slack, their shared knowledge drive, the filed rate, the rule library and the myriad of training modules in their existing systems of record.
Which culminated in us delivering four modules as part of the implementation.
Answer Objects
For each source, it was made into a record that carried the question it answered, the answer, the responsible team, an effective date, the jurisdiction and a link to whatever it supersedes.
Which meant the source doc is now preserved and linked, but never restated.
Every record was assigned an owner, and without an owner or an effective date, it cannot be used in returning an answer.
Retrieval
Four methods ran together.
An exact token search to find class codes, form numbers and endorsement references
Vector search to find a paraphrase
Recency breaks, which meant it had to be tied to a newer record
And finally a source authority that ranks a filed guideline above a training deck, linked to a chat thread
The system does not use an LLM to decide which source takes precedence, but rather uses encoded rules to choose the right action.
The Citation Requirement
Unless a retrieved record carries an effective date covering the question, no answer is returned.
In instances where none does, the system declines, routing the question to the right person and the team responsible.
The system has permission to say that it does not have an answer, as opposed to hallucinating. This feedback gets routed and gets used as training for better further interactions.
The Review Queue
Every declined question gets logged, and the owning team's answer becomes a new record with a date and an owner.
Repeated questions are ranked by frequency, which means the client reviews the gaps creating the most interruptions first.
The queue is the only way by which the answer corpus grows, and the client runs it.
Approximately 85% of the system's operations are deterministic: retrieval, comparison and rule checks.
LLMs are applied for two tasks: distilling docs into records, and assembling a cited answer from retrieved evidence.
Governance
These controls were set before development began, as fundamental guiding principles.
The system does not answer questions on binding authority, outright refusal or coverage.
On questions it answers, it states the written guidelines and the responsible underwriter's name.
Filed rate and rule content is returned as an excerpt, as is, with its jurisdiction and effective date.
Every answer is recorded with the question, the records it actually retrieved, along with the citations returned for said user.
Records are retained for approximately seven years, matching the client's existing errors and omissions policy.
The system does not morph into a source of truth. Instead, it goes about making the company's existing sources retrievable, dated and accountable.
The Results
12 users received the system in week 6. And we were able to complete the staged rollout to the remaining 57 users in week eight.
Here are some figures:
Measure | Before | After |
|---|---|---|
Time to independent productivity | 17.4 weeks | 5.4 weeks |
Median time to an answer | 4.2 hours | 2 minutes |
Ninetieth-percentile time to an answer | 31 hours | 2.9 hours |
Average senior specialist hours per week on repeat questions | 16.8 | 8.3 |
Questions resolved without a person | Not measured | 50% |
Answers declined for want of a citable source | Not measured | 9.1% |
Live contradictions in the corpus | 147 | 6 |
Weekly users, of 69 eligible | Not measured | 61 |
The ramp reduction has three parts, with the system accounting for one of them directly.
Licensing and carrier appointments take about three weeks, which remain unchanged.
Instructor-led product training fell from 4.2 weeks to 1.6 weeks, as the client re-wrote the curriculum once the reference material became self-serve.
The remaining 10.2 weeks reflect repeated knowledge gaps, answer delays and dependency on experienced staff, which fell to 0.8 weeks.
The system greatly reduced the need to use instructor time for information that could be retrieved safely from written material.
While a majority of interactions were greatly reduced, some continued to exist due to working relationships between colleagues.
Adoption was also treated as a management and product problem. Questions covered by the system were redirected to it before a specialist responded.
The Operating Model
The system was designed to run without an engineering team on standby.
Document changes are processed incrementally, meaning sources that remain unchanged are not re-processed.
Retrieval, comparison, authority ranking and effective date checks remain deterministic.
Engineering is retained for scheduled maintenance, connector changes and rule classes.
The firm assigns one person who dedicates very little capacity to route declined questions, confirm ownership and ensure approved answers carry an effective date.
The teams that are responsible for the underlying rules are responsible for their content.
The economics
Gross operating cost removed: $293,000 to $333,000 a year
Each line was reconciled by the client's finance team against the approved headcount plan and accounts payable.
Operating cost removed | Annualised |
|---|---|
Service and support requisitions deferred, 2 of 3 approved | $184,000–$208,000 |
Contract technical trainer discontinued | $68,000–$84,000 |
Onboarding content vendor not renewed | $41,000 |
Total | $293,000–$333,000 |
No positions were eliminated. First year realised savings are lower than the annualised total because two requisitions lapsed in months four and seven.
Cost to operate: $45,000 to $65,000 a year
The mature operating model has four cost components.
Operating cost | Annualised |
|---|---|
Infrastructure, retrieval, storage and monitoring | $8,000–$12,000 |
Model usage | $3,000–$8,000 |
Scheduled engineering maintenance and rule changes | $15,000–$20,000 |
Corpus ownership and review queue, 0.10–0.15 FTE | $19,000–$25,000 |
Total | $45,000–$65,000 |
Larger costs stem from maintaining the retrieval system and assigning a person to own the exceptions.
The operating model works on four conditions:
The firm continues to maintain the underlying source docs.
Document changes are processed incrementally.
The review queue is limited to declined and disputed questions.
Existing identity, security, storage and document systems are re-used.
Net recurring operating saving: $228,000 to $288,000 a year
Net of the cost to operate, the system returns $228,000 to $288,000 in recurring annual savings.
For every dollar spent operating the system, it removes approximately $4.50 to $7.40 in recurring operating cost, equivalent to a recurring operating ROI of approximately 350% to 640%.
We have refrained from a first year ROI calculation, as the said time period is yet to pass.
The Supported Upside Case
A third service requisition remained approved at the measurement cut-off date. It was not included in the operational savings.
If that requisition is also avoided, it would remove a further $92,000 to $104,000 in annual operating cost.
The resulting economics would be:
Measure | Supported upside case |
|---|---|
Gross operating cost removed | $385,000–$437,000 |
Cost to operate | $45,000–$65,000 |
Net recurring operating saving | $320,000–$392,000 |
Gross benefit-to-cost ratio | 5.9x–9.7x |
Recurring operating ROI | 492%–871% |
These figures are not included in the headline result.
What is not counted
Four senior specialists recovered a combined 34 hours a week, or approximately 1,768 hours a year.
No positions were eliminated, which means that time is reported as released capacity and is not given a dollar value.
Few hires reached optimal levels of independent productivity 10 weeks earlier. No dollar value is assigned.
Knowledge retained when a specialist leaves is not counted.
No revenue figures claimed.
Reduced exposure to outdated or contradictory guidance is also not valued.
The system reduced live contradictions from 147 to six, but the client had no accepted method for converting that reduction into a financial figure.
Where this is suited best
Best suited for functions roughly above 25 people, with several hundred product or rule variations and at least 12 months of searchable history.
Works in instances where documentation might be out of date, and correcting it is a pre-requisite rather than a part of this work.
The system works within the confines of an existing written playbook.
Question volumes, cycle times, ramp durations and adoption figures are drawn from the client's operating systems and from the deployed system's own logs.
Cost figures were reconciled by the client's finance team against payroll, accounts payable, approved requisitions and executed contracts.
Read also
