RetentionOS
Who is likely to leave in 60 days, why, and whether an offer is worth the money.
- Role
- Solo — product spec, gap analysis, model, backend, UI
- Stack
- Python with no dependencies, logistic regression, exact Shapley values
- Timeline
- November 2026
- Status
- Prototype · 85 automated tests
A retention spec that could not survive its own numbers.
RetentionOS started as my own spec for business broadband. Before building it, I reviewed the spec: 39 gaps, 11 critical. No churn label, churn probability stored as an input, and an ROI formula that rewarded discounting everyone.
From the gap analysisThe mockup’s “net saved Rp840 million” for an account paying Rp45 million a month is impossible; the upper bound is about Rp396 million.
Four fixes to my own spec.
Each closes a gap from the review.
- 01
Protected value, not full CLV
Offer value = drop in churn probability × customer value. Stayers get the discount too, so blind discounting loses money.
- 02
Evaluation in the open
Holdout by account and time, grouped cross-validation, confidence intervals, a naive baseline and calibration — not one accuracy number.
- 03
Four-eyes approval
Large or unrecommended offers wait for a retention manager. Nobody approves their own offer, and a rejection needs a reason.
- 04
Nothing goes to the customer
It only drafts messages. The spec’s “apply offer & email” button had no control, so I removed it.
Calling the top 20% catches about 69% of churners.
| Finding | Value |
|---|---|
| Accounts flagged for the next 60 days | 31 (9 high, 22 watch) |
| Expected to actually churn | about 9 |
| Churners caught by contacting the top 20% | about 69% |
| Caught by the naive “usage down 30%” rule | 6% |
| Expected programme net value | +Rp97 million |
Synthetic data. Offer responses are assumptions, not A/B results; the model finds patterns the generator planted.
Four views around one call queue.
A risk radar, an account panel, cohort survival with RFM migration, an accuracy page, and an approval log.
Who works from the queue
| User | Main question |
|---|---|
| Account manager | Which of my accounts should I call first, and what should I offer? |
| Retention manager | Is this incentive worth its cost, and should I approve it? |
The guide page opens on the product name set behind a cut-out miniature town, the tallest roofs covering its base, over the line “Know earlier which customers may leave, and what to do about it.” As you scroll, the town rises over the name; all motion switches off when the device asks for reduced motion.
85 tests, and only 16 churn events to test against.
- Not yet: an A/B test of real offers, or accuracy on real customer data.
- Confidence intervals are wide. There is no authentication: the role picker is a demo mode.
What the 85 tests cover
Data and model, analytics and ROI, API contracts, security hardening, approval rules, and the facts in drafted messages: “no open tickets” is never written when ticket data is missing.