The Creator Health Score: One Number That Tells Us Exactly Where to Focus
Managing 30 creators, we couldn't tell which ones needed attention until it was too late. We built a composite health score from five API data sources that catches revenue cliffs weeks before they hit.
At three creators, everything was manageable. We knew each creator’s numbers by feel. We knew who was having a good week, who needed a push on content, whose chatter team was slacking. The information lived in our heads because three creators is a small enough portfolio that it fits there.
At twelve creators, cracks started showing. We were missing things. A creator who had been quietly declining for six weeks got a review call and we were blindsided by the numbers. We thought she was mid-table. She was nearly at break-even and trending the wrong direction.
At thirty creators, we had a genuine operational problem. There was no way to hold thirty creator performance pictures in working memory. We were triaging by whoever was loudest, or whoever happened to come up in conversation, or whoever we’d reviewed most recently. It was not a system. It was noise management dressed up as management.
The solution we built was a single composite health score for every creator — a number from 0 to 100 that synthesized revenue trend, fan retention, engagement, reach growth, and payout health into one signal. The score runs nightly. Every morning, we sort by health score ascending and look at the bottom five. Those are the creators who need attention today.
It caught a creator heading toward a revenue cliff three weeks before she hit it. That was enough time to change course.
Why One Number Works Better Than Five Dashboards
The instinct when building a monitoring system is to build comprehensive dashboards. Track everything. Show revenue, retention, engagement, reach, and balance side by side. Let the data speak.
The problem is that five dashboards require a human to synthesize them. When you’re looking at thirty creators, that synthesis takes time you don’t have — and it’s inconsistent. One week you notice the retention chart. The next week you’re focused on the revenue line and you miss that the retention chart has been quietly deteriorating for three weeks.
A composite score with defined weights forces you to make the synthesis decision once, at design time, rather than re-making it implicitly every time you review a creator. Once the score is defined, the daily review is five minutes: sort ascending, look at the bottom, act on what’s there.
The tradeoff is that you lose nuance. A creator can have a low score for very different reasons — a retention problem looks the same as a revenue trend problem at the top level. That’s fine. The score tells you who to look at. The breakdown tells you why.
The Five Components and Their Weights
We landed on these weights after three months of iteration. They’re calibrated to our agency’s business model — subscription-primary with meaningful PPV revenue — and may need adjustment for different creator profiles.
Revenue trend (30%) — The most important signal. A creator whose revenue has been declining for 21+ consecutive days is in a meaningfully different position than one who had a bad week. We use a 30-day rolling average compared to the prior 30-day rolling average to smooth volatility.
Fan retention (25%) — Subscriber count is a vanity metric. Net retention — how many fans are staying versus leaving — is what matters for long-run revenue stability. We track 30-day retention rate and compare it to the creator’s own 90-day average to detect deterioration.
Engagement (20%) — Open rates, reply rates, and PPV purchase rates. A creator can have a stable subscriber count but declining engagement, which is a leading indicator of future churn. Engagement decline precedes subscriber decline by two to four weeks in most cases we’ve observed.
Reach growth (15%) — New subscriber acquisition rate. A creator with strong retention but zero new fan acquisition is stable but not growing. Reach growth matters less than the other signals in the short term but is the primary driver of long-term portfolio value.
Balance health (10%) — Available payout balance relative to monthly revenue. Creators who are consistently drawing down their balance faster than they’re earning — or whose balance has been flat for unusual periods — sometimes have operational issues worth surfacing.
The Scoring Algorithm
import requests
from datetime import datetime, timedelta
import statistics
API_KEY = "your_api_key"
BASE_URL = "http://157.180.79.226:4024/api/v1"
headers = {"X-API-Key": API_KEY}
def get_statistics_overview(creator_id):
response = requests.get(
f"{BASE_URL}/statistics/overview",
headers=headers,
params={"creatorId": creator_id}
)
response.raise_for_status()
return response.json()
def get_fan_stats(creator_id):
response = requests.get(
f"{BASE_URL}/stats/fans",
headers=headers,
params={"creatorId": creator_id, "limit": 2000}
)
response.raise_for_status()
return response.json()
def get_payout_statistics(creator_id):
response = requests.get(
f"{BASE_URL}/payouts/statistics",
headers=headers,
params={"creatorId": creator_id}
)
response.raise_for_status()
return response.json()
def score_revenue_trend(overview):
rev_30d = overview.get("revenue30d", 0)
rev_prev_30d = overview.get("revenuePrev30d", 1)
if rev_prev_30d == 0:
return 50 # No baseline, neutral score
ratio = rev_30d / rev_prev_30d
if ratio >= 1.20:
return 100
elif ratio >= 1.05:
return 85
elif ratio >= 0.95:
return 70
elif ratio >= 0.80:
return 45
elif ratio >= 0.65:
return 25
else:
return 0
def score_fan_retention(fan_stats):
fans = fan_stats.get("fans", [])
if not fans:
return 50
active = sum(1 for f in fans if f.get("subscriptionStatus") == "active")
expired_30d = sum(
1 for f in fans
if f.get("subscriptionStatus") == "expired"
and f.get("daysSinceExpiration", 999) <= 30
)
total_recent = active + expired_30d
if total_recent == 0:
return 50
retention_rate = active / total_recent
if retention_rate >= 0.80:
return 100
elif retention_rate >= 0.70:
return 80
elif retention_rate >= 0.58:
return 60
elif retention_rate >= 0.45:
return 35
else:
return 10
def score_engagement(overview):
open_rate = overview.get("dmOpenRate30d", 0)
ppv_conversion = overview.get("ppvConversionRate30d", 0)
reply_rate = overview.get("fanReplyRate30d", 0)
open_score = min(100, open_rate * 200) # 50% open rate = 100
ppv_score = min(100, ppv_conversion * 500) # 20% PPV conversion = 100
reply_score = min(100, reply_rate * 333) # 30% reply rate = 100
return (open_score * 0.4) + (ppv_score * 0.4) + (reply_score * 0.2)
def score_reach_growth(overview):
new_subs_30d = overview.get("newSubscribers30d", 0)
new_subs_prev_30d = overview.get("newSubscribersPrev30d", 1)
if new_subs_prev_30d == 0:
return 50
ratio = new_subs_30d / new_subs_prev_30d
if ratio >= 1.30:
return 100
elif ratio >= 1.10:
return 80
elif ratio >= 0.90:
return 60
elif ratio >= 0.70:
return 35
else:
return 10
def score_balance_health(payout_stats):
balance = payout_stats.get("currentBalance", 0)
monthly_avg_revenue = payout_stats.get("avgMonthlyRevenue", 1)
if monthly_avg_revenue == 0:
return 50
ratio = balance / monthly_avg_revenue
if ratio >= 1.5:
return 100
elif ratio >= 0.75:
return 80
elif ratio >= 0.35:
return 55
elif ratio >= 0.10:
return 30
else:
return 10
def compute_health_score(creator_id):
overview = get_statistics_overview(creator_id)
fan_stats = get_fan_stats(creator_id)
payout_stats = get_payout_statistics(creator_id)
components = {
"revenue_trend": (score_revenue_trend(overview), 0.30),
"fan_retention": (score_fan_retention(fan_stats), 0.25),
"engagement": (score_engagement(overview), 0.20),
"reach_growth": (score_reach_growth(overview), 0.15),
"balance_health": (score_balance_health(payout_stats), 0.10),
}
weighted_score = sum(score * weight for score, weight in components.values())
return {
"creator_id": creator_id,
"health_score": round(weighted_score, 1),
"components": {k: round(v[0], 1) for k, v in components.items()},
"grade": "A" if weighted_score >= 80 else
"B" if weighted_score >= 65 else
"C" if weighted_score >= 50 else
"D" if weighted_score >= 35 else "F"
}
def run_portfolio_health_scan(creator_ids):
results = []
for creator_id in creator_ids:
try:
result = compute_health_score(creator_id)
results.append(result)
except Exception as e:
print(f"Error scoring {creator_id}: {e}")
results.sort(key=lambda x: x["health_score"])
print(f"\n{'Creator':<20} {'Score':>6} {'Grade':>6} {'Revenue':>10} {'Retention':>10} {'Engagement':>12}")
print("-" * 68)
for r in results:
c = r["components"]
print(
f"{r['creator_id']:<20} "
f"{r['health_score']:>6.1f} "
f"{r['grade']:>6} "
f"{c['revenue_trend']:>10.1f} "
f"{c['fan_retention']:>10.1f} "
f"{c['engagement']:>12.1f}"
)
print(f"\nNeeds immediate attention (score < 45):")
for r in results:
if r["health_score"] < 45:
print(f" {r['creator_id']} — score {r['health_score']} ({r['grade']})")
for comp, val in r["components"].items():
flag = " <-- LOW" if val < 40 else ""
print(f" {comp}: {val}{flag}")
return resultsThe Creator We Almost Missed
Three weeks into running the health score system, we got an alert on a creator who had been a consistent mid-table performer for two years. Her health score had dropped from 71 to 38 over eight days. Not a single component was catastrophically low — but revenue trend had slipped to 28, retention was at 44, and engagement had fallen to 35. All three moving together in the same direction at the same time.
Without the score, she would have looked fine in our weekly check-in. Revenue was still positive. Subscriber count hadn’t collapsed. She would have been a green light in any dashboard built around point-in-time snapshots rather than trend direction.
With the score, she was the first name on our review list Monday morning.
When we dug into the component breakdown, the engagement drop was the oldest signal. It had started declining 19 days before the retention and revenue signals appeared. Her fans were becoming less responsive, but the subscriber count hadn’t moved yet because most subscriptions were still mid-cycle and hadn’t come up for renewal.
The intervention was straightforward: a content strategy session, a new chatter who specialized in re-engagement, and a targeted PPV sequence for fans in the at-risk engagement tier. Six weeks later, her health score was back to 68. The revenue cliff we had projected — a 35% decline if the engagement trend continued through two full renewal cycles — never materialized.
Running It at Scale
We run the portfolio scan every night across all 30 creators. The script takes about four minutes on our infrastructure. Every morning, the output is the same format: creators sorted by health score, bottom five flagged for review, component breakdowns available for anyone in the D or F tier.
The daily review meeting is fifteen minutes. We look at the bottom of the list, assign an owner to each flagged creator, and move on. Most days, the flagged creators are the same ones as yesterday — a recovery takes weeks, not days — so the conversation is usually about progress rather than new discovery.
The health score has also changed how we talk to creators. When we get on a call with a creator who is trending toward a C or D grade, we have a structured breakdown of why. It’s not “we think your engagement is slipping” — it’s “your engagement component dropped from 65 to 38 over 21 days, and here is what the fan data shows is driving it.” The specificity of the conversation is different when you have a number with a decomposed explanation behind it.
Calibration Notes
Three things we’ve learned about maintaining the model:
The weights are not universal. We calibrated them for our creator mix. A creator who is pure subscription with no PPV revenue should have engagement weighted differently — PPV conversion is less relevant, and DM reply rate and content response rate matter more. We keep a separate weight profile for our top three creators whose revenue model differs from the rest.
New creators need a warmup period. A creator with 60 days of data produces noisy scores because the baseline comparisons are thin. We filter new creators out of the main ranking until they have 90 days of history, and track them separately with higher tolerance thresholds.
Watch for score manipulation. Once the team knows the score exists, there is a natural incentive to optimize for the score rather than the underlying business. If you catch someone gaming a component — for example, inflating new subscriber counts through promotions that attract low-quality fans who churn immediately — the score will eventually catch it through the retention component, but there’s a lag. Combine the score with qualitative judgment, not replace it.
For more on the individual components that feed into this score, see our posts on fan retention signals and PPV revenue optimization. For scaling this approach across a larger agency, see our scaling retrospective.
Thirty creators is too many to track by feel. The health score does not replace judgment — it focuses it. Every morning, you know exactly which creator needs your attention and why. That is worth more than any dashboard.
See the full analytics API capabilities on the pricing page, or start building your own health scoring system with the API documentation.