This is a story about building an anomaly detection system for influencer vetting and realizing that the technical problem is actually the easy part. The hard part is: what do you do when the AI finds something weird?
We built a system to flag unusual patterns in creator behavior—sudden follower spikes, engagement patterns that don’t match historical norms, audience composition shifts, that kind of thing. The AI works great. It catches real anomalies.
But then what?
Early on, we’d flag an anomaly and immediately escalate to a human reviewer. The reviewer would look at it, try to figure out if it’s a red flag or just normal variance, and make a judgment call. Problem: different reviewers would come to different conclusions about the same anomaly. We’d have inconsistency.
So we built decision trees. If anomaly type A + context B + risk score > C, then escalate. Otherwise, auto-pass. This helped with consistency. But it also meant we were now making automated decisions based on rules that were essentially arbitrary.
We got burned. Twice. We auto-passed creators who later turned out to have engaged in coordinated inauthentic behavior. Our decision tree didn’t catch them because the anomaly didn’t align with our pre-defined patterns. The tactics had evolved.
So we changed the approach. Now we:
1. Flag everything that’s even slightly anomalous. Don’t try to pre-judge what matters. Let humans see the full picture.
2. Give reviewers context, not just the flag. Instead of showing “engaging rate = 7.8%, historical avg = 4.2%, anomaly score = 0.78”, we show: “This creator posted a viral video 3 days ago. Engagement spike is proportional to virality. Audience composition stable. No red flags.” Context helps judgment.
3. Track what we get wrong. When a creator we pass ends up being problematic, or a creator we flag ends up being fine, we log it. We’re building a feedback loop to understand where our judgment is failing.
4. Update decision rules quarterly, not once and done. Fraud tactics change fast. Legitimate behavior patterns change fast. If your decision framework is stale, it’s going to fail.
The biggest insight: anomalies aren’t automatically problems. They’re just deviations from expected patterns. Most anomalies have explanations. A creator who suddenly gets 10x engagement probably went viral. That’s not fraud; that’s success. The AI flags it because it’s unusual. You need humans to understand why it’s unusual and whether that matters.
We’re getting to a place where the workflow works reasonably well:
- AI provides the signal (here’s what’s unusual).
- Humans provide the judgment (here’s why it probably matters or doesn’t).
- We learn together (when we get it wrong, we update).
But it’s not perfect, and I don’t think it ever will be. Fraud evolves. Tactics shift. You’re always going to have edge cases.
For anyone building similar systems: don’t expect AI anomaly detection to be a magic bullet. It’s a tool that helps you focus human attention. But it requires thoughtful integration with human judgment, and it requires continuous learning.
Have you built anomaly detection into your vetting workflow? What does that actually look like when you operationalize it? And what are the edge cases that keep tripping you up?