We’ve hit a wall with our fraud detection setup, and I think I finally understand why. We were training our models on global engagement patterns—follower growth curves, like-to-comment ratios, posting frequency—but we weren’t accounting for regional norms.
In Russia, it’s completely normal for a lifestyle influencer to post 3-4 times a day. In the US, that looks suspicious. An audience that’s 60% from Russia and 40% from Southeast Asia is healthy for a Russian creator but might be a red flag for a US-focused brand. We were flagging legitimate creators as high-risk because we were using one-size-fits-all thresholds.
So we’ve started doing something different. We bring together experts from different regions—our Russian team, our US partners, some consultants with regional knowledge—and they surface real examples of bot networks, engagement pods, and authentic growth patterns they’ve seen in the wild. We’re collecting these as training data, tagged by region and creator type (nano-influencer vs. macro, for example).
The AI model is getting better, but it’s not because we threw more compute at it. It’s because the training data is now grounded in actual expert knowledge about how engagement actually works in different markets.
Has anyone else tried this approach? How are you sourcing your training data for fraud detection models?
This is exactly what we’re struggling with in our expansion. We have great data on the Russian market, but when we tried to scale to Europe, our fraud models went haywire. What looked like normal engagement in Moscow looked like artificial inflation in Berlin.
Your approach of involving regional experts in labeling training data makes sense. But I have a practical question: how many examples do you need before the model actually gets reliable? We’re bootstrapping this, so we can’t hire experts in every region. Are you crowdsourcing some of this data, or strictly using internal team knowledge?
Also, how often are you retraining? Fraud tactics evolve fast, and I’m worried about blind spots emerging six months down the line.
Strong observation about regional norms. We’ve seen this in our DTC campaigns—bot behavior looks different depending on the market’s influencer maturity level. In emerging markets, you see more aggressive pod activity. In saturated markets, the fraud is more sophisticated and harder to detect with basic metrics.
One thing I’d push back on slightly: be careful not to over-engineer your training data collection. We spent months building a crowdsourced annotation system across 12 countries, and it introduced more noise than signal because our crowd annotators didn’t always understand the context. We eventually consolidated to 3-4 regional experts who understood both the platform dynamics and the market. Much smaller team, much higher quality labels.
What’s your approach to drift? Are you monitoring model performance on new campaigns, or are you just running batch retraining on a schedule?
Here’s the real-world pain point: even with a good model, you still need humans to make final judgment calls. We’ve been using a two-stage system—AI flags potential fraud, then our regional teams do a 10-minute deep dive before we pass or fail a creator.
The benefit is that we’re collecting human decisions as additional training data. So the model gets better over time based on what our experts actually decided. It’s a flywheel. And honestly, it’s saved us from some bad partnerships that the AI alone would have missed—usually it’s the gut feeling of someone who knows the market.
My question for you: are you tracking false positives? That’s where we’re seeing the cost. Every legitimate creator we reject is a missed opportunity and potential reputation damage.
One more piece—be transparent about your vetting process with creators. We started sharing our fraud detection criteria (at a high level, not revealing exact thresholds) with partners. Creators appreciate knowing what we’re looking at, and it actually helps them understand industry standards. The legitimate ones are usually fine with it; the suspicious ones get defensive, which is itself useful data.