Calibrating AI fraud detection across languages—how I'm training models on bilingual data to catch regional differences

DTC & UGC Dynamos
1

I’ve been wrestling with AI fraud detection for influencer campaigns, and one problem kept biting us: our anomaly-detection model worked well for English-language accounts, but it was way off for Russian creators. The model was treating perfectly normal Russian engagement patterns as suspicious, and missing real fraud because it didn’t understand regional context.

The breakthrough came when I realized the problem wasn’t with the AI—it was with what we were training it on. Our training data was overwhelmingly English-language accounts from US platforms. We were asking the model to learn patterns from one ecosystem and apply them everywhere. Of course it failed.

So I started building a bilingual training dataset. I pulled real engagement data from both US and Russian creators (anonymized, consent from platforms, the whole process), categorized outcomes as “authentic,” “suspicious,” and “confirmed fraud,” and retrained the model.

What changed immediately: the model stopped over-flagging Russian creators. It learned that Russian comment sections tend to have higher concentration of comments from a smaller pool of engaged followers—that’s normal there, not necessarily bot behavior. It learned regional patterns for posting times, engagement timing, and audience language diversity.

But here’s where it got interesting: the model also started catching fraud patterns it had completely missed before. Once it understood the language and regional context, it could identify coordinated inauthentic behavior with much higher accuracy because it knew what authentic-but-unusual looked like versus actually-fake.

I’m not replacing human judgment—the model flags potential fraud, regional experts review it, and we make final calls. But the quality of flags has jumped dramatically.

The hard part: getting good, representative training data. I had to work with local creators, agencies, and platform contacts to collect real examples of fraud in each market. That took time and trust-building.

Have you tried retraining fraud-detection models specifically for your markets, or are you mostly relying on pre-built tools that might be optimized for English-language accounts?

2

This is the exact problem we ran into when we started scaling campaigns to non-English markets. Pre-built fraud tools are almost always trained on English-language data, which means they have huge blind spots in other regions.

What we did similarly: we built a comparison framework. We took creators we knew were authentic and creators we knew were fraudulent in each market, and we extracted features: engagement rates, comment language diversity, follower growth trajectory, audience location distribution, etc.

Then we created market-specific baselines. A Russian creator with 5,000 followers might have 200-300 engaged comments per post and that’s healthy. A US creator with the same follower count might only get 50 comments, but that’s their normal. If you apply one standard, you flag the Russian creator incorrectly.

Question: How are you handling false positives in your model validation? When you retrained on bilingual data, did accuracy improve across both languages equally, or was improvement uneven?

3

Also—critical question—how are you dealing with data drift? Fraudsters evolve their tactics, platforms change their algorithms, and creator behavior shifts seasonally. Is your model static or are you continuously retraining? We found that without quarterly retraining, our model’s accuracy degraded by about 15-20% year-over-year.

4

This is exactly the kind of infrastructure problem we’re going to face soon with our European expansion. We’re small, so we can’t build production ML systems yet, but we’re already seeing that the fraud patterns in new markets are different from what we’re used to.

Have you found any shortcuts before going full ML? Like, specific heuristics or red flags that work across markets consistently, even without retraining the model? We’re looking for a scaling solution that doesn’t require us to become data scientists.

5

Great approach. For our clients, we’ve simplified this into a tiered system: we use a pre-built tool as a first filter (catches obvious stuff), then we run specific checks for each market (engagement rate thresholds, audience distribution, growth curve), then regional experts do final review.

It’s not as sophisticated as retraining a model, but it scales better for smaller agencies. Clients appreciate the transparency—they can see exactly why we flagged something.

Question: when you retrained your model, did you involve actual creators or fraud-affected brands in the validation process, or was it all internal?

6

I’m curious how creators like me show up in this data. Are you training on real accounts, or are fraudsters also in the training set? Because I worry that AI models sometimes conflate “unusual” with “fraudulent,” and creators who post at weird times or have uneven engagement for legitimate reasons get caught in the net.

For context: sometimes my engagement spikes because I posted a trending sound, or drops because I took a break. Is that kind of time-series volatility something your model learned to distinguish?