I built 300 synthetic nurse personas across three specialties, anchored to real federal workforce data (the HRSA National Sample Survey of Registered Nurses), and ran them through an automated validation engine. The goal: could synthetic respondents reproduce the actual reasons nurses leave their jobs?
On the big relationship, it did great. When a nurse pointed to patient load as their breaking point, they almost always turned out to be a detractor. That is the kind of directional signal synthetic data is genuinely good at, and it matched what the federal data says about why nurses quit.
Then I looked at the eNPS question, the classic 0-to-10 "would you recommend this job." Across 300 personas, the first run produced four unique numbers, and 82% of the nurses piled onto just two of them.
Real nurses don't answer that way. Ask 300 of them and you get a spread: bitter 2s, tired but loyal 7s, the occasional 9. My AI quietly pulled everyone toward a few safe values. The technical term is variance collapse. The plain-English version is that the model mostly agreed with itself.
I kept the survey exactly the same and changed how I grounded the personas. Instead of leaning on a single source, I moved to a triangulated approach, pulling from several independent anchors so the personas weren't all shaped by the same reference point. Then I ran it again. And again.
Each pass, the spread widened. Four unique scores became five. The concentration on the top two answers dropped from 82% to 78% to 66%. The variance was coming back.
But here is the part that matters: even after all that, the scores stayed clustered. Better, clearly. Realistic? Not yet. The model still wanted to bunch people up, and no amount of grounding fully talked it out of that.
Every limitation in this study is documented in the full report, including:
Synthetic data doesn't fail loudly. It never throws an error. It hands you clean, reasonable-looking numbers that lean in a predictable direction: toward the average, toward agreeable, toward the center. If no one is specifically checking for it, it walks straight into a stakeholder meeting.
That is exactly why this work needs a second, independent set of eyes. The vendor who sold you the boost has little reason to mention that the variance is still collapsed under the hood. I don't sell the data, so I'm happy to be the one who checks.
One dataset, one week, one plain-language memo on where it holds and where it doesn't. No retainer.