Would you trust synthetic data for customer research?

Question

As I'm hunting Opinio.ai, I'm curious about the opinions of people regarding synthetic data. With the recent advance in generative AI, is it possible to generate synthetic buyer personas and ask their opinion about your messaging. Will you trust the result?

Nikola Kozuljevic · Accepted Answer

@my3_murthy how about many of these studies? :) Check them on OpinioAI blog - Academic Research (can't post a link apparently).

These are mainly scientific papers, so not much space for ebbing the findings. Some interesting nuggets there.

to quote one of them:

"Large language models exhibit societal biases associated with demographic information, including race, gender, and others. Endowing such language models with personalities based on demographic data can enable generating opinions that align with those of humans.

Building on this idea, we propose “random silicon sampling,” a method to emulate the opinions of the human population sub-group. Our study analyzed 1) a language model that generates the survey responses that correspond with a human group based solely on its demographic distribution and 2) the applicability of our methodology across various demographic subgroups and thematic questions.

Through random silicon sampling and using only group-level demographic information, we discovered that language models can generate response distributions that are remarkably similar to the actual U.S. public opinion polls.

Moreover, we found that the replicability of language models varies depending on the demographic group and topic of the question, and this can be attributed to inherent societal biases in the models. Our findings demonstrate the feasibility of mirroring a group’s opinion using only demographic distribution and elucidate the effect of social biases in language models on such simulations."

Tim · Answer

In my experience, AI/ML models trained on synthetic data may perform well on that data but fail to apply to or summarize real-world scenarios. They tend to miss real world details - huge in the world of engineering. These could lead to inaccurate predictions.

My3 Murthy · Answer

Maybe- I am totally open to it because I have seen the potential of synthetic data. What would turn my maybe to a yes is a study that shows me that the results you get from synthetic data and regular data sets is similar. hope that helps!

Would you trust synthetic data for customer research?

Replies