Synthetic data often looks tidy. It can produce smooth trends, consistent outputs, and clean-looking patterns. That can be appealing when you are under pressure to deliver answers quickly. 

But the real world is not tidy. 

Consumer sentiment moves for reasons that do not follow a neat curve. Categories get disrupted. A brand has a crisis. A competitor changes pricing. A new product becomes a cultural signal. News cycles alter mood and intent in a matter of hours. Seasonality shows up in unexpected ways. A synthetic model that has not kept up can keep delivering stable outputs while the underlying natural data has shifted. 

That is the core danger. It can look reliable because it is consistent, when consistency is exactly what you should be suspicious of in a volatile environment. 

One of the main limitations comes down to keeping up with time-based things happening in the real world. “Geopolitical events, seasonality, holiday season news, big scary news from a brand in the category” can be disruptors in how consumers feel and behave toward brands. If the model cannot cope with those changes, it “will mute or not show the reality that’s going on.” 

When those disruptions hit, the first question is not whether synthetic can generate data. It can. The question is whether the synthetic output is still reflecting reality, right now. 

The first place synthetic data breaks is time. 

A synthetic data set trained on a year of data may not be able to cope with real-world change in the underlying natural data. That is when decisions get risky, because teams are reacting to a signal that is no longer aligned with what consumers are actually experiencing. 

“A synthetic data set that’s been trained on, let’s say, a year or so of data may not be able to cope with that real-world change in the underlying natural data. And so it will mute or not show the reality that’s going on, and that can lead to incorrect decisions being made from the data,” says JT. 

This is where temporal drift starts to matter. Over time, the world changes even when there is no single dramatic event. If the model is not updated, it becomes less good at predicting the world around it, not just specific events. That is what JT is getting at when he calls out drift directly. “If you are not training your model regularly, then that’s the technical term, it will be less good at predicting the world around it in general.” 

Synthetic only holds up when updating is treated as part of the work, not as a nice-to-have. The model has to be trained and retrained with regularity. The workflow needs an established way of updating the model, and it needs a habit of checking synthetic outputs against the natural data they are meant to represent. 

This is also where teams get caught. Synthetic data is sometimes presented like a one-time solution: train it once, then run it whenever you need answers. That is the wrong expectation. Even if the output is fully synthetic, it still needs to be kept up to date with changes in the real world. 

The second place synthetic data breaks is when something new is happening. 

Markets do not only change in the center. They change at the edges, in niche segments, new occasions, and unusual behaviors that start small and then grow. Those are often the moments insight teams most want to see early. 

The problem is that synthetic data cannot respond to what it has not learned. “There are also edge cases and unusual behaviors that do occur in the real world that won’t be seen if we have a model that is not trained sufficiently to respond to that particular type of event.” 

That can look like a niche segment forming that the model does not recognize. It can look like a new occasion or category entry point that is starting to matter, but does not show up clearly because the synthetic system does not have the knowledge to respond in the right way. 

This is not just a measurement issue, but also a decision issue. If emerging behavior gets muted or missed, teams can end up optimizing against the past and calling it insight. The risk is not only missing what is new. The risk is being confident there is nothing new to see. 

This is another reason why natural data is important. You need fresh inputs from real people to reveal what is changing, especially in the early stages when patterns are still forming. Without that, the model can keep reproducing what it already knows. 

Bias is the third failure mode, and it is one of the easiest to underestimate. 

JT calls this out as a major issue. “Bias amplification is a big issue as well. The models will attempt to replicate and represent everything in the data set, including biases.” 

Those biases can come from response bias, question wording bias, or differences in how audiences behave within the survey itself. Synthetic data does not automatically clean those problems up. If anything, it can make them bigger. 

This becomes especially risky in situations where teams most want synthetic data to help, like low-incidence or niche audiences. If that audience is not well understood and not represented with good-quality respondents, “all of those errors and features and bias in that data will be amplified in the model,” warns JT. 

The uncomfortable part is that it can still look good. “We will test it and it will look good, but in reality, it’s amplifying the bias.” A clean output is not proof of a clean input. A stable pattern is not proof of truth. 

Bias amplification is why governance cannot be bolted on at the end. It has to be built into how synthetic data is trained, validated, and updated. Without that, synthetic data can scale the very issues research teams spend years trying to control. 

This creates a system where technology, data, research, creative, and media are all part of the same conversation. 

Greg describes this as “a matter of orchestration.” Each function brings something different, and the value comes from coordinating them, not isolating them. 

For many organizations, this is the real lesson: research and technology work best when the teams that use them work together. 

There is another break point that matters when the goal is innovation. 

JT’s concern is that novelty, newness, and relevance can be lost with poorly trained models. In creative testing, product innovation testing, and ad hoc pricing or concept work, synthetic data can be used to make tools faster and more cost-effective. But the trade-off is that the model will start to optimize toward what has worked before. 

That is not what creative teams want. “Aren’t we looking for the new ad that really catches the attention, not an ad that seems to catch the attention of the average past ad, which synthetic data might generate.” 

Novelty is time-sensitive. What feels fresh in one quarter can feel invisible in the next. So sensitivity to novelty depends on the same thing that drives the earlier failure modes. It depends on staying current and staying connected to real feeds of real consumers. That is why JT frames this as a pace and regularity problem, not a one-time analytics project. 

“This is not the old world of analytics of once and done like a traditional segmentation might have done. Now this is a real-time model that lives and breathes with real feeds of real consumers.” 

If you are using synthetic-enabled tools in innovation workflows, the question to keep in mind is simple. Is the system helping you find what is new, or is it quietly pulling you back toward what is familiar? 

Synthetic data can be useful. It can also lead to incorrect decisions when it is treated like a shortcut. 

Most of the failure modes are the same at the root. The real world changes and the model does not cope. It mutes or does not show what is really going on. Or something new starts forming, a niche segment, a new occasion, an unusual behavior, and the system does not respond in the right way. Or bias in the underlying data gets replicated and amplified. 

So, synthetic data only holds up when it is kept up to date and kept under control. That means training models regularly. It means having an established way of updating them. It means testing synthetic outputs against natural data and being clear about where synthetic has been used and why. 

It also means staying critical. Smooth and stable outputs can look reassuring, but stability is not the same as accuracy. If the world is moving, you want your data to show that movement, not wash it out. 

This is where research rigor matters. Teams that are grounded in market research standards and production discipline are already used to methodology, disclosure, and healthy skepticism. Synthetic data does not remove the need for that. It raises it. 

“Natural data first.” JT’s perspective comes from working in tracking workflows where decisions move quickly and data quality matters. Delineate is built around asking real people their opinions through surveys, because that remains “a great signal on the world around us.” Synthetic data can help, but it does not replace that foundation. It is an augment. 

In practice, synthetic data shows up across the survey supply chain at Delineate, from testing surveys to collecting and processing data, and then providing datasets for analytics. JT talks about using it where it adds value, like when sample sizes are small, and you cannot confidently slice hard-to-reach audiences. He also points to using synthetic data to train models in different ways and to support quality assurance, like spotting outliers and anomalies. 

The line is not about whether synthetic is allowed. It is about control. A synthetic dataset trained on past data “may not be able to cope” when the real world changes, and it can “mute or not show the reality that’s going on.” So it only holds up if it is kept up to date, with regular updating, testing, and control. And it has to be disclosed. 

JT is clear on what Delineate will not do. “We will never use synthetic data exclusively. We will never stop testing and controlling the use of it, and we will never fail to disclose when and where we’re using it.” 

The appeal of synthetic data is easy to understand. It promises speed, scale, and more analysis for less cost. In the right use cases, it can deliver real value. But JT’s caution is that it does not get a free pass just because it looks tidy. When the world shifts, a model trained on last year’s reality may not cope. It can mute or not show the reality that is going on, and that is how incorrect decisions get made. 

So the question is not whether synthetic can generate outputs. It can. The question is whether those outputs are still reflecting the real world, right now. That is why synthetic is not a one-and-done. It only holds up with regular updating, testing, and control, and with clear disclosure. 

At Delineate, the line is consistent. Natural data first, synthetic data as an augmentation, where it adds value. Never exclusively. Never without testing and control. Never without disclosure. And never in a way that blurs the difference between synthetic and natural data. 

If you are exploring where synthetic fits in your brand tracking, campaign tracking, or analytics workflows, Delineate can help you pressure-test it against natural data, set the right guardrails, and use it in a way that stays reality-based. Get in touch to discuss your use case. 

Related: Synthetic Data in Market Research: An Expert View on Why Natural Data First Still Wins