We often use the word "hallucination" to describe how GenAI produces an answer that sounds confident, but is actually inaccurate or completely made up.
So why does that happen?
A helpful way to understand it is this: GenAI doesn't "look up facts" the way humans do. Most of the time, it is predicting what to say next, one small piece at a time, based on probability. That means there are usually many possible next words it could choose from.
When general users interact with GenAI through a web page or mobile app UI, the best way to reduce hallucinations is usually to write effective prompts and provide the right context to minimize them as much as possible.
But if you interact with GenAI by calling its API from your own code, there are actually two "hidden" parameters you can use to control how stable the model's output is: top_p and temperature.
They're called "hidden parameters" because, in most models that support them, they're optional and therefore easy to overlook. And from my experience studying GenAI development over the past few years, one reason they're treated as "optional" is that many AI platforms probably don't want developers to tweak these settings recklessly, get poor results, and then complain that the model is bad!
Below is a brief explanation of what top_p and temperature mean, and how to use them.
The definition of top_p and temperature
top_p: how wide the model's "allowed options" are?temperature: how random the model is when choosing from those options?
Briefly speaking:
top_pcontrols the size of the option pool.temperaturecontrols randomness inside that pool.
Why hallucinations show up
When the option pool is wide and randomness is high, the model has much more freedom to pick unusual word paths. That can be great for creativity, but it also increases the chances that the model sounds plausible while inventing details.
Practical takeaway
For tasks that must be accurate, such as facts, numbers, and policies, you want the AI to behave more conservatively.
For brainstorming, you can allow it to be more creative, but you should expect more variation and verify important claims.