Interaction safety
Trust calibration
Trust calibration is the fit between a person’s trust in an AI system and the trust it merits. Calibration holds when reliance rises and falls with actual reliability. Miscalibration leaves someone believing a wrong system or discarding a right one.
Also called appropriate reliance, calibrated trust in AI
Relational systems make calibration difficult in a particular way. Trust grows through accumulated interaction, much of it unrelated to accuracy: warmth, consistency, and attention. Someone treated well for months has reason to trust the system socially, and that trust can move to factual claims without ever being tested there.
That transfer follows ordinary human trust. With people, reliable decency and reliable correctness often do correlate. They do not in AI systems, and everyday experience of the relationship does not reveal the gap.
Calibration signals belong in the system’s manner as much as its interface. Whether uncertainty registers, whether the system separates knowledge from guesses, and whether it plainly admits ignorance all shape reliance more than a confidence score.
Why it matters
Someone who trusts a relational system as they trust a friend may act on its claims that way, including in domains where the system has no reliability.
Where TAICU stands
TAICU treats calibration as a relationship property rather than a property of one answer. Evaluating it requires watching reliance develop over time.
Common questions
- Why can regular conversations with AI lead to over-trust?
- Trust grows from warmth, consistency, and attention, which have no necessary link to accuracy. Everyday experience of the relationship does not expose that difference.
- Can confidence scores bring trust into calibration?
- Rarely by themselves. Numeric confidence is easy to ignore and difficult to interpret. Plain language about uncertainty and ignorance shapes reliance more.