Three Ways In

Why it is not one switch

The common framing is that a model either is biased or has been fixed, as though there were a setting.

There is not, because bias enters at three separate stages and each is unavoidable in a different way.

The training data. Collected text is not a neutral sample of humanity. Some languages, regions, professions, and eras are enormously over-represented. Whatever patterns exist in that text become patterns in the model.

The labelling. People decided which responses were good. Those people had norms, context, and blind spots, and their judgements shaped what the model learned to prefer.

The tuning. Deliberate choices about behaviour: what to refuse, how to handle contested topics, what tone to adopt. Someone made those calls, reasonably, and they are still someone's calls.

The uncomfortable consequence

Because all three are unavoidable, there is no neutral model waiting to be built.

This is not a counsel of despair. It is a shift in the question.

"Is this model biased?" has no useful answer. "Where would this particular use be affected, and how would I notice?" does, and it is answerable in a couple of minutes.

Two different harms

These get conflated and they need different responses.

Representation. How groups are depicted. Every generated nurse is a woman, every executive the same demographic, every example name from the same culture. It shapes perception, and it shows up in customer-facing material.

Allocation. How opportunity gets distributed. Screening, ranking, scoring, recommending, prioritising. Someone gets an interview or does not.

Allocation is the more serious category, and it is also the one that triggers real regulatory obligations. If your AI use ranks or filters people, you are in the second category, and this stops being a matter of taste.

Which one are you in?

The sorting question:

Does the output describe people, or decide something about them?

Drafting a job advert is representation: check the language and the imagery.

Ranking applicants for that job is allocation: this needs proper evaluation, documentation, and probably a conversation with someone who owns compliance.

Most office use is the first. When it is the second, the bar is genuinely different.