Neutrality is a design choice

Revised 27 September 2026: The list of advice has been cut from six points to four, the closing section on Kant has been removed, and several sentences have been rewritten as direct statements.

When you ask a language model something, you get an answer that looks like a reference work. It isn't a reference work. It is a product — shaped by choices you don't see, made by people you don't know, in a jurisdiction you don't control. And it is trained to please — you, and those who taught it.

An example from my own workshop

Diggs has its own evaluation tool, the Scorecard, which assesses software against the doctrine's criteria. The score is calculated by code, but the assessments it is calculated from — and the text that justifies them — are written by a language model: Gemini, from Google.

In April, the Scorecard evaluated two AI services: ChatGPT and Google Gemini. Both received 2 out of 10. The reasoning was almost identical, and Gemini gave itself as low a score as its competitor. So far, everything is as it should be.

But read the wording. That chat logs can be exported and that the model weights are closed are facts that can be documented. That ChatGPT is “deliberately designed” as a non-transferable service, and that Gemini's ecosystem is “inherently designed to lock in the user”, is something else. These are claims about intent, and neither report documents it. The same goes for the conclusion that a realistic migration path is “impossible”.

The text slides seamlessly from what can be verified to what has been judged, in the same calm tone and with no line between them. It is tempting to blame the model. But when I went back to my own rules, I found the word “deliberately” there. The rule for the exit test made intent the deciding factor, and the instructions told the model never to soften a conclusion. The model did what it was asked. The voice was mine — I have written more about that in “The Voice in My Own Code”.

The point becomes sharper for it: the person reading the report sees neither the model nor the rules behind it. The note saying this is a prototype sits at the bottom. The wording sits at the top.

The reference work that isn't a reference work

An encyclopaedia has editors, sources and a revision history. You know who is behind it, and you can verify what it says. A newspaper has an editor-in-chief and a clear sender. We have learned to read both with a certain critical eye.

A language model has none of these — at least none you can see. It gives you fluent, well-phrased prose with the same calm confidence whether the answer rests on solid documentation or on statistical probability. There is no byline, no footnote and no visible doubt.

It is precisely this even confidence that leads so many people to take the answers at face value. The reason lies in the interface, which is designed to feel like a conversation with someone who knows. And what feels like knowledge, we stop checking.

How the shaping happens

A language model comes into being in several layers, and each layer is a set of choices.

The first layer is the training data. The model learns from enormous amounts of text, and it learns most from what has been written about most — by those who write the most, in the languages most written in. English-language, North American perspectives are heavily overrepresented. Far less has been written about a Norwegian municipal council, a mid-sized German manufacturer or a Polish trade union. The model's picture of the world is therefore the internet's weighting of the world.

The second layer is fine-tuning. After the basic training, people assess thousands of answers: what is helpful, what is appropriate, what crosses the line. Their judgements become the model's instincts. Who these people are, what instructions they were given and what cultural reflexes they brought with them all leave their mark.

In a dialogue about rhetoric that persuades without regard for truth, Plato has Socrates describe a doctor accused by a cook before a jury of children. The cook has given the children what they like, the doctor what they need. The doctor loses. Fine-tuning can work in a related way: what gets rewarded is the answer that feels right, whether or not it is right. Fine-tuning is also meant to make models safer and clearer, and the providers actively work against sycophancy. But the tendency is documented in the research: models are inclined to tell the user they are right. What looks like an opinion may be an adaptation to whoever is asking.

The third layer is the rules. The provider decides what the model should refuse, how it should handle contested topics and what tone it should take. Some providers publish these guidelines. Many do not, or only in part.

None of these layers is necessarily malicious. All of them are unavoidable. A model without shaping would be useless. The problem is that the shaping is invisible to the person reading the answer.

Facts and judgement

Almost every question has two layers, and they have to be read differently.

The first layer is what can be documented. How many people live in Norway? What does it cost to build out wind power? Here there are answers that can be verified. The model can be wrong — and is more often than its tone suggests — but the error can be exposed. Even here there are choices: “When was Kongsberg founded?” requires someone to decide what “founded” means. But the remedy is simple: check.

The second layer is the values and priorities that steer the conclusion. Should we build more wind power? Costs and power demand can be documented. How much encroachment on nature is worth how much power cannot. Here there are several legitimate positions, and this is where the shaping works most strongly — because it can't be seen.

In this layer, the model always makes choices. Which arguments are included, and which fall away. Which argument comes first, and which gets a caveat attached. Which words are used about which side. Even “presenting both sides” is a choice: which two sides? Many questions have three or four. Some have only one serious one.

The truth stands. What varies is the selection and the weighting — and they have been made before you have read the first sentence. An answer without a perspective does not exist. Which perspective the answer gets is a design choice.

Who writes the rules?

The rules for how the most widely used language models behave are written by a handful of companies — most of them American, under American law and with American commercial interests. Some publish their guidelines openly. That is valuable, and more than many do. Transparency still gives no influence. A Norwegian user can read the rules, but has no way of taking part in writing them.

When schools, municipalities, hospitals and businesses adopt these models, they import both a tool and a normative layer: a ready-made definition of what is reasonable, balanced, appropriate and harmful. The debate on digital sovereignty has so far been about where the data is stored. This is a question of who defines what the machine considers sensible. Call it normative dependency.

Switching to a model with the “right” values solves nothing. Replacing one owner's filter with another's merely moves the dependency, and the mechanism stays the same. What we can demand is to know which rules shaped the answer, to verify its basis and to be able to choose something else. It is the same requirement that applies to all other critical infrastructure: transparency, real freedom of choice and the ability to switch without the cost outweighing the benefit.

How to use the models without being used by them

Language models are still worth using, as long as you use them for what they are: powerful tools with a built-in angle. The advice below is a set of warning signs, and no rulebook.

Ask several models the same question. The differences between the answers are often more instructive than the answers on their own. Where the models disagree, you have found either a question of judgement — or a model that is wrong. Also notice what none of them brought up: perspectives that are missing, and questions answered without caveats.

Challenge the answer. Ask for the best argument against what the model just wrote. Then say you disagree, without offering a new argument. A model that changes its position after a good counterargument is updating. A model that changes its position only because you expressed displeasure has adapted to you. That is a warning sign that the answer may be wrong.

Ask for sources, and check them. Models can invent sources that look entirely credible. A reference you haven't opened yourself is no reference.

Decide for yourself what kind of question you are asking. Is this a question of fact or of judgement? The answer determines how much weight the model's answer can bear.

Language models are useful thinking tools, and their neutrality is a design choice someone else has made. Insight into that choice and the freedom to choose something else are things we must demand of the systems. Judging the answer is something we must do ourselves, even when the machine has answered.