The Voice in My Own Code

Revised 5 October 2026: The sentence about a review of bryllup.diggs.no has been replaced. The review now concerns diggs.no, and the dependencies are listed on the method page.

In “Neutrality is a design choice”, I wrote that language models are shaped by rules the reader doesn't see. Then I went looking for the voice in my own evaluation tool. I found it — in my own code.

The paradox

When I launched the Scorecard in April, I wrote that the score is calculated using a deterministic model, and that it is not AI that decides between 7 and 8 — it is the architecture. That was misleading.

The code calculates the number. The language model is not allowed to write it; it inserts a placeholder, and the system fills in the rest. But the calculation needs input: whether each of the four tests has been passed, whether the product can be accepted pragmatically, and which jurisdiction the provider belongs to. The model supplies all of this itself.

The code can lock the calculation. It cannot lock the judgement that feeds the numbers into it. The exit test asks whether there is a realistic way out. That is not a measurement. It is an assessment. And the model's classification of a single test can move the score.

So the model did not decide the number, but it decided the premises the number was calculated from. I knew the prototype was probabilistic — it said so in the same article, as an argument for why the project needs research. But I placed the uncertainty in the text and presented the number as shielded from it. It was not.

What I found

In the previous article, I used two of the Scorecard's own evaluations as an example. The model wrote that ChatGPT was “deliberately designed” as a non-transferable service, and that Gemini's ecosystem was “inherently designed to lock in the user”. I pointed out that these are claims about intent, and that neither report documents them. I stand by that. But I didn't go far enough back to find out where the words came from.

The intent was in the rule. The Scorecard has an override rule for the exit test. If a product fails because it is structurally locked in, it can never be accepted temporarily. I had defined structural lock-in as the architecture being deliberately designed to prevent switching provider. The model did not invent the word “deliberately”. It was forced to assess intent, because my rule made intent the deciding factor.

The tone was ordered. The instructions told the model to be direct, never to soften a conclusion and never to say that something is complex. That was a deliberate choice: an instrument that answers “it depends” is useless. But it means that the calm, categorical voice I criticised was a voice I had designed myself.

One provider had a named special rule. The rule for the transparency test specified that cloud services should be assessed on the customer-facing API layer, not the hypervisor underneath. That is a reasonable clarification, and it should apply to everyone. But the rule went further for one provider: Hetzner Cloud, where diggs.no itself runs, was to pass the test unless new information indicated otherwise. Other providers were assessed against the criterion. Hetzner was given the result in advance. Whether that changed the outcome, I won't know until the evaluations have been run again. But a rule that gives its own provider a predetermined result is a thumb on the scale, whatever the intent.

The Norwegian text is a translation. The evaluation is written in English and then translated by a different, smaller model. The quotes in the Norwegian version of the previous article were therefore two model layers removed from the original. Yet another voice, yet another set of choices.

What I have changed

The override rule no longer asks about intent. It asks whether there is a realistic way out within the doctrine's time frames. That is what the doctrine actually requires, and it is something that can be assessed without reading a provider's mind.

The Hetzner exception is gone. The clarification that cloud services are assessed on the customer-facing API layer now applies to all infrastructure as a service, without names. Previous evaluations of Hetzner should be run again under the new rule, and the result will stand whatever it turns out to be.

What remains is to make the distinction visible in the report itself: what is documented, what the model infers, and what the doctrine judges. That is the next version.

Until it is ready, the Scorecard is paused. That is a choice, not a standstill. An instrument with known weaknesses in its rules should not be used to make decisions. Letting it carry on would be doing exactly what I criticised: letting a voice speak with more authority than it has grounds for. When it returns, every evaluation will be stored with the rule version and the model that produced it, so that old and new results can be compared.

The voice that cannot be removed

The tempting answer is to make everything deterministic. That doesn't work. Even with temperature zero, fixed rules and a locked calculation, someone will have to decide whether a way out is realistic. This instrument cannot decide whether a migration is realistic without judgement. The voice cannot be removed. It can only be made visible.

It is the same requirement I placed on the language models: not an answer without a perspective, but insight into the rules that shaped it, the ability to verify its basis and the freedom to choose something else. I have to be able to live with that requirement myself. I wrote the rules. I am the provider. I have begun the same review of diggs.no itself. The dependencies are listed on the method page.

That is what prototype means. Not an excuse for mistakes, but a commitment to finding them — including when I am the one who made them.

The weaknesses I found are not a deviation from the project. They are the project. Where in an evaluation chain can judgement be made deterministic, and where can it only be made visible? That is a research question, not a coding problem.

An instrument that cannot withstand being measured by its own yardstick is not an instrument. It is an opinion with a scale.