Last week I read something about a standard for writing technical documentation in plain, unambiguous English. I did not save the link and could not remember the exact name, nor did I have time to search for it.

This week, I made changes to a part of the codebase I wasn’t very familiar with, so I had to chat with Claude and Codex to understand it better.

I observed something that had been bothering me for a while: Claude keeps reaching for complicated words, abstractions, and slightly fancy descriptions instead of simple domain terms. Thus I half-remembered this idea, and I started saying:

Explain to me in Simple Technical English

I have typed this phrase many times this week, and I felt it worked very well for me, so I have put it in the user memory or in the system instructions for all agents.

Last night I remembered to search for fuzzy thing that I was rememering and found it: ASD-STE100 Simplified Technical English.

But this got me thinking: I wrote “Simple Technical English,” and this standard is called “ASD-STE100 Simplified Technical English”. If so, would my output be better if I had used the proper term?

Thus, I did some research about it. You can see the entire experiment I run on Claude and Codex here where I shared the prompts used, the code that I used for testing and the full results.

What is ASD-STE100

Simplified Technical English is a controlled natural language. It began in the late 1970s as AECMA Simplified English and was first released in 1986. The European airline industry wanted aircraft maintenance manuals that a non-native English speaker could not misread.

It is maintained by the ASD Simplified Technical English Maintenance Group, and it has two parts:

1. 53 writing rules in 9 sections, because grammar and sentence shape carry as much ambiguity as vocabulary does.
2. A dictionary of about 900 approved words, because a small vocabulary removes choice, and choice is where ambiguity enters.

The rule I like most is this one: each approved word has one meaning and one part of speech. As a non-native English speaker, this feels very good to me in interactions with LLMs.

I also liked the hard limits, such as the requirement that descriptive sentences be no more than 25 words. This is because the official goal is for STE to “make technical texts easier to understand for all readers” and “reduce Human Factor risks.”

It sounds great to me. If a maintenance manual is ambiguous, an aircraft is repaired incorrectly. The standard exists because vague technical prose has a body count.

LLMs are producing a lot of text, and reading it takes a lot of effort. Because of the fancy language they sometimes use (looking at you, Claude), there is a risk of misunderstanding or, even worse, a kind of understanding fatigue, and the chances of approving or going on with something wrong are increasing.

How I tested this

I chose 4 pieces of code from two of my own projects, and for each one I wanted to test 3 cases:

1. Control: no style clause at all. It defaults to all my skills and what I have installed. It is not a clean environment, but all the other experiments will have the same environment
2. Simple Technical English: this is the phrase that I used which is not naming the proper standard
3. ASD-STE100 Simplified Technical English: the standard name

Nothing else was added, removed, or changed in the entire system prompt, harness, or context. All agents got the same prompts in the same context and were running on the same setup I have.

I orchestrated all this via Claude, which ran the agents (including its own): three Claude and three Codex for each piece of code for each experiment. I ran two full sessions so we could we collected more data. I stopped after two sessions because this is not a rigorous study but rather something I intended to check.

The code that they got was looking like this:

My main complaint was that the model was “inventing” or picking abstractions that obscure real understanding, so I wanted to offer a couple of situations that could push the model in that direction.

One example of a run and the output

I will not write about each example run here, as it would become a very long article. But you can read all of them in the repo that I published at github.com/lucianghinda/llm-experiments.

The output

So allow me share the first one, as it is a simple concern for a controller that implements a idempotent write, and it includes a part that calls something like rescue_with_handler because rescue_from runs outside the around_action.

Here is the part that I give to LLMs to explain: why is there a begin block here:

# `rescue_from` runs at the controller level, *outside* this around_action,
# so any post-yield code is skipped when the action raises.
begin
  yield
rescue StandardError => error
  raise unless rescue_with_handler(error)
end

write_idempotency_entry(cache_key, body_hash) if response.status >= 200 && response.status < 500

I sent this to Claude and Codex with the same instructions for each agent. The prompts are a bit different between agents, but as the purpose was not to compare Claude with Codex but to compare them with themselves in terms of how they behave, I think it is good enough.

Here is Claude with no style instruction:

The problem is a layering one. rescue_from handlers registered by ErrorResponder are installed at the controller level, which is outside this around_action. If the action raises, the exception propagates out of the around block first, and everything after yield is skipped.

Here is Claude asked for Simple Technical English:

This block looks strange. The comment in the file describes the reason.

Rails runs rescue_from handlers at the controller level. That level is outside the around_action. If the action raises an error, Rails immediately leaves the wrapper. The code after yield never runs. The save step is lost.

And here is Claude asking for ASD-STE100:

Rails keeps the rescue_from handlers at the controller level. These handlers operate outside of the around_action callback. If the action raises an error, Rails goes directly to a handler. Thus, the code after the yield does not run.

I think I prefer the second one, but it depends on the person.

I have shared only a small part of the entire response here. You can read all of them here.

The numbers

I stripped the code fences, tables, and headings from every output and measured only the text; for me,e is the explanation the agent returned.

I have created this table. Each cell holds two separate measurements, run 1 then run 2, so 18.6 / 17.4 means the first run averaged 18.6 words per sentence and the second run 17.4.

target                 Claude ctrl    Claude STE    Claude ASD     Codex ctrl
-----------------------------------------------------------------------------
01 concern             18.6 / 17.4    8.1 / 8.3     9.7 / 9.8     11.7 / 13.9
02 method, domain      16.9 / 19.3    9.1 / 8.8    10.2 / 10.3    10.7 / 13.8
03 method, algorithm   17.5 / 16.3    9.3 / 9.9     9.0 / 10.3    11.9 / 10.5
04 query object        19.6 / 17.0   10.7 /  9.4   11.0 / 10.6    12.2 / 15.0

My reading of these results is:

1. Claude with no style instructions, 8 runs: 16.3 to 19.6 words per sentence.
2. Claude with either style: 8.1 to 10.7 words per sentence.

I will also share the results below for Codex, but I included the control here to show that, by default, even in the control experiment, Codex outputs fewer words.

I did not expect the control number to be that stable between the two runs, and maybe I should have run more runs, but that was the time I had. But one thing to notice is that it appears not to be so dependent on the code itself. The averages and median are both very, very close:

A second observation here is looking at the vocabulary that Claude used in the control group:

explicit(2) tradeoff fingerprint corrupting degrades snapshot
deliberately deterministic contract essentially dispatches
propagates installed layering subtle reconstruction fingerprints
namespaced engages machinery quota consequence deliberate
convention encoded

All of these are English words, and some we might use in day-to-day interactions, but together they create a fog.

Their usage went from 26 (the control) to 6 with the vague clause and to 1 with the strict one. In the second run, the same experiment gave 14, then 1, then 0.

While the numbers vary widely, I can see a trend I like: the use of those foggy words is decreasing.

About Codex

I run Codex via orchestration with Claude, but I also replicated the results by running them from Codex (a kinda 3rd semi-run - just Codex form inside Codex).

| Target | Codex ctrl | Codex STE | Codex ASD | Claude ctrl |
|---|---|---|---|---|
| 01 concern | 11.7 / 13.9 | 10.7 / 11.4 | 9.6 / 8.3 | 18.6 / 17.4 |
| 02 method, domain | 10.7 / 13.8 | 15.1 / 19.4 | 9.3 / 9.9 | 16.9 / 19.3 |
| 03 method, algorithm | 11.9 / 10.5 | 13.9 / 11.1 | 8.3 / 8.9 | 17.5 / 16.3 |
| 04 query object | 12.2 / 15.0 | 11.0 / 11.5 | 8.3 / 8.3 | 19.6 / 17.0 |

Looking at the results, we can see that Codex by default writes succinct statements. “Simple Technical English” does not make it succinct; in some runs, it even made the statements longer than the control.

By default, Codex is a better fit to explain code than Claude because it is shorter and simpler.

And this makes the entire experiment more about Claude than Codex, as Codex is already brief by default.

You can see here a comparison of the control run:

What about understanding and facts

Style and vocabulary are just one side of using LLMs to understand codebases. The second part is making sure the explanation still tells me what I should know or pay attention to.

Thus, I went through each of the four source files and wrote six facts about each. Things like: “there is no lock, so two concurrent first requests both execute” or “created_at is stored but never read”, “this until loop would spin forever if the invariant from count_matches ever broke”.

For experiments 2, 3, and 4, I picked those facts from the source before reading a single output. For experiment 1, I had already read the control run, so that row is weaker evidence than the other three.

Then I checked which one of those facts show up in which output and gave it a score.

Run 1:

Run 2:

Or to see this in some other way:

Here I calculated averages and medians of missed facts:

A couple of conclusions from this:

1. Seems like while Codex’s default output is much better than Claude’s from the perspective of foggy words, it also misses some facts by default. So being brief does not mean deeper understanding
2. For Claude: the ASD-STE100 explanation, while being short and clean, they do miss some facts. The same seems to happen for Codex too, so from the perspective of code explanations, the standard does not improve the accuracy of understanding facts about the code.
3. For Claude: The “Simple Technical English” has an average of 0.50 and a median of 0.0 with even a case where it got an extra fact that the control did not provide.
4. For Codex: The “Simple Technical English” performed the same or even worse than “ASD-STE100,” and in the end I would not suggest using any of them but rely on the default (no style specification)

To summarize: For Claude, “Simple Technical English” (shortened to STE in the tables I shared) performs better than “ASD-STE100” for explanations that include code-based facts. While for Codex both “Simple Technical English” and “ASD-STE100” perform worse, and by default Codex responds with fewer facts about code than Claude, but the answer is short.

Two assumptions I had that were proven wrong

I thought that the documented algorithm (ASD-STE100) would perform better because it has explicit instructions to follow. But it seems the more blurry instruction “Simple Technical English” performed better.

I also believed that when I asked for “Simple Technical English,” it would also remain brief, but this was not the case in all runs.

Conclusion

I have added this to the Claude user memory:

When explaining code or systems to me, use Simple Technical English. Short sentences. One idea per sentence. Use the domain words that already exist in the code instead of inventing abstractions for them.

I am keeping “Simple Technical English” because the experiment showed it performs well when removing “foggy words”.

I added “use domain words that already exist” to instruct it to reuse those concepts we use elsewhere in the codebase. This also could be an experiment on its own when I have time.

I am putting this in the user memory cause I want it for all my interactions with Claude.

I am not making any changes to Codex, as it seems to be already doing so from a “foggy” or “brief” perspective. Do they have a system prompt in the harness about this? I did not have time to dig deeper.

Resources

• ASD-STE100 Simplified Technical English, official site
• ASD-STE100 Issue 9 specification, PDF
• ASD Europe, Simplified Technical English
• Simplified Technical English on Wikipedia
• TechScribe, ASD Simplified Technical English
• Boeing Simplified English Checker
• ASD-STE100 rules repackaged as a Claude Code skill
• Stripe API, idempotent requests

All 60 raw outputs, the exact prompts used for every run, and the measurement scripts are in the experiments/ folder next to this article. What the independent Codex arm does and does not control is written up in RUN-NATIVE-PROVENANCE.md.