Skip to content
Bundu Foundation

Research

··3 min read

Exact or guess: compiler fixes that never change what a program means

Mzizi's compiler tags every suggested fix as exact or guess, and applies exact fixes automatically. The rule behind the tag, and the review that caught us breaking it.

By Bryan Fawcett

An agent writing code spends most of its time in a loop: edit, compile, read the errors, edit again. The compiler's errors are the only teacher in that loop. Mzizi's compiler is designed around it.

One line of JSON per real error

mz check --agent prints the whole program's diagnostics as newline-delimited JSON:

  • Deterministic order, so the same program always produces the same output.
  • One diagnostic per real error. Blocks close with end <kind> <name> and newlines are tokens, so the parser can recover line by line instead of cascading.
  • A fix where the compiler knows one, tagged exact or guess.

mz fix applies every exact fix in one pass, and a second run must change nothing. That is tested.

The rule behind the tag

The whole design rests on one rule: an exact fix must never change what a program means. If applying it could make the program do something different, it is a guess. A guess is shown to the agent, which decides; it is never applied on its own.

The rule matters most for the mistakes an agent is likely to make: writing another language's spelling. Mzizi recognises many of them, with diagnostic MZ0962, and says what the Mzizi form is.

  • len(s) becomes s.length(): exact. Same meaning.
  • "h" in s becomes s.contains("h"): exact.
  • s.indexOf(t) becomes s.find(t): guess, because JavaScript returns -1 when the text is absent, and Mzizi returns an option.
  • parseInt(s) becomes s.parse_int(): guess, because the two accept different inputs.

The review that caught us

During the text-methods work, an independent review found that we had broken the rule. Someone coming from Rust writes s.len(). We offered s.length() as an exact fix. But Rust's len() counts bytes, and Mzizi's length() counts Unicode scalar values: "é" is two bytes and one character. JavaScript's s.length was the same mistake in another form, since it counts UTF-16 units.

Those fixes are now guesses, along with Python's strip(), which removes more characters than Mzizi's trim(). The agent still sees the suggestion. Nothing changes the meaning of its code without its decision.

It is a small fix, and that is the point. A compiler that edits code automatically has to be trustworthy in exactly these corners, and the corners are where other languages disagree.

Why this is research, not just engineering

The charter bets that dense, accurate diagnostics make an agent's repair loop converge faster. Our September pilots found the opposite for a ~7B model: it read the diagnostics and resubmitted the same file. So the open questions are measurable ones. Does a diagnostic that names the replacement help a small model more than one that names the rule? Do exact fixes, applied automatically, close the gap? The kill-criterion run is designed to answer questions like these, and it has not run yet.

Back to newsroom