My calculator is a transformer

Thu, 30 Apr 2026 00:00:00 +0000

TLDR: Article describes a method to define “registers”, “heads” and non-linear “compute” functions which are compiled and distilled into a standard Transformer model. This is used to develop a simple and deterministic RPN interpreter.

Code accompanying this project can be found here.

AI disclaimer: code ≈ 50%, non-agentic. Article writing = 0%.

A few weeks ago I read a blog post from Percepta.ai entitled, Can LLMs be computers?¹. The article frames this as an alternative to tool calling. Instead of writing out a program and using an external tool to evaluate it, just build an exact, deterministic interpreter right into the Transformer network. This framing led to some criticism regarding the approach, particularly “what is the point?” and “where are the benchmarks?” These are fair questions, but I think it unfortunately detracted from what I thought was a rather fascinating topic, which is: how do you build a deterministic interpreter into a Transformer?

Revsiting the Decision Transformer with Tic Tac Toe

Tue, 25 Nov 2025 00:00:00 +0000

Note: code accompanying this project can be found here.

This weekend, as an afterthought, I guess due to some background threads that had been accumulating in my head for a while, I decided to finally take a quick & dirty shot at one of those things that’s fascinated me, but not quite enough to actually get around to trying it until quite some time has passed: the Decision Transformer¹ (DT). I remember reading about this when it came out and just being a bit.. I dunno, simultaneously amazed and annoyed. I had recently finished a round of studying reinforcement learning again, and at the time it seemed like people were just throwing transformers at everything to see what sticks. At the same time, the idea seemed to just make sense: what if we just.. condition a sequence prediction task on how well we want it to perform? But it also completely subverted everything I understood about RL and the importance of Bellman value estimation. Anyways, I left it, just curious to see what would follow it up, and I guess I didn’t hear much more, although there was certainly some related work being done.

Blog on Stephen Sinclair

My calculator is a transformer

Revsiting the Decision Transformer with Tic Tac Toe