Since consciousness had first dawned, in that laboratory so many million for miles sunward, all Hal’s powers and skills had been directed towards one end. The fulfillment of his assigned programme was more than an obsession; it was the only reason for his existence. Undistracted by the lusts and passions of organic life, he had pursued that goal with absolute single-mindedness of purpose.

- 2001: A Space Odyssey, Arthur C. Clarke

Computers are fascinating things. It is said, we conjure the spirit of computers with our spells!

How do these spirits think? Is it like a human brain?

One of my first working software was a neural network, $3$ days to write on paper and code in numpy. It was trained on the Titanic dataset. Going from regression (n$00$b) to NN gave $~30\%$ improvement and when I added some eval data, it became even better (n$00$b). But it was very cool! A .npz file with numbers that could do a better “if-else” than I could code and I could put it in a pendrive and carry it around. I thought these weights would be like gold and NN would take over the world (not n$00$b).

$10$ years later that prediction was true. LLMs are a type of NN and they are already insanely good. AGI is finally here (sarcasm) and it has changed culture forever. But they haven’t replaced the if-else code we write, rather NN provides a very rich perception and augmentation to that decision making.

I recently wanted to do research in AI.

Just like everyone-fucking-else.

But I knew it was not LLMs, Agentic, AGI but teaching NNs to behave like computers.

So I packed up my bag, decided to leave Hype city. Meet strange computers and find the Grand Old Automata. Learn how to build a new computer. I will travel away, finding the roots of how we got here. Study things in jungly fringes of the field. Understand the neural alchemy from shiny beings. I have left these essays as a map for future travellers walking on the path of automata.

• #$1$ - Automata: This covers the Chomsky’s Hierarchy and machines required to solve them.
• #$2$ - IRL Automata: The four types of Register Machines.
• #$3$ - ???
• #$4$ - ???
• #$5$ - ???
• #$6$ - ???

All are independent essays can be read in any order.

What do we do when we meet the Grand Old Automata? Will we discover a new type of comptuer? Or will we return empty handed? Always remember what Ilya, the Monk once said:

It turns out I briefly touched this subject $6$ years ago in the early part of $2020$. Aah the covid days.

• Teaching Machines Rules and Algorithms
• Reading list on Neural Graph Execution