Robot Self Awareness

1 · Brief

How do we understand what the self is? As humans, we learn to walk through practice, but if you drop any of us into a new environment, we'll still figure out how to move. Yes, we might fall, yet over time we train ourselves to travel at a decent clip (relative to what, exactly, is the interesting question).

Now flip the situation: place a perfectly functioning human body in that same unfamiliar world but strip away any sense of what the body is. Progress would be significantly slower.

That intuitive grasp of our own body is what we call self-awareness—consider it as tackling the old Descartes puzzle—“How does the mind know the body is its own?”—but from the lens of proprioception.

What we’ve done is help a simple robot ant develop that same sense—first in simulation, then in real life.

2 · Why It Matters

Learning machines shouldn't need a bespoke controller for every new task—they should adapt on the fly. A robot that actually "knows" its body could pivot from walking to climbing (or from Mars dust to factory floor) with far less retraining. Besides, building a self-aware robot is just plain cool—and a crucial step toward more general, trustworthy AI.

3 · How We're Tackling It (High-Level)

Meta-learning inspiration – Approaches like MAML (Model-Agnostic Meta-Learning) show how to jump-start learning across tasks by fine-tuning a single base model. We take a different angle: instead of hard-coding a separation, we try to encourage the policy to naturally separate reusable body knowledge from task-specific behavior.

Train one ant on a variety of behaviors – Rather than building a different controller for each skill, we train a single policy across multiple distinct behaviors. The goal is to pressure the network to reuse whatever “body understanding” it has already built, even as the task changes.

Representation probes (the “X-ray” of the policy) – Reward curves tell you if the agent is improving. I’m also looking inside the network to see what it’s encoding. By comparing hidden-layer activity across behaviors, I measure which internal components stay consistent (candidate “self”) versus which reorganize with the objective (candidate “skill layer”). In other words: digging into the black box and trying to make the policy legible.

4 · Early Signals & Analysis

A few patterns have shown up repeatedly:

Separation works when it’s forced. With explicit constraints, we can reliably split out structure that behaves like “body knowledge,” confirmed through neuron matching and low-dimensional probes.

But the more exciting result is selective stability without forcing it. Even without explicit separation constraints, multi-behavior policies show a stable internal core alongside more flexible task-specific parts.

Behavioral evidence lines up with internal evidence. Multi-task models tend to bounce back faster when revisiting a behavior or encountering shifted conditions (like new terrains), and qualitatively the emerging gaits look more like they’re reusing prior body understanding rather than relearning from scratch.

Inside the network, structure persists. Correlated neuron clusters remain recognizable across behaviors—consistent with an internal body representation—and my analysis dashboards track how that “stable core” grows, persists, or gets overwritten across checkpoints.

Overall: we’re seeing signs of a controller that carries forward reusable body knowledge while swapping out the parts that need to change for the current task.

5 · Under the Hood

I’ve built a broad stack around this project: RL design (phase-switch training, per-terrain replay), experiment ops (reproducible launches, checkpoint/resume, GPU scheduling), and extensive analysis (TensorBoard reads, quick rollout videos, and most importantly representation & model-comparison tools / mechanistic interpretability to see what the policy learned and how it differs across runs).

A big accelerator has been running in Isaac: the combination of a strong physics stack and vectorized environments (thousands of parallel simulations) has made results both more reproducible and consistently higher quality, while dramatically shortening iteration time.

On the engineering side it’s Python/PyTorch + Isaac, YAML configs, Bash/tmux, lightweight tests, Git hygiene, and clear docs (Markdown/LaTeX, runbooks, postmortems).

My desk is set up for fast loops: code in the middle, server control and git on the right, AI + prompt/coding on the left, a rotating doc screen for specs/PRs, and a "focus" screen with a concert/EDM set.

6 · Acknowledgements

Research conducted in the Creative Machines Lab, Columbia University under Professor Hod Lipson.

Questions or ideas? aj3337@columbia.edu