I’m actually building the system-level approach this memo hints at.
I’m not from a lab or an academic group, but I’ve been working on a post-transformer inference method where you extract a low-rank “meaning field” from a frozen Llama-70B layer and train a small student model to generate those fields directly.
The idea is similar to what this memo describes, but with an empirical implementation.
It isn’t about bigger models.
It’s about reorganizing the system around meaning and structure, then treating the transformer as a teacher rather than the final destination.
I’d genuinely appreciate critique or replication attempts from people here. HN tends to give the most honest feedback.
The commercial labs aren't building what they say they are.
AGI is invoked like a destination but kept conveniently vague. Meanwhile, the cognitive partner people actually imagine when they say "AI" requires architecture the labs aren't really pursuing.
I’m not from a lab or an academic group, but I’ve been working on a post-transformer inference method where you extract a low-rank “meaning field” from a frozen Llama-70B layer and train a small student model to generate those fields directly.
The idea is similar to what this memo describes, but with an empirical implementation.
I just open-sourced the reference version here:
GitHub: https://github.com/Anima-Core/an1-core Paper + DOI: https://zenodo.org/records/17873275
It isn’t about bigger models. It’s about reorganizing the system around meaning and structure, then treating the transformer as a teacher rather than the final destination.
I’d genuinely appreciate critique or replication attempts from people here. HN tends to give the most honest feedback.
AGI is invoked like a destination but kept conveniently vague. Meanwhile, the cognitive partner people actually imagine when they say "AI" requires architecture the labs aren't really pursuing.