https://huggingface.co/blog/AbstractPhil/geovocab-chunking
This principle has been cooking for quite a while. There's enough experimentation to show that it's useful, and the more experiments that run show it's well beyond useful. It's a legitimate and repeatable geometric scaling principle that can be directly utilized to teach smaller models to usefully exhibit the behavior of considerably larger models without requiring massive distillation runs.
This exact behavior can be utilized to teach student models to accept direct learned behavior from expert teachers while still retaining the teachers in multimodal differential utilization; which includes image models learning from text to make videos, video models learning from music to construct text, and so on.
These geometric primitives directly align to the survivability of the big number mathematics that CAN exist in conjunction with what is being directly utilized, guaranteeing the very behavior that cannot exist is differentiated into a larger spectrum of differentiated utilization.
This geometric vocabulary does not attempt to bypass what is known, but instead constructs utilizable behavioral constructive constraints based on what can exist, providing dense and robust capacity to those variants while existing within invariant space that can expand or collapse as necessary to the need.