© 2026

· SPEECH

The Voice Came Loose. The state of speech AI: cloning, codecs and super-resolution

Speech is now editable, and a voice is a parameter, not a person.

01

Three seconds

In January 2023 Microsoft’s VALL-E cloned a voice from a three-second sample. It treated speech like text: tokens from a neural audio codec, predicted one after another. Codecs like SoundStream and EnCodec made that possible, and the framing took over the field.

By 2026 zero-shot cloning is a product feature: a few seconds of audio carries timbre, accent and rhythm, often across languages.

02

Beyond tokens

Newer systems drop discrete tokens. VoxCPM2, released this year, works in a continuous latent space. It encodes at 16 kHz and decodes at 48 kHz, so super-resolution comes free. It was trained on over two million hours in 30 languages, with open weights.

03

Editing, not recording

The bigger shift is control. Research codecs try to separate timbre from prosody: keep the speaker, change the delivery. SpeechX folds noise removal, speaker extraction and word-level editing into one model.

Each edit is a fresh generation, not a lossless one, but the ear accepts it as continuous. Audio is now as editable as text. Fix a fluffed line after the fact and nobody hears the seam.

04

Real time

Native speech-to-speech models like Kyutai’s Moshi answer in around 200 milliseconds and listen while they talk. Most products still chain recognition, a language model and synthesis, trading speed for control. That is now a design choice, not a research wall.

05

Aura

In 1935 Walter Benjamin wrote that mechanical reproduction strips a work of art of its aura, its presence in one time and place. Recording did that to the voice. Generation finishes the job. A voice no longer implies a throat.

I find that unsettling and beautiful. The voice is an instrument anyone can play, and who is speaking becomes a question of context, not sound.

KEEP READING

  1. 01The World Is Not Logged. Why real-world data, not intelligence, limits AIThe world records everything, in light, heat and wear. Almost none of it reaches a model. That gap, not intelligence, is what limits AI.Deep Tech
  2. 02System 1 Is Solved. AI agents now fail at spatial reasoning, long horizons and contextFast, perceptual work is done: agents read screens, click and type as well as people. What breaks now is the slow work of holding a space, a long task and a lot of context in mind.Deep Tech
  3. 03Reverse Benchmarking. Measure AI on what the leaders neglect, not what they optimiseBenchmarks make every lab chase the same scores. The useful question runs the other way: where does the best model still disappoint?Deep Tech
  4. 04Leveling Up Intelligence. A game analogy for RL environments, agents, benchmarks and evals“RL environment” is a buzzword thrown around as if it were a new machine. The loop it names is simple. What is hard is deciding which problem the score is supposed to stand for.Deep Tech
  5. 05You Cannot Hear a Fake. Why voice deepfake detection fails, and what replaces itDeepfake detection is an arms race the detector loses. Stop asking “is this fake?” and ask “can this prove it is real?”Speech
  6. 06Deconstruct to Be Reborn. Taking the self apart to find what is load-bearingYou were born one way. You can be born again, if you let yourself be taken apart first.Philosophy
  7. 07Comparison Is the Thief of Joy. But it is also the one-sided window of potential, and only you can act on itComparison steals joy when you read it as a verdict. Read it as a window and it shows you what is possible. Everyone builds an ego out of what they compare themselves to. Some build it proud and some build it to please, and both end up serving the crowd. So measure yourself against something perfect.Philosophy
  8. 08Problem Relativity. The size of a problem depends on where you standA problem has no absolute size, only a size measured from where you stand.Philosophy
  9. 09The Issue Is the Primitive. Running companies of humans and AI agents on Linear’s modelThe best operating system for a company of humans and agents already exists: the issue tracker.Systems
  10. 10The Curse of Knowledge. Why experts explain badly, and prompts fail the same wayOnce you know something, you can’t imagine not knowing it. That ruins how you teach, write and prompt.Mind
  11. 11The Story Is Not the Wall. What high agency is, and why AI makes it matter moreHigh agency is treating the limits other people describe as a story you may edit.Agency
  12. 12The Workplace AI Paradox. Why AI amplifies friction in crowded companiesAI amplifies whatever it touches. In a small, strong team it amplifies output. In a crowded one it amplifies friction.Work