<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>mattdf</title>
    <link>https://mattdf.xyz/</link>
    <description>Physics and Machine Learning | &lt;a href=&#34;https://mattdf.com/&#34;&gt;Matthew Di Ferrante&lt;/a&gt;</description>
    <pubDate>Tue, 21 Apr 2026 10:44:16 +0000</pubDate>
    <image>
      <url>https://i.snap.as/loZmY8b3.png</url>
      <title>mattdf</title>
      <link>https://mattdf.xyz/</link>
    </image>
    <item>
      <title>The Scaling Laws are already broken: smaller models win out on reasoning long term</title>
      <link>https://mattdf.xyz/the-scaling-laws-are-already-broken-smaller-models-win-out-on-reasoning-long?pk_campaign=rss-feed</link>
      <description>&lt;![CDATA[The SoTA LLMs that score highest on standard benchmarks all have   100B parameter counts, but these consist of mainly “flat” tasks: single-prompt problems with short, self-contained answers. The current scaling curves that plot test loss vs. parameter count show smooth power-law gains and suggest that more weights yield monotonic progress. However, these curves are misleading: they measure token-level accuracy, not whole-task reliability across longer, chained sequences of actions. &#xA;&#xA;Once models need to maintain that correctness through hundreds or thousands of dependent steps (writing, compiling, running, reading, revising, etc), they break down. Below is my argument for why parameter growth or increased test-time compute alone cannot overcome that shift, and why smaller, modular, hierarchy-aware systems will ultimately likely dominate.&#xA;&#xA;!--more--&#xA;&#xA;Concept breadth versus chain length&#xA;&#xA;Define  &#xA;&#xA;\\(P\\) — active parameters  &#xA;\\(\varepsilon(P)\sim P^{-\alpha}\\) — average per-token error, with \\(\\alpha\approx0.27 - 0.34\\) in current scaling law fits&#xA;\\(L\\) — steps in the task chain  &#xA;\\(S\\) — probability the entire chain succeeds  &#xA;&#xA;br/&#xA;With an autoregressive decoder, token errors accumulate multiplicatively:  &#xA;&#xA;$$&#xA;S=(1-\varepsilon)^{L}\approx e^{-\varepsilon L}.&#xA;$$&#xA;&#xA;Increasing \\(P\\) reduces \\(\varepsilon\\) only polynomially; doubling the horizon multiplies required parameters by \\(2^{1/\alpha}\\).&#xA;&#xA;Because \\(\varepsilon\\) falls only polynomially while the chain penalty is exponential, success collapses once \\(L\gg P^{\alpha}\\). Long-horizon coding loops (\\(L\\!\sim\\!10^{3\\!-\\!4}\\)) cross this frontier even for 100B-parameter models.&#xA;&#xA;Our brains avoid the collapse by chunking: macro-actions compress thousands of micro moves into tens of high-level decisions (write a function, run tests, skim a diff), so effective chain length is logarithmic in problem size (\\(L{\text{eff}}\\!\ll\\!L\\)), and keeps \\(S\\) reasonable.&#xA;&#xA;Information-theoretic capacity bend&#xA;&#xA;Dense networks reuse the same weights for every concept. Interference noise between gradient updates grows like \\(\sqrt{C/P}\\) for \\(C\\) stored concepts. &#xA;&#xA;Holding accuracy fixed therefore demands:&#xA;&#xA;\\[&#xA;\text{P}\gtrsim\frac{C}{\varepsilon^{2}},&#xA;\\]&#xA;&#xA;So after a certain breadth the marginal parameters per additional concept rise super-linearly. Empirically we are still on the pre-bend “Chinchilla” slope, but real world benchmarks that have longer sequences (coding agents, multi-hop tool use, etc) are already beginning to hint at saturation.&#xA;&#xA;Salience collapse: capacity-driven memorization versus reasoning&#xA;&#xA;Large dense models can drive their training loss down by allocating surplus parameters to rote lookup rather than deeper abstraction. The extra capacity expands the surface area of memorised fragments faster than it expands the manifold of compositional reasoning.&#xA;&#xA;Let  &#xA;&#xA;\\(P\\) — parameter count  &#xA;\\(dg(P)\propto P^{\beta}\\) — degrees of freedom dedicated to generalisable abstractions  &#xA;\\(dm(P)\propto P^{\gamma}\\) — degrees of freedom tied up in thin, input-specific memorisation with \\(\gamma   \beta\\)&#xA;&#xA;br/&#xA;For a task requiring effective reasoning load \\(D\\), success needs:&#xA;&#xA;\\[&#xA;dg(P)\gtrsim D&#xA;\\]&#xA;&#xA; Yet gradient noise from memorization grows with \\(dm(P)\\), so the signal-to-noise ratio degrades as \\(P^{\beta-\gamma}\\). Beyond a crossover \\(P^{\star}\\) where \\(\beta=\gamma\\) further scale reduces reasoning fidelity even as perplexity improves.&#xA;&#xA;Policy–gradient alignment (RLHF, DPO, “policy refinement”) attempts to re-inject salience by adding an auxiliary loss \\(L{\text{policy}}\\) that penalizes obviously spurious continuations. Each new domain or safety constraint requires a fresh reward model and multiple PPO passes, so alignment compute grows roughly&#xA;&#xA;\\[&#xA;\text{FLOPs}{\text{align}}\;\approx\;k\,P\,T,&#xA;\\]&#xA;&#xA;with \\(T\\) human preference samples and \\(k\\!  \\!1\\) PPO epochs. Alignment absorbs a larger share of the total training budget as \\(P\\) rises, pushing the economic wall forward only marginally.&#xA;&#xA;In summary, larger models over-memorise, then we must pay extra compute to prune the memorization down to human-relevant abstractions. Small or sparsely activated models can skip this step entirely.&#xA;&#xA;Online / continual learning worsens this wall&#xA;&#xA;Replay-bound economics&#xA;br/&#xA;&#xA;To avoid catastrophic forgetting when new data arrives, dense models must replay old samples or regularise all weights:&#xA;&#xA;\\[&#xA;\text{marginal FLOPs} \;\approx\; P\;(1+R),&#xA;\\]&#xA;&#xA;where \\(R\\) is the replay ratio. For \\(P\\!\sim\\!10^{11}\\) even \\(R\\!=\\!0.1\\) becomes financially prohibitive eventually.&#xA;A 1B-parameter model can be kept current daily; a 1T-parameter one cannot.&#xA;&#xA;Drift scaling  &#xA;&#xA;Let \\(S=\|\Delta\theta\|/\|\theta\|\\) be relative weight drift per task. &#xA;&#xA;With sparse or adapter-based updates,&#xA;&#xA;\\[&#xA;P{\text{active}}\\!\ll\\!P&#xA;\\] &#xA;&#xA;so &#xA;&#xA;\\[&#xA;S\propto \sqrt{C/P{\text{active}}}&#xA;\\]&#xA;&#xA;keeping interference controllable. Dense updates keep \\(P{\text{active}}\\!=\\!P\\) and hit the \\(C/\varepsilon^{2}\\) wall quickly.&#xA;&#xA;Test-time compute is a brittle workaround&#xA;&#xA;“Chain-of-thought’’ (CoT) prompting, self-consistency sampling, and tree-of-thought search attempt to average out the exponential failure \\(S=(1-\varepsilon)^{L}\\) by generating many reasoning traces per query and selecting the first or majority-voted success.&#xA;&#xA;Let  &#xA;&#xA;\\(K\\) — sampled chains at inference time  &#xA;\\(C{\text{tok}}\\) — tokens per chain (≈ \\(L\\))  &#xA;\\(S\\) — single-chain success (\\(S\\!\ll\\!1\\))&#xA;&#xA;br/&#xA;The probability that at least one chain succeeds is  &#xA;&#xA;\\[&#xA;P{\text{any}} \;=\; 1-(1-S)^{K}\;\approx\;1-e^{-KS}.&#xA;\\]&#xA;&#xA;To reach \\(P{\text{any}}\ge 0.63\\) one needs &#xA;&#xA;\\[&#xA;K\gtrsim 1/S&#xA;\\]&#xA;&#xA;Given \\(S\approx e^{-\varepsilon L}\\), this implies&#xA;&#xA;\\[&#xA;K \;\approx\; e^{\,\varepsilon L}.&#xA;\\]&#xA;&#xA;Inference FLOPs therefore scale as  &#xA;&#xA;\\[&#xA;\text{FLOPs} = KC{\text{tok}} \propto Le^{\varepsilon L}&#xA;\\]&#xA;&#xA;exponential in the very horizon the method tries to tame. At long \\(L\\) the compute wall arrives almost as rapidly as for parameter scaling.&#xA;&#xA;Moreover, CoT inherits the same salience gaps: the model still evaluates each token in isolation, so spurious low-level correlations leak into the trace. Voting or heuristic scoring only filters outcomes; it does not raise the underlying \\(S\\).&#xA;&#xA;In essence, test-time brute force buys reliability by paying an exponential tax that sooner or later dwarfs both training and deployment budgets, so it&#39;s another band-aid over the \\(e^{-\varepsilon L}\\) issue rather than a fundamental solution.&#xA;&#xA;Latent scratchpad suffers from the same issue&#xA;&#xA;The class of “latent-scratchpad” systems, like OpenAI’s o-series, DeepSeek-R1, etc, does the speculative exploration during training, then distills the surviving path into a single forward pass for test-time. We get two extra cost terms from this approach.&#xA;&#xA;Hidden-loop factor  &#xA;During inference each output token is no longer a single transformer pass; it is the result of an unrolled micro-loop of depth \\(r\\) embedded in the block (e.g. iterative attention updates, look-ahead planners, or a value-head scoring step).  &#xA;&#xA;\\[&#xA;\text{FLOPs} \;=\; r\,C{\text{tok}},\qquad r  1 .&#xA;\\]&#xA;&#xA;If \\(r\\) is fixed, runtime grows only linearly.  In practice, models raise \\(r\\) with prompt length to maintain context-wide coherence (DeepSeek’s planner re-invokes itself each tool call), so \\(r=r0+\kappa L\\) and the cost becomes \\(O(L^2)\\).&#xA;&#xA;Trace-distillation cost  &#xA;Let \\(K{\text{train}}\\) be the number of sampled reasoning traces stored for imitation.  Distillation loss reduces token error as  &#xA;&#xA;\\[&#xA;\varepsilon\{\text{distilled}} \approx \varepsilon{\text{base}}/\sqrt{K\{\text{train}}}&#xA;\\]&#xA;&#xA;but the gradient noise from contradicting traces adds a variance term \\(\sigma^2\propto K\{\text{train}}\\).  Required optimisation steps scale like \\(\sigma^2/\varepsilon^2\sim K\{\text{train}}^2\\).  Hence:&#xA;&#xA;\\[&#xA;\text{FLOPs} \;\approx\; K\{\text{train}}^2\,M\,C\{\text{tok}} .&#xA;\\]&#xA;&#xA;The approach trades a one-shot exponential inference wall for a quadratic training wall plus a super-linear runtime factor.  For long-horizon tasks (\\(L\\!\sim\\!10^3\\)) even moderate \\(K{\text{train}}\\) (10–30) rivals the raw cost of running many CoT samples at test time.&#xA;&#xA;Residual fragility  &#xA;The distilled single path is still a chain of length \\(L\\); only the selection occurred offline. Any unmodelled context shift resurrects the original error law &#xA;\\(S=(1-\varepsilon)^{L}\\). The hidden loop cannot retroactively explore alternatives, so test-time flexibility is gone.&#xA;&#xA;Net effect  &#xA;Latent-scratchpad models compress exploration into training but pay:&#xA;&#xA;\\(O(K{\text{train}}^2)\\) extra pre-compute,&#xA;\\(O(rL)\)–\(O(L^2)\\) test-compute,&#xA;unchanged exponential sensitivity to horizon drift.&#xA;&#xA;br/&#xA;They remain a band-aid for the \\(e^{-\varepsilon L}\\) failure mode.&#xA;&#xA;Empirical fault line: long-range coding tasks&#xA;&#xA;HumanEval, GSM8K, MMLU (the benchmarks that motivated current scaling laws) fit in ≤ 15 reasoning steps. They primarily measure local reasoning, and say nothing about the model&#39;s capacity for long range credit assignment. &#xA;&#xA;Ask the same models to:&#xA;&#xA;iteratively fix a 10,000+ line codebase,  &#xA;pass a compiler-run-debug loop,  &#xA;coordinate across multiple tools and environments&#xA;&#xA;br/&#xA;and error chains will quickly explode.&#xA;&#xA;The perplexity gains observed in the flatter context disappear or become meaningless, implying that the unit of action, not the language loss, is the limiting factor.&#xA;&#xA;How smaller/sparser models can cope better&#xA;&#xA;The recipe that emerges from the above analysis is to shift optimization effort away from brute-forced token-level accuracy and toward explicit structuring of computation. We can achieve this through two orthogonal efforts:&#xA;&#xA;Horizon compression – collapse long micro-token chains into a handful of macro actions that can be supervised or rolled back individually.  &#xA;Selective activation – ensure that only a small, context-relevant slice of the parameter matrix participates in each forward/backward pass so new knowledge only perturbs a bounded region.&#xA;&#xA;br/br/&#xA;&#xA;These techniques convert the exponential failure surface \\(e^{-\varepsilon L}\\) into a tractable polynomial and bound the \\(C/\varepsilon^{2}\\) interference bleed. They do so through parameter reuse rather than parameter accumulation, keeping training and inference budgets within a single-GPU or single-node envelope.&#xA;&#xA;Reliability ↔ Continual-Learning Trade-offs of Modular Techniques&#xA;&#xA;|Design move | Chain reliability | Continual-learning cost|&#xA;|---|---|---|&#xA;Hierarchical planner + executor | Cuts \\(L_{\text{eff}}\\)​ by 10–100× | Only planner/executor slice retrained&#xA;Verifier-in-the-loop | Detects &amp; rolls back errors every macro step | No weight update needed&#xA;Sparse MoE routing | Active params per token ≤ 10-50B | Replay on experts only&#xA;Retrieval memory | Offloads rare concepts, reduces interference | New info logged, not re-trained&#xA;LoRA / adapters | Drift confined to hand-sized matrices | Update cost ∝ adapter size&#xA;&#xA;Each technique lowers either \\(L\\) or interference, breaking the exponential failure mode without enlarging \\(P\\).&#xA;&#xA;Conclusion&#xA;&#xA;The mainstream scaling narrative extrapolates single-step accuracy and misses the exponential fragility of long chains. Past a short term horizon, dense parameter growth has severely diminishing returns - essentially it hits a wall: every new concept conflicts with all previous ones, while each additional reasoning step multiplies failure odds.&#xA;&#xA;Hierarchical control, external verification, sparse activation and retrieval shift the curve: they compress the effective chain length and cut interference without bloating the active model. The systems that leverage small, modular, tool-using principles will likely achieve human-level macro accuracy first, even as gigantic models keep hitting the \\(e^{-\varepsilon L}\\) limit.&#xA;&#xA;-- a href=&#34;https://x.com/matthewdif&#34;Matthew Di Ferrante/a&#xA;&#xA;br /&#xA;&#xA;!--emailsub--&#xA;]]&gt;</description>
      <content:encoded><![CDATA[<p>The SoTA LLMs that score highest on standard benchmarks all have &gt;100B parameter counts, but these consist of mainly “flat” tasks: single-prompt problems with short, self-contained answers. The current scaling curves that plot test loss vs. parameter count show smooth power-law gains and suggest that more weights yield monotonic progress. However, these curves are misleading: they measure token-level accuracy, not whole-task reliability across longer, chained sequences of actions.</p>

<p>Once models need to maintain that correctness through hundreds or thousands of dependent steps (writing, compiling, running, reading, revising, etc), they break down. Below is my argument for why parameter growth or increased test-time compute alone cannot overcome that shift, and why smaller, modular, hierarchy-aware systems will ultimately likely dominate.</p>



<h2 id="concept-breadth-versus-chain-length" id="concept-breadth-versus-chain-length">Concept breadth versus chain length</h2>

<p>Define</p>
<ul><li>\(P\) — active parameters<br/></li>
<li>\(\varepsilon(P)\sim P^{-\alpha}\) — average per-token error, with \(\alpha\approx0.27 – 0.34\) in current scaling law fits</li>
<li>\(L\) — steps in the task chain<br/></li>
<li>\(S\) — probability the entire chain succeeds<br/></li></ul>

<p><br/>
With an autoregressive decoder, token errors accumulate multiplicatively:</p>

<p>$$
S=(1-\varepsilon)^{L}\approx e^{-\varepsilon L}.
$$</p>

<p>Increasing \(P\) reduces \(\varepsilon\) only polynomially; doubling the horizon multiplies required parameters by \(2^{1/\alpha}\).</p>

<p>Because \(\varepsilon\) falls only polynomially while the chain penalty is exponential, success collapses once \(L\gg P^{\alpha}\). Long-horizon coding loops (\(L\!\sim\!10^{3\!-\!4}\)) cross this frontier even for 100B-parameter models.</p>

<p>Our brains avoid the collapse by <em>chunking</em>: macro-actions compress thousands of micro moves into tens of high-level decisions (write a function, run tests, skim a diff), so effective chain length is logarithmic in problem size (\(L_{\text{eff}}\!\ll\!L\)), and keeps \(S\) reasonable.</p>

<h3 id="information-theoretic-capacity-bend" id="information-theoretic-capacity-bend">Information-theoretic capacity bend</h3>

<p>Dense networks reuse the same weights for every concept. Interference noise between gradient updates grows like \(\sqrt{C/P}\) for \(C\) stored concepts.</p>

<p>Holding accuracy fixed therefore demands:</p>

<p>\[
\text{P}\gtrsim\frac{C}{\varepsilon^{2}},
\]</p>

<p>So after a certain breadth the marginal parameters per additional concept rise super-linearly. Empirically we are still on the pre-bend “Chinchilla” slope, but real world benchmarks that have longer sequences (coding agents, multi-hop tool use, etc) are already beginning to hint at saturation.</p>

<h2 id="salience-collapse-capacity-driven-memorization-versus-reasoning" id="salience-collapse-capacity-driven-memorization-versus-reasoning">Salience collapse: capacity-driven memorization versus reasoning</h2>

<p>Large dense models can drive their training loss down by allocating surplus parameters to rote <em>lookup</em> rather than deeper abstraction. The extra capacity expands the surface area of memorised fragments faster than it expands the manifold of compositional reasoning.</p>

<p>Let</p>
<ul><li>\(P\) — parameter count<br/></li>
<li>\(d_g(P)\propto P^{\beta}\) — degrees of freedom dedicated to generalisable abstractions<br/></li>
<li>\(d_m(P)\propto P^{\gamma}\) — degrees of freedom tied up in thin, input-specific memorisation with \(\gamma &gt; \beta\)</li></ul>

<p><br/>
For a task requiring effective reasoning load \(D\), success needs:</p>

<p>\[
d_g(P)\gtrsim D
\]</p>

<p> Yet gradient noise from memorization grows with \(d_m(P)\), so the signal-to-noise ratio degrades as \(P^{\beta-\gamma}\). Beyond a crossover \(P^{\star}\) where \(\beta=\gamma\) further scale <em>reduces</em> reasoning fidelity even as perplexity improves.</p>

<p>Policy–gradient alignment (RLHF, DPO, “policy refinement”) attempts to <em>re-inject</em> salience by adding an auxiliary loss \(L_{\text{policy}}\) that penalizes obviously spurious continuations. Each new domain or safety constraint requires a fresh reward model and multiple PPO passes, so alignment compute grows roughly</p>

<p>\[
\text{FLOPs}_{\text{align}}\;\approx\;k\,P\,T,
\]</p>

<p>with \(T\) human preference samples and \(k\!&gt;\!1\) PPO epochs. Alignment absorbs a larger share of the total training budget as \(P\) rises, pushing the economic wall forward only marginally.</p>

<p>In summary, larger models <em>over-memorise</em>, then we must pay extra compute to prune the memorization down to human-relevant abstractions. Small or sparsely activated models can skip this step entirely.</p>

<h2 id="online-continual-learning-worsens-this-wall" id="online-continual-learning-worsens-this-wall">Online / continual learning worsens this wall</h2>

<h3 id="replay-bound-economics" id="replay-bound-economics">Replay-bound economics</h3>

<p><br/></p>

<p>To avoid catastrophic forgetting when new data arrives, dense models must replay old samples or regularise all weights:</p>

<p>\[
\text{marginal FLOPs} \;\approx\; P\;(1+R),
\]</p>

<p>where \(R\) is the replay ratio. For \(P\!\sim\!10^{11}\) even \(R\!=\!0.1\) becomes financially prohibitive eventually.
A 1B-parameter model can be kept current daily; a 1T-parameter one cannot.</p>

<h3 id="drift-scaling" id="drift-scaling">Drift scaling</h3>

<p>Let \(S=|\Delta\theta|/|\theta|\) be relative weight drift per task.</p>

<p>With sparse or adapter-based updates,</p>

<p>\[
P_{\text{active}}\!\ll\!P
\]</p>

<p>so</p>

<p>\[
S\propto \sqrt{C/P_{\text{active}}}
\]</p>

<p>keeping interference controllable. Dense updates keep \(P_{\text{active}}\!=\!P\) and hit the \(C/\varepsilon^{2}\) wall quickly.</p>

<h2 id="test-time-compute-is-a-brittle-workaround" id="test-time-compute-is-a-brittle-workaround">Test-time compute is a brittle workaround</h2>

<p>“Chain-of-thought’’ (CoT) prompting, self-consistency sampling, and tree-of-thought search attempt to <em>average out</em> the exponential failure \(S=(1-\varepsilon)^{L}\) by generating many reasoning traces per query and selecting the first or majority-voted success.</p>

<p>Let</p>
<ul><li>\(K\) — sampled chains at inference time<br/></li>
<li>\(C_{\text{tok}}\) — tokens per chain (≈ \(L\))<br/></li>
<li>\(S\) — single-chain success (\(S\!\ll\!1\))</li></ul>

<p><br/>
The probability that <em>at least one</em> chain succeeds is</p>

<p>\[
P_{\text{any}} \;=\; 1-(1-S)^{K}\;\approx\;1-e^{-KS}.
\]</p>

<p>To reach \(P_{\text{any}}\ge 0.63\) one needs</p>

<p>\[
K\gtrsim 1/S
\]</p>

<p>Given \(S\approx e^{-\varepsilon L}\), this implies</p>

<p>\[
K \;\approx\; e^{\,\varepsilon L}.
\]</p>

<p><strong>Inference FLOPs</strong> therefore scale as</p>

<p>\[
\text{FLOPs} = KC_{\text{tok}} \propto Le^{\varepsilon L}
\]</p>

<p>exponential in the very horizon the method tries to tame. At long \(L\) the compute wall arrives almost as rapidly as for parameter scaling.</p>

<p>Moreover, CoT inherits the same salience gaps: the model still evaluates each token in isolation, so spurious low-level correlations leak into the trace. Voting or heuristic scoring only <em>filters</em> outcomes; it does not raise the underlying \(S\).</p>

<p>In essence, test-time brute force buys reliability by paying an exponential tax that sooner or later dwarfs both training and deployment budgets, so it&#39;s another band-aid over the \(e^{-\varepsilon L}\) issue rather than a fundamental solution.</p>

<h3 id="latent-scratchpad-suffers-from-the-same-issue" id="latent-scratchpad-suffers-from-the-same-issue">Latent scratchpad suffers from the same issue</h3>

<p>The class of “latent-scratchpad” systems, like OpenAI’s <em>o-series</em>, DeepSeek-R1, etc, does the speculative exploration during <em>training</em>, then <em>distills</em> the surviving path into a single forward pass for test-time. We get two extra cost terms from this approach.</p>

<h4 id="hidden-loop-factor" id="hidden-loop-factor">Hidden-loop factor</h4>

<p>During inference each output token is no longer a single transformer pass; it is the result of an <strong>unrolled micro-loop</strong> of depth \(r\) embedded in the block (e.g. iterative attention updates, look-ahead planners, or a value-head scoring step).</p>

<p>\[
\text{FLOPs} \;=\; r\,C_{\text{tok}},\qquad r&gt;1 .
\]</p>

<p>If \(r\) is fixed, runtime grows only linearly.  In practice, models raise \(r\) with prompt length to maintain context-wide coherence (DeepSeek’s planner re-invokes itself each tool call), so \(r=r_0+\kappa L\) and the cost becomes \(O(L^2)\).</p>

<h4 id="trace-distillation-cost" id="trace-distillation-cost">Trace-distillation cost</h4>

<p>Let \(K_{\text{train}}\) be the number of sampled reasoning traces stored for imitation.  Distillation loss reduces token error as</p>

<p>\[
\varepsilon_{\text{distilled}} \approx \varepsilon_{\text{base}}/\sqrt{K_{\text{train}}}
\]</p>

<p>but the <strong>gradient noise</strong> from contradicting traces adds a variance term \(\sigma^2\propto K_{\text{train}}\).  Required optimisation steps scale like \(\sigma^2/\varepsilon^2\sim K_{\text{train}}^2\).  Hence:</p>

<p>\[
\text{FLOPs} \;\approx\; K_{\text{train}}^2\,M\,C_{\text{tok}} .
\]</p>

<p>The approach trades a <em>one-shot</em> exponential inference wall for a <strong>quadratic training wall</strong> plus a super-linear runtime factor.  For long-horizon tasks (\(L\!\sim\!10^3\)) even moderate \(K_{\text{train}}\) (10–30) rivals the raw cost of running many CoT samples at test time.</p>

<h4 id="residual-fragility" id="residual-fragility">Residual fragility</h4>

<p>The distilled single path is still a chain of length \(L\); only the <em>selection</em> occurred offline. Any unmodelled context shift resurrects the original error law
\(S=(1-\varepsilon)^{L}\). The hidden loop cannot retroactively explore alternatives, so test-time flexibility is gone.</p>

<h4 id="net-effect" id="net-effect">Net effect</h4>

<p>Latent-scratchpad models <strong>compress</strong> exploration into training but pay:</p>
<ul><li>\(O(K_{\text{train}}^2)\) extra pre-compute,</li>
<li>\(O(rL))–(O(L^2)\) test-compute,</li>
<li>unchanged exponential sensitivity to horizon drift.</li></ul>

<p><br/>
They remain a band-aid for the \(e^{-\varepsilon L}\) failure mode.</p>

<h2 id="empirical-fault-line-long-range-coding-tasks" id="empirical-fault-line-long-range-coding-tasks">Empirical fault line: long-range coding tasks</h2>

<p>HumanEval, GSM8K, MMLU (the benchmarks that motivated current scaling laws) fit in ≤ 15 reasoning steps. They primarily measure <em>local</em> reasoning, and say nothing about the model&#39;s capacity for long range credit assignment.</p>

<p>Ask the same models to:</p>
<ul><li>iteratively fix a 10,000+ line codebase,<br/></li>
<li>pass a compiler-run-debug loop,<br/></li>
<li>coordinate across multiple tools and environments</li></ul>

<p><br/>
and error chains will quickly explode.</p>

<p>The perplexity gains observed in the flatter context disappear or become meaningless, implying that the <strong>unit of action</strong>, not the language loss, is the limiting factor.</p>

<h2 id="how-smaller-sparser-models-can-cope-better" id="how-smaller-sparser-models-can-cope-better">How smaller/sparser models can cope better</h2>

<p>The recipe that emerges from the above analysis is to shift optimization effort away from brute-forced token-level accuracy and toward <strong>explicit structuring of computation</strong>. We can achieve this through two orthogonal efforts:</p>
<ol><li><strong>Horizon compression</strong> – collapse long micro-token chains into a handful of macro actions that can be supervised or rolled back individually.<br/></li>
<li><strong>Selective activation</strong> – ensure that only a small, context-relevant slice of the parameter matrix participates in each forward/backward pass so new knowledge only perturbs a bounded region.</li></ol>

<p><br/><br/></p>

<p>These techniques convert the exponential failure surface \(e^{-\varepsilon L}\) into a tractable polynomial and bound the \(C/\varepsilon^{2}\) interference bleed. They do so through parameter <em>reuse</em> rather than parameter <em>accumulation</em>, keeping training and inference budgets within a single-GPU or single-node envelope.</p>

<h3 id="reliability-continual-learning-trade-offs-of-modular-techniques" id="reliability-continual-learning-trade-offs-of-modular-techniques">Reliability ↔ Continual-Learning Trade-offs of Modular Techniques</h3>

<table>
<thead>
<tr>
<th>Design move</th>
<th>Chain reliability</th>
<th>Continual-learning cost</th>
</tr>
</thead>

<tbody>
<tr>
<td>Hierarchical planner + executor</td>
<td>Cuts \(L_{\text{eff}}\)​ by 10–100×</td>
<td>Only planner/executor slice retrained</td>
</tr>

<tr>
<td>Verifier-in-the-loop</td>
<td>Detects &amp; rolls back errors every macro step</td>
<td>No weight update needed</td>
</tr>

<tr>
<td>Sparse MoE routing</td>
<td>Active params per token ≤ 10-50B</td>
<td>Replay on experts only</td>
</tr>

<tr>
<td>Retrieval memory</td>
<td>Offloads rare concepts, reduces interference</td>
<td>New info logged, not re-trained</td>
</tr>

<tr>
<td>LoRA / adapters</td>
<td>Drift confined to hand-sized matrices</td>
<td>Update cost ∝ adapter size</td>
</tr>
</tbody>
</table>

<p>Each technique lowers either \(L\) or interference, breaking the exponential failure mode without enlarging \(P\).</p>

<h2 id="conclusion" id="conclusion">Conclusion</h2>

<p>The mainstream scaling narrative extrapolates single-step accuracy and misses the exponential fragility of long chains. Past a short term horizon, dense parameter growth has severely diminishing returns – essentially it hits a wall: every new concept conflicts with all previous ones, while each additional reasoning step multiplies failure odds.</p>

<p>Hierarchical control, external verification, sparse activation and retrieval shift the curve: they compress the effective chain length and cut interference without bloating the active model. The systems that leverage small, modular, tool-using principles will likely achieve human-level macro accuracy first, even as gigantic models keep hitting the \(e^{-\varepsilon L}\) limit.</p>

<p>— <a href="https://x.com/matthewdif">Matthew Di Ferrante</a></p>

<p><br/></p>


]]></content:encoded>
      <guid>https://mattdf.xyz/the-scaling-laws-are-already-broken-smaller-models-win-out-on-reasoning-long</guid>
      <pubDate>Fri, 25 Apr 2025 10:55:11 +0000</pubDate>
    </item>
    <item>
      <title>Why Quantum Computing will take another 50 years</title>
      <link>https://mattdf.xyz/why-quantum-computing-will-take-another-50-years?pk_campaign=rss-feed</link>
      <description>&lt;![CDATA[This article is meant to be a short, well sourced summary about why we will not have quantum computing any time soon, with evidence that shows we have not made any meaningful progress for decades, at least nowhere near the level the PR lies from the tech industry would have led you to believe.&#xA;&#xA;   The actual challenges for quantum computing&#xA;      Decoherence time&#xA;      Gate fidelity&#xA;      Error correction&#xA;   The current landscape&#xA;      The only real qubit &#34;computation&#34; that has ever been done&#xA;      But I thought bigger numbers have been factored?&#xA;      Misleading and meaningless &#34;benchmarks&#34;&#xA;      What about the new types of qubits?&#xA;      Claims of &#34;improved algorithms&#34;&#xA;      Large scale manufacturing of &#34;Quantum hardware&#34;&#xA;   Scale of requirements for a real, useful quantum computer&#xA;   Conclusion&#xA;&#xA;br/&#xA;!--more--&#xA;&#xA;The actual challenges for quantum computing&#xA;&#xA;For real quantum computing, there are a few parameters that really matter. We are many, many orders of magnitudes away from the level we need to be on each of these parameters when it comes to realizing a real quantum computer that can do meaningful computation. Each category is a Manhattan project level effort (x10) to overcome, and all three need to be overcome, or you have nothing.&#xA;&#xA;Decoherence time&#xA;&#xA;This is how long the entangled state can remain stable/not collapse, and not just when it&#39;s doing &#34;nothing&#34;, but when the qubits are actually being operated on and moved through the chip/system. This gets harder the bigger the chip, the more complicated the qubit connectivity structure, and the more qubits that are entangled together. So far no (computationally) meaningful coherence has been achieved for anything other than a few qubits (less than 10).&#xA;&#xA;|decoherence visualization|&#xA;|:--:| &#xA;|A very simplified visualization of quantum decoherence event|&#xA;&#xA;On top of the state being more fragile by default due to Zeeman/Stark effects and other issues, the larger the number of qubits in a system, the higher the chance that control signals (microwave pulses for superconducting qubits, laser pulses for ions, etc.) interfere with each other due to exponentially increasing crosstalk,  worsening decoherence time even more.&#xA;&#xA;Gate fidelity&#xA;&#xA;A &#34;gate&#34; performs logical operations on qubits. For both electrons and photons, gates are not perfect. In isolation, a single gate might show high fidelity (e.g. 99%), but once you try to assemble a large, computation-supporting circuit that involves many gates in parallel, qubit routing, and frequent measurements, the effective error rate quickly climbs (often well past 50%) because every layer of gates and every source of crosstalk compounds. Many published 99% fidelities apply only to contrived demonstrations with few gates under highly controlled conditions, and as such are meaningless in a computation context. Once you entangle a significant number of qubits or shuttle states around, the errors multiply, requiring exponentially greater overhead for error correction.&#xA;&#xA;|quantum circuit topologies|&#xA;|:--:| &#xA;| Different quantum circuit topologies. The all-to-all configuration is the hardest to achieve, but even though the others are easier, they introduce additional complexity elsewhere (like needing more gates and more complex control systems) |&#xA;&#xA;Even if you can make small batches of qubits that are very locally stable, and try to &#34;link&#34; them together, large-scale algorithms require qubits to communicate efficiently in a &#34;fully connected&#34; manner. Physical layouts (e.g., 2D arrays in superconducting chips, non-local connections) heavily limit which qubits can directly interact. Swapping states over many &#34;hops&#34; to deal with poor connectivity inflates error even more. Robust long-distance quantum links are far from a solved problem.&#xA;&#xA;Error correction&#xA;&#xA;Because quantum states are noisy generally, and statistical in nature, colossal efforts are required to correct for errors so that the final result actually makes sense rather than being a random distribution. For universal QC, all operations also have to be &#34;reversible&#34;, and compounding error rates destroy that. This means that to actually have stable computation, in a quantum computer that would be able to factor real private keys using Shor, for 1,000-2,000 &#34;logical qubits&#34; you need millions of physical, error correcting qubits.&#xA;&#xA;|quantum error correction circuit|&#xA;|:--:| &#xA;| Example circuit that performs a hadamard transform on the one data qubit and its two ancillas (supporting qubits that enable reversible computation), and phase error correction through a Toffoli gategate). (source) |&#xA;&#xA;On top of this, error correction schemes (e.g., surface codes) require frequent and precise measurements of ancillary qubits without collapsing computational states. Even with the best error correcting codes, physical gate fidelity has to be below an error threshold of 0.01% to prevent errors from outpacing corrections.&#xA;&#xA;The error correcting overhead for the computation grows exponentially with the number of logical qubits needed, so scaling to even hundreds (much less thousands) of logical qubits is essentially impossible at the moment. There&#39;s also a time overhead - a large share of computational cycles are spent on error detection and correction, slowing down actual computation, which then requires greater decoherence time to offset... and so on.&#xA;&#xA;There&#39;s no &#34;free lunch&#34;, if you try to cheat by avoiding fully dealing with the problem complexity from one of these 3 areas, you make your life 100x harder when solving for the other 2.&#xA;&#xA;The current landscape&#xA;&#xA;Let&#39;s cover what the true state of the art is to set a baseline and recalibrate expectations created by misleading QC press releases. &#xA;&#xA;Some quick background: quantum computers come in 2 main flavors: electronic (manipulating electrons/charged particles), and photonic (manipulating photons, which have no charge).&#xA;&#xA;Electrons are easier to manipulate because they have charge and mass - so they aren&#39;t constantly trying to fly into everything at the speed of light - but they are far more susceptible to thermal noise and stray field effects. Photons are more robust to noise, but are harder to control, have greater signal loss and suffer from lower interaction rates, which still mess up your circuits - just in a different way.&#xA;&#xA;The only real qubit &#34;computation&#34; that has ever been done&#xA;&#xA;If you want to perform actually valuable quantum computation that gives you real quantum advantage, you need true, stable and fully entangled qubits - otherwise you cannot run general purpose algorithms like Shor or Grover that allow you to break cryptography or solve hard problems.&#xA;&#xA;For electronic qubits, the largest number factored is &#34;21&#34;, through a real quantum circuit of 4 qubits done in 2012: Computing prime factors with a Josephson phase qubit quantum processor&#xA;&#xA;For photonic qubits, the record for the largest number factored is &#34;15&#34;, also with 4 qubits, done in 2007: Demonstration of Shor&#39;s quantum factoring algorithm using photonic qubits&#xA;&#xA;There has been zero progress since. Additionally, both these photonic and electronic qubit implementations of Shor&#39;s algorithm relied on prior knowledge of the factors. These demos used a compiled or semi-classical version of the algorithm, meaning they optimized the quantum circuit based on precomputed classical information about the solution. So, it essentially useless for computation, and was only meant to show ability of fine qubit control.&#xA;&#xA;|1301.7007|&#xA;|:--:|&#xA;| Excerpt from &#34;Pretending to factor large numbers on a quantum computer&#34; |&#xA;&#xA;This is common in QC, because it&#39;s basically impossible to create any kind of real meaningful circuit that runs on actual, properly entangled qubits.&#xA;&#xA;But I thought bigger numbers have been factored?&#xA;&#xA;There are many other sources claiming that bigger numbers have been factored, like  143 or  56153, but this is often twisted into misleading nonsense by news articles. This is not Shor, these papers use either ising hamiltonian models or adiabatic quantum computing, which are processes closer to quantum annealing. They are converting the &#34;factoring of the number&#34; into an &#34;optimization&#34; problem, where the lowest energy state that the system settles to will automatically have the factor answer - but THESE ARE NOT GENERAL ALGORITHMS. It is not possible to use these algorithms to factor a private key that is 256 bits long. The bigger the number, the harder it is to encode into an optimization problem that fits an ising state - and eventually it becomes impossible as there&#39;s no way to map that solution into a (sensible) optimization problem. Below is an analogical visualization of the process.&#xA;&#xA;|AQC example|&#xA;|:--:|&#xA;| An analogy for Adiabatic Quantum Computation. The image shows a &#34;linear regression&#34; process (finding the best-fit for a line, given a set of points). The strings are poked through the holes and tied to the rod and pulled tight. If the same pulling force is applied to each string, the balance from all the different tensions will make the rod &#34;settle&#34; into the best fit. This is essentially how adiabatic quantum computation works, but in higher dimensions. You want to find a set of &#34;holes&#34; (a state encoding) that when &#34;pulled&#34; (electrons settling into lowest energy state), will give your your best-fit solution (in this case, the factors of a large number).&#xA;&#xA;These quantum annealing adjacent processes exploit quantum effects, but they&#39;re not leveraging the full computation ability of entanglement. It&#39;s more like how you can solve a maze by feeding water into it, and the flow of water will naturally take the path of least resistance, so the path where the water flows strongest is the solution.&#xA;&#xA;|water solving a maze|&#xA;|:--:|&#xA;| Water flow solving a maze (source) |&#xA;&#xA;Full quantum computation through entangled states is a different mechanism. A universal quantum computer uses sequences of carefully controlled operations (gates) to manipulate the inner properties of the entire entangled quantum state. The machine can interfere and redirect amplitudes in that exponentially large &#34;probability space,&#34; effectively doing many computations at once and selectively reinforcing the right outcomes via quantum interference. Quantum annealers don&#39;t provide that kind of control, so they can&#39;t access the full exponential power hidden in an entangled system.&#xA;&#xA;Misleading and meaningless &#34;benchmarks&#34;&#xA;&#xA;Many other papers that use different &#34;benchmarks&#34; for achieving quantum computation breakthroughs are also misleading, like Google Willow which uses Random Circuit Sampling (RCS) as a benchmark - to basically have a quantum system that can sample its own state... and they state that this it does this a septillion times faster than a classical computer... but of course it does? These kinds of benchmarks are equivalent to saying the following:&#xA;&#xA;  Our new computer, which is an aquarium tank full of water that we shake, simulates  fluid dynamics at a scale of 10^17 molecules a septillion times faster than if we were to try to simulate every particle in that whole aquarium tank on a classical computer&#xA;&#xA;It&#39;s not computing anything. The aquarium tank, just like the quantum system doing RCS, is not a &#34;simulator&#34;, it&#39;s just the real thing evolving its state according to the laws of physics. Of course it&#39;s &#34;faster&#34; than simulation of the same set of particles inside software.&#xA;&#xA;| QC utility |&#xA;|:--:|&#xA;| You might notice that all the things that quantum computers are &#34;claimed&#34; to be useful for at the moment are physics related |&#xA;&#xA;Although they&#39;re completely useless for computation, the ability of these systems to have fine-grained control over quantum states, electrons and photons, is useful for probing fundamental physics - as the image above shows. By creating precise entangled or semi-entangled systems, we can measure interactions and dynamics that are prohibitively complex through classical methods. It allows for exploration of exotic phases of matter, simulations of quantum field interactions, and the testing foundational principles of quantum mechanics in a setting with higher control and noise isolation than might be possible otherwise.&#xA;&#xA;What about the new types of qubits?&#xA;&#xA;Other claims, like those in Microsoft&#39;s paper on Majorana Zero Mode QC and the accompanying blog post of using &#34;new physics&#34; to create better qubits, border on intentional falsehoods. Microsoft has not found Majorana fermions, and has not confirmed topological states of any sort. The only way to prove that they created a real Majorana Zero Mode state would have been to do a non-abelian braid operation (exchange of MZM states), and they did not do this, so it does not rule out alternative effects that merely mimic majorana signatures but that do not offer topological protection. Even if there was observation of some topological protection, scaling this design up to multi-qubit entanglement would likely destroy it regardless, since the Kitaev chain model used is too fragile to do so robustly with the given design.&#xA;&#xA;|paper vs blog|&#xA;|:--:|&#xA;| An example of the egregious misrepresentation of a paper&#39;s results in the press release (paper left, blog right) |&#xA;&#xA;The comportment of these companies when it comes to announcing their QC supremacy is reaching the point that it&#39;s indistinguishable from fraud. Microsoft has released a very similar paper on this before which they eventually had to retract from Nature: Authors retract Nature Majorana paper, apologize for “insufficient scientific rigour”&#xA;&#xA;These kinds of approaches are in the &#34;exotic quasiparticle&#34; category, and scaling the creation of these stable quasiparticles substrates in a real system with crosstalk and actual functioning gates, rather than a toy setup that does nothing, is astronomically difficult. These are the &#34;holy grail&#34; because they avoid having to deal with quantum error correction, by providing topological protection from noise, but the approaches rely on novel, unproven or unsolved physics to actually work. If we could create and control these kinds of exotic quasiparticles at scale, the first step would not be quantum computing, rather we would already be using these techniques to make faster classical computers that leveraged spintronics or similar exotic physical effects that leverage quasiparticle fields (like with Quantum GPS).&#xA;&#xA;It&#39;s important to note that even if we do solve topological qubit creation enough to actually enable non-trivial quantum computation, it does not solve the gate fidelity problem. In fact, in some ways it may make it even harder because topological states are very particular about how they want to move through a medium (or field) and interact with other particles. For example, in topological insulators created by strong spin-orbit coupling/11%3ACoordinationChemistryIII-ElectronicSpectra/11.02%3AQuantumNumbersofMultielectronAtoms/11.2.02%3ASpin-OrbitCoupling), particles will have very rigid paths as they move through electromagnetic fields - applying the desired transforms (which are also just modulation the same EM field) to this particle without causing it to break out of its topologically locked path is an extremely hard, unsolved problem, especially when you have to deal with spatial constraints of physical reality.&#xA;&#xA;1]: Even if the paper holds up, it&#39;s a kitaev chain model for 1D quasiparticle majorana fermions, so it&#39;s not &#34;confirming&#34; the existence of fundamental majorana fermions any more than [Weyl Semimetals (which already experimentally support topological states) confirm the existence of Weyl Fermions&#xA;&#xA;Claims of &#34;improved algorithms&#34;&#xA;&#xA;There are many papers claiming that the quantum supremacy apocalypse moment will come &#34;sooner&#34; than expected now because of some new algorithm that can crack a bitcoin private key with &#34;just&#34; 50 million gates, but they are once again nonsense. They often mix the notion of physical and logical qubits, or in this case, assume a qubit gate fidelity, connectivity, and coherence time that is likely not physically realizable.&#xA;&#xA;|6 million qubit computer|&#xA;|:--:|&#xA;| The quantum computer the algorithm in the paper requires to run. Getting   6 million qubits to link together reliably in this way is essentially impossible, as the section below on real scaling challenges will illustrate |&#xA;&#xA;Usually these grand-claim papers rely on specialized arithmetic functions (e.g. quantum lookup tables, &#34;clean&#34; ancillas, custom group arithmetic), and don&#39;t give a full uncomputation scheme or a complete space-time resource breakdown. It&#39;s easy to say &#34;our modular multiplication is 2x faster&#34;, but if that approach needs significantly more qubits or more complicated control logic, the gains disappear.&#xA;&#xA;[2]: Support qubits that store &#34;scratch&#34; state used for enabling reversible computation&#xA;&#xA;Large scale manufacturing of &#34;Quantum hardware&#34;&#xA;&#xA;There are many low value press releases disguised as papers lately claiming that some company has been able to start mass manufacturing quantum computers, they are essentially all the same, with the same lies-by-omission, so I will just pick one and you can apply the same analysis to all of them. &#xA;&#xA;  A manufacturable platform for photonic quantum computing&#xA;&#xA;As with all &#34;progress reports&#34; in this category, they focus heavily on component-level performance, but avoid system-level benchmarks because they have no ability to actually create a real quantum computer by linking together these components.&#xA;&#xA;|quantum modules|&#xA;|:--:|&#xA;| This illustration, like all the others in this paper (and all other papers of this nature), shows high fidelity across single-hops, with idealized examples of clean/trivial quantum states. They state &#34;if in the simplest case, the photon actually makes it through, it does so with high fidelity&#34; but omit information how many attempts are required to get one photon to make it through and actually be detected. This only gets worse with scale. |&#xA;&#xA;For example, this is a photonics platform based on SFWM (Spontaneous Four-Wave Mixing). The paper handwaves away the unreliable nature of their photon sources. SFWM is inherently probabilistic, which means many attempts are needed to successfully generate the required photons. This interferes with actually linking these modules together in any meaningful way. There&#39;s some mention of BTO (Barium Titanate) electro-optic switches for multiplexing, but it&#39;s not enough to make any kind of meaningful computation possible.&#xA;&#xA;Scaling this system requires immense resources across many verticals. FBQC (Fusion Based Quantum Computation) needs enormous numbers of photons, and there&#39;s no concrete estimates stated of how many physical components would be needed for practically useful algorithms. &#xA;&#xA;In the next section, I&#39;ll use this paper as an example of how hard it would actually be to scale.&#xA;&#xA;Scale of requirements for a real, useful quantum computer&#xA;&#xA;If we do some rough math, for running Shor on a 256-bit ECC key (such as those used in bitcoin), we can see how ridiculous it gets. Shor requires at least 1,000 logical qubits to break such keys, so let&#39;s use that as the baseline.&#xA;&#xA;For a realistic error rate, each logical qubit might require a code with distance ~15-25. Using a surface code or similar, this translates to ~1,000-2,500 physical qubits per logical qubit.&#xA;&#xA;Each resource state preparation is probabilistic, success probability of fusion operations in the paper is ~99% for toy setups, but multiple attempts are needed per fusion operation in a real setup. As such, success probability per photon source is low (~1-10%). For reliable operation, system needs ~10-100x multiplexing.&#xA;&#xA;So our requirements for this system are:&#xA;&#xA;1,000 logical qubits × 2,000 physical qubits per logical qubit = 2,000,000 physical qubits&#xA;Multiplexing overhead (10-100x): 20-200 million photon sources&#xA;Number of detectors: Similar to number of sources, 20-200 million&#xA;Number of switches and other optical components: Roughly 10x the number of sources, so 200 million to 2 billion&#xA;&#xA;br /&#xA;&#xA;The chip in the paper has at most a few dozen active components (sources, detectors, switches). While for a real quantum computer, we need hundreds of millions to billions. That&#39;s 8-9 orders of magnitude from this proof of concept. &#xA;&#xA;|real quantum computer|&#xA;|:--:|&#xA;|This is probably around the size of this photonics computer if you were to actually try to realize it with current technology (and probably still fail)|&#xA;&#xA;That&#39;s not even counting the added complexity for the required control systems and needing to handle extremely fast and sophisticated classical feedback to keep up with the quantum error‐correction cycles, which would easily be another bottleneck (the system is bound by the &#34;reaction limit&#34;), or source of further instability, as it would be a huge amount of computation that would create heat and other noise, and is just an extremely non-trivial classical component with its own physics and engineering challenges that have not been solved at all at scale.&#xA;&#xA;Manufacturing hundreds of millions to billions of these components with high yield / low defect rate, connecting them in a computationally useful way, and keeping signal loss &amp; error rates from multiplying exponentially, while also having to keep the entire massive system at 2 kelvin, is not something humanity is capable of with our current technology and understanding of physics.&#xA;&#xA;[3]: &#34;Reaction limit&#34; refers to the minimum runtime imposed by the need to measure qubits, process those measurement results classically (including error‐decoding), and then feed the outcome back into the quantum circuit. Many fault‐tolerant gates in a surface‐code architecture require this real‐time classical feedback to proceed. Even if the physical qubits and gates can operate at very high speed, no step that depends on measurement results can finish sooner than the classical processing latency. Thus, if the algorithm has a certain &#34;reaction depth&#34; (number of measurement‐and‐feedforward layers), and each layer requires (say) 10 µs of classical processing, then you cannot run the computation in under reactiondepth x 10 µs, no matter how fast the hardware otherwise is.&#xA;&#xA;Conclusion&#xA;&#xA;We are nowhere near being able to realize useful quantum computation. It is not coming in 10 or 20 years. If we have figured out how to create real, stable topological qubits in 10-20 years, the progress can start then. Until that point, you should disregard pretty much all news in this category.&#xA;&#xA;Our current state of the art is as far off from real quantum computing, as using a magnifying glass and the sun to mark wood is to EUV lithography.&#xA;&#xA;-- a href=&#34;https://x.com/matthewdif&#34;Matthew Di Ferrante/a&#xA;&#xA;br /&#xA;&#xA;!--emailsub--]]&gt;</description>
      <content:encoded><![CDATA[<p>This article is meant to be a short, well sourced summary about why we will not have quantum computing any time soon, with evidence that shows we have not made any meaningful progress for decades, at least nowhere near the level the PR lies from the tech industry would have led you to believe.</p>
<ul><li><a href="#the-actual-challenges-for-quantum-computing">The actual challenges for quantum computing</a>
<ul><li><a href="#decoherence-time">Decoherence time</a></li>
<li><a href="#gate-fidelity">Gate fidelity</a></li>
<li><a href="#error-correction">Error correction</a></li></ul></li>
<li><a href="#the-current-landscape">The current landscape</a>
<ul><li><a href="#the-only-real-qubit-computation-that-has-ever-been-done">The only real qubit “computation” that has ever been done</a></li>
<li><a href="#but-i-thought-bigger-numbers-have-been-factored">But I thought bigger numbers have been factored?</a></li>
<li><a href="#misleading-and-meaningless-benchmarks">Misleading and meaningless “benchmarks”</a></li>
<li><a href="#what-about-the-new-types-of-qubits">What about the new types of qubits?</a></li>
<li><a href="#claims-of-improved-algorithms">Claims of “improved algorithms”</a></li>
<li><a href="#large-scale-manufacturing-of-quantum-hardware">Large scale manufacturing of “Quantum hardware”</a></li></ul></li>
<li><a href="#scale-of-requirements-for-a-real-useful-quantum-computer">Scale of requirements for a real, useful quantum computer</a></li>
<li><a href="#conclusion">Conclusion</a></li></ul>

<p><br/>
</p>

<h2 id="the-actual-challenges-for-quantum-computing" id="the-actual-challenges-for-quantum-computing">The actual challenges for quantum computing</h2>

<p>For real quantum computing, there are a few parameters that really matter. We are many, <em>many</em> orders of magnitudes away from the level we need to be on each of these parameters when it comes to realizing a real quantum computer that can do meaningful computation. Each category is a Manhattan project level effort (x10) to overcome, and all three need to be overcome, or you have nothing.</p>

<h3 id="decoherence-time" id="decoherence-time">Decoherence time</h3>

<p>This is <a href="https://www.cambridge.org/core/books/quantum-error-correction/introduction-to-decoherence-and-noise-in-open-quantum-systems/E35DA780DBF36618E0715D97E36264A0">how long the entangled state can remain stable/not collapse</a>, and not just when it&#39;s doing “nothing”, but when the qubits are actually being operated on and moved through the chip/system. This gets harder the bigger the chip, the more complicated the qubit connectivity structure, and the more qubits that are entangled together. So far no (computationally) meaningful coherence has been achieved for anything other than a few qubits (less than 10).</p>

<table>
<thead>
<tr>
<th align="center"><img src="https://i.snap.as/bEZtuWo5.png" alt="decoherence visualization"/></th>
</tr>
</thead>

<tbody>
<tr>
<td align="center"><em>A very simplified visualization of quantum decoherence event</em></td>
</tr>
</tbody>
</table>

<p>On top of the state being more fragile by default due to <a href="https://en.wikipedia.org/wiki/Zeeman_effect">Zeeman</a>/<a href="https://en.wikipedia.org/wiki/Stark_effect">Stark</a> effects and other issues, the larger the number of qubits in a system, the higher the chance that control signals (microwave pulses for superconducting qubits, laser pulses for ions, etc.) interfere with each other due to <a href="https://journals.aps.org/prxquantum/abstract/10.1103/PRXQuantum.3.020301">exponentially increasing crosstalk</a>,  worsening decoherence time even more.</p>

<h3 id="gate-fidelity" id="gate-fidelity">Gate fidelity</h3>

<p>A <a href="https://en.wikipedia.org/wiki/Quantum_logic_gate#:~:text=Quantum%20logic%20gates%20are%20the,are%20for%20conventional%20digital%20circuits.&amp;text=Unlike%20many%20classical%20logic%20gates,computing%20using%20only%20reversible%20gates.">“gate”</a> performs logical operations on qubits. For both electrons and photons, <a href="https://www.epfl.ch/labs/lcpt/research/quantum_fidelity/">gates are not perfect</a>. In isolation, a single gate might show high fidelity (e.g. 99%), but once you try to assemble a large, computation-supporting circuit that involves many gates in parallel, qubit routing, and frequent measurements, the effective error rate quickly climbs (often well past 50%) because every layer of gates and every source of crosstalk compounds. Many published 99% fidelities apply only to contrived demonstrations with few gates under highly controlled conditions, and as such are meaningless in a computation context. Once you entangle a significant number of qubits or shuttle states around, the errors multiply, requiring exponentially greater overhead for error correction.</p>

<table>
<thead>
<tr>
<th align="center"><img src="https://i.snap.as/tmePBK6C.png" alt="quantum circuit topologies"/></th>
</tr>
</thead>

<tbody>
<tr>
<td align="center"><em>Different quantum circuit topologies. The all-to-all configuration is the hardest to achieve, but even though the others are easier, they introduce additional complexity elsewhere (like needing more gates and more complex control systems)</em></td>
</tr>
</tbody>
</table>

<p>Even if you can make small batches of qubits that are very locally stable, and try to “link” them together, large-scale algorithms require qubits to communicate efficiently in a “fully connected” manner. Physical layouts (e.g., 2D arrays in superconducting chips, non-local connections) heavily limit which qubits can directly interact. Swapping states over many “hops” to deal with poor connectivity inflates error even more. <a href="https://en.wikipedia.org/wiki/Quantum_network#Repeaters">Robust long-distance quantum links</a> are far from a solved problem.</p>

<h3 id="error-correction" id="error-correction">Error correction</h3>

<p>Because quantum states are noisy generally, and statistical in nature, colossal efforts are required to correct for errors so that the final result actually makes sense rather than being a random distribution. For universal QC, all operations also have to be <a href="https://en.wikipedia.org/wiki/Reversible_computing">“reversible”</a>, and compounding error rates destroy that. This means that to actually have stable computation, in a quantum computer that would be able to factor real private keys using <a href="https://en.wikipedia.org/wiki/Shor%27s_algorithm">Shor</a>, for 1,000-2,000 “logical qubits” you need <em>millions</em> of physical, <a href="https://arxiv.org/abs/quant-ph/9606012">error correcting qubits</a>.</p>

<table>
<thead>
<tr>
<th align="center"><img src="https://i.snap.as/tWGPt9kH.png" alt="quantum error correction circuit"/></th>
</tr>
</thead>

<tbody>
<tr>
<td align="center"><em>Example circuit that performs a <a href="https://en.wikipedia.org/wiki/Quantum_logic_gate#Hadamard_gate">hadamard transform</a> on the one data qubit and its two ancillas (supporting qubits that enable reversible computation), and phase error correction through a <a href="https://en.wikipedia.org/wiki/Quantum_logic_gate#Toffoli_(CCNOT)_gate">Toffoli gate</a>. <a href="https://www.nature.com/articles/s41586-022-04986-6">(source)</a></em></td>
</tr>
</tbody>
</table>

<p>On top of this, error correction schemes (e.g., <a href="https://arthurpesah.me/blog/2023-05-13-surface-code/">surface codes</a>) require frequent and precise measurements of ancillary qubits without collapsing computational states. Even with the best error correcting codes, physical gate fidelity has to be below an error threshold of 0.01% to prevent errors from outpacing corrections.</p>

<p>The <a href="https://arxiv.org/abs/2302.04317">error correcting overhead</a> for the computation grows exponentially with the number of logical qubits needed, so scaling to even hundreds (much less thousands) of logical qubits is essentially impossible at the moment. There&#39;s also a time overhead – a large share of computational cycles are spent on error detection and correction, slowing down actual computation, which then requires greater decoherence time to offset... and so on.</p>

<p>There&#39;s no “free lunch”, if you try to cheat by avoiding fully dealing with the problem complexity from one of these 3 areas, you make your life 100x harder when solving for the other 2.</p>

<h2 id="the-current-landscape" id="the-current-landscape">The current landscape</h2>

<p>Let&#39;s cover what the true state of the art is to set a baseline and recalibrate expectations created by misleading QC press releases.</p>

<p>Some quick background: quantum computers come in 2 main flavors: electronic (manipulating electrons/charged particles), and photonic (manipulating photons, which have no charge).</p>

<p>Electrons are easier to manipulate because they have charge and mass – so they aren&#39;t constantly trying to fly into everything at the speed of light – but they are far more susceptible to thermal noise and stray field effects. Photons are more robust to noise, but are harder to control, have greater signal loss and suffer from lower interaction rates, which still mess up your circuits – just in a different way.</p>

<h3 id="the-only-real-qubit-computation-that-has-ever-been-done" id="the-only-real-qubit-computation-that-has-ever-been-done">The only real qubit “computation” that has ever been done</h3>

<p>If you want to perform actually valuable quantum computation that gives you real quantum advantage, you need true, stable and fully entangled qubits – otherwise you cannot run general purpose algorithms like <a href="https://en.wikipedia.org/wiki/Shor%27s_algorithm">Shor</a> or <a href="https://en.wikipedia.org/wiki/Grover%27s_algorithm">Grover</a> that allow you to break cryptography or solve hard problems.</p>

<p>For electronic qubits, the largest number factored is “21”, through a real quantum circuit of 4 qubits done in 2012: <a href="https://www.nature.com/articles/nphys2385">Computing prime factors with a Josephson phase qubit quantum processor</a></p>

<p>For photonic qubits, the record for the largest number factored is “15”, also with 4 qubits, done in 2007: <a href="https://arxiv.org/abs/0705.1684">Demonstration of Shor&#39;s quantum factoring algorithm using photonic qubits</a></p>

<p>There has been zero progress since. Additionally, both these photonic and electronic qubit implementations of Shor&#39;s algorithm relied on prior knowledge of the factors. These demos used a compiled or semi-classical version of the algorithm, meaning they optimized the quantum circuit based on precomputed classical information about the solution. So, it essentially useless for computation, and was only meant to show ability of fine qubit control.</p>

<table>
<thead>
<tr>
<th align="center"><img src="https://i.snap.as/I4xh4wOU.png" alt="1301.7007"/></th>
</tr>
</thead>

<tbody>
<tr>
<td align="center"><em>Excerpt from <a href="https://arxiv.org/abs/1301.7007">“Pretending to factor large numbers on a quantum computer”</a></em></td>
</tr>
</tbody>
</table>

<p>This is common in QC, because it&#39;s basically impossible to create any kind of real meaningful circuit that runs on actual, properly entangled qubits.</p>

<h3 id="but-i-thought-bigger-numbers-have-been-factored" id="but-i-thought-bigger-numbers-have-been-factored">But I thought bigger numbers have been factored?</h3>

<p>There are many other sources claiming that bigger numbers have been factored, like  <a href="https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.108.130501">143</a> or  <a href="https://arxiv.org/pdf/1411.6758">56153</a>, but this is often twisted into misleading nonsense by news articles. This is not Shor, these papers use either <a href="https://arxiv.org/abs/1808.08927">ising hamiltonian models</a> or <a href="https://en.wikipedia.org/wiki/Adiabatic_quantum_computation">adiabatic quantum computing</a>, which are processes closer to <a href="https://en.wikipedia.org/wiki/Quantum_annealing">quantum annealing</a>. They are converting the “factoring of the number” into an <a href="https://en.wikipedia.org/wiki/Quadratic_unconstrained_binary_optimization">“optimization” problem</a>, where the lowest energy state that the system settles to will automatically have the factor answer – but <strong>THESE ARE NOT GENERAL ALGORITHMS</strong>. It is not possible to use these algorithms to factor a private key that is 256 bits long. The bigger the number, the harder it is to encode into an optimization problem that fits an ising state – and eventually it becomes impossible as there&#39;s no way to map that solution into a (sensible) optimization problem. Below is an analogical visualization of the process.</p>

<table>
<thead>
<tr>
<th align="center"><img src="https://i.snap.as/y1Q7sQyl.jpg" alt="AQC example"/></th>
</tr>
</thead>

<tbody>
<tr>
<td align="center"><em>An analogy for Adiabatic Quantum Computation. The image shows a “linear regression” process (finding the best-fit for a line, given a set of points). The strings are poked through the holes and tied to the rod and pulled tight. If the same pulling force is applied to each string, the balance from all the different tensions will make the rod “settle” into the best fit. This is essentially how adiabatic quantum computation works, but in higher dimensions. You want to find a set of “holes” (a state encoding) that when “pulled” (electrons settling into lowest energy state), will give your your best-fit solution (in this case, the factors of a large number).</em></td>
</tr>
</tbody>
</table>

<p>These quantum annealing adjacent processes exploit quantum <em>effects</em>, but they&#39;re not leveraging the full computation ability of entanglement. It&#39;s more like how you can solve a maze by feeding water into it, and the flow of water will naturally take the path of least resistance, so the path where the water flows strongest is the solution.</p>

<table>
<thead>
<tr>
<th align="center"><img src="https://i.snap.as/hzY7729S.png" alt="water solving a maze"/></th>
</tr>
</thead>

<tbody>
<tr>
<td align="center"><em>Water flow solving a maze <a href="https://www.youtube.com/watch?v=81ebWToAnvA">(source)</a></em></td>
</tr>
</tbody>
</table>

<p>Full quantum computation through entangled states is a different mechanism. A universal quantum computer uses sequences of carefully controlled operations (gates) to manipulate the inner properties of the entire entangled quantum state. The machine can interfere and redirect amplitudes in that exponentially large “probability space,” effectively doing many computations at once and selectively reinforcing the right outcomes via quantum interference. Quantum annealers don&#39;t provide that kind of control, so they can&#39;t access the full exponential power hidden in an entangled system.</p>

<h3 id="misleading-and-meaningless-benchmarks" id="misleading-and-meaningless-benchmarks">Misleading and meaningless “benchmarks”</h3>

<p>Many other papers that use different “benchmarks” for achieving quantum computation breakthroughs are also misleading, like <a href="https://blog.google/technology/research/google-willow-quantum-chip/">Google Willow</a> which uses <a href="https://en.wikipedia.org/wiki/Quantum_random_circuits">Random Circuit Sampling</a> (RCS) as a benchmark – to basically have a quantum system that can sample its own state... and they state that this it does this a septillion times faster than a classical computer... but of course it does? These kinds of benchmarks are equivalent to saying the following:</p>

<blockquote><p>Our new computer, which is an aquarium tank full of water that we shake, simulates  fluid dynamics at a scale of 10^17 molecules a septillion times faster than if we were to try to simulate every particle in that whole aquarium tank on a classical computer</p></blockquote>

<p>It&#39;s not computing anything. The aquarium tank, just like the quantum system doing RCS, is not a “simulator”, it&#39;s just the real thing evolving its state according to the laws of physics. Of course it&#39;s “faster” than simulation of the same set of particles inside software.</p>

<table>
<thead>
<tr>
<th align="center"><img src="https://i.snap.as/VLXFvpy1.png" alt="QC utility"/></th>
</tr>
</thead>

<tbody>
<tr>
<td align="center"><em>You might notice that all the things that quantum computers are “claimed” to be useful for at the moment are physics related</em></td>
</tr>
</tbody>
</table>

<p>Although they&#39;re completely useless for computation, the ability of these systems to have fine-grained control over quantum states, electrons and photons, is useful for probing fundamental physics – as the image above shows. By creating precise entangled or semi-entangled systems, we can measure interactions and dynamics that are prohibitively complex through classical methods. It allows for exploration of exotic phases of matter, simulations of quantum field interactions, and the testing foundational principles of quantum mechanics in a setting with higher control and noise isolation than might be possible otherwise.</p>

<h3 id="what-about-the-new-types-of-qubits" id="what-about-the-new-types-of-qubits">What about the new types of qubits?</h3>

<p>Other claims, like those in Microsoft&#39;s paper on <a href="https://www.nature.com/articles/s41586-024-08445-2">Majorana Zero Mode QC</a> and the <a href="https://news.microsoft.com/source/features/innovation/microsofts-majorana-1-chip-carves-new-path-for-quantum-computing/">accompanying blog post</a> of using “new physics” to create better qubits, border on intentional falsehoods. Microsoft has not found <a href="https://en.wikipedia.org/wiki/Majorana_fermion">Majorana</a> fermions[^1], and has not confirmed topological states of any sort. The only way to prove that they created a real Majorana Zero Mode state would have been to do a <a href="https://en.wikipedia.org/wiki/Anyon#Non-abelian_anyons">non-abelian braid operation</a> (exchange of MZM states), and they did not do this, so it does not rule out alternative effects that merely mimic majorana signatures but that <a href="https://physics.aps.org/articles/v16/24">do not offer topological protection</a>. Even if there was observation of some topological protection, scaling this design up to multi-qubit entanglement would likely destroy it regardless, since the <a href="https://en.wikipedia.org/wiki/Kitaev_chain">Kitaev chain</a> model used is too fragile to do so robustly with the given design.</p>

<table>
<thead>
<tr>
<th align="center"><img src="https://i.snap.as/W8EBJQh2.jpeg" alt="paper vs blog"/></th>
</tr>
</thead>

<tbody>
<tr>
<td align="center"><em>An example of the egregious misrepresentation of a paper&#39;s results in the press release (paper left, blog right)</em></td>
</tr>
</tbody>
</table>

<p>The comportment of these companies when it comes to announcing their QC supremacy is reaching the point that it&#39;s indistinguishable from fraud. Microsoft has released a very similar paper on this before which they eventually had to retract from Nature: <a href="https://retractionwatch.com/2021/03/08/authors-retract-nature-majorana-paper-apologize-for-insufficient-scientific-rigour/">Authors retract Nature Majorana paper, apologize for “insufficient scientific rigour”</a></p>

<p>These kinds of approaches are in the <a href="https://en.wikipedia.org/wiki/Quasiparticle">“exotic quasiparticle”</a> category, and scaling the creation of these stable quasiparticles substrates in a real system with crosstalk and actual functioning gates, rather than a toy setup that does nothing, is astronomically difficult. These are the “holy grail” because they avoid having to deal with quantum error correction, by providing <a href="https://en.wikipedia.org/wiki/Topological_insulator">topological protection</a> from noise, but the approaches rely on novel, unproven or unsolved physics to actually work. If we could create and control these kinds of exotic quasiparticles at scale, the first step would not be quantum computing, rather we would already be using these techniques to make faster classical computers that leveraged <a href="https://en.wikipedia.org/wiki/Spintronics">spintronics</a> or similar exotic physical effects that leverage quasiparticle fields (like with <a href="https://phys.org/news/2024-07-gps-problem-quantum-tools-compact.html">Quantum GPS</a>).</p>

<p>It&#39;s important to note that even if we do solve topological qubit creation enough to actually enable non-trivial quantum computation, it <strong>does not</strong> solve the gate fidelity problem. In fact, in some ways it may make it even harder because topological states are very particular about how they want to move through a medium (or field) and interact with other particles. For example, in topological insulators created by strong <a href="https://chem.libretexts.org/Bookshelves/Inorganic_Chemistry/Inorganic_Chemistry_(LibreTexts)/11%3A_Coordination_Chemistry_III_-_Electronic_Spectra/11.02%3A_Quantum_Numbers_of_Multielectron_Atoms/11.2.02%3A_Spin-Orbit_Coupling">spin-orbit coupling</a>, particles will have very rigid paths as they move through electromagnetic fields – applying the desired transforms (which are also just modulation the same EM field) to this particle without causing it to break out of its topologically locked path is an extremely hard, unsolved problem, especially when you have to deal with spatial constraints of physical reality.</p>

<p>[1]: Even if the paper holds up, it&#39;s a kitaev chain model for 1D quasiparticle majorana fermions, so it&#39;s not “confirming” the existence of fundamental majorana fermions any more than <a href="https://en.wikipedia.org/wiki/Weyl_semimetal">Weyl Semimetals</a> (which already experimentally support topological states) confirm the existence of <a href="https://en.wikipedia.org/wiki/Weyl_equation">Weyl Fermions</a></p>

<h3 id="claims-of-improved-algorithms" id="claims-of-improved-algorithms">Claims of “improved algorithms”</h3>

<p>There are many papers claiming that the quantum supremacy apocalypse moment will come “sooner” than expected now because of some <a href="https://arxiv.org/pdf/2306.08585">new algorithm that can crack a bitcoin private key with “just” 50 million gates</a>, but they are once again nonsense. They often mix the notion of physical and logical qubits, or in this case, assume a qubit gate fidelity, connectivity, and coherence time that is likely not physically realizable.</p>

<table>
<thead>
<tr>
<th align="center"><img src="https://i.snap.as/1cyKaFyu.png" alt="6 million qubit computer"/></th>
</tr>
</thead>

<tbody>
<tr>
<td align="center"><em>The quantum computer the algorithm in the paper requires to run. Getting &gt;6 million qubits to link together reliably in this way is essentially impossible, as the section below on real scaling challenges will illustrate</em></td>
</tr>
</tbody>
</table>

<p>Usually these grand-claim papers rely on specialized arithmetic functions (e.g. quantum lookup tables, “clean” ancillas[^2], custom group arithmetic), and don&#39;t give a full uncomputation scheme or a complete space-time resource breakdown. It&#39;s easy to say “our modular multiplication is 2x faster”, but if that approach needs significantly more qubits or more complicated control logic, the gains disappear.</p>

<p>[2]: Support qubits that store “scratch” state used for enabling reversible computation</p>

<h3 id="large-scale-manufacturing-of-quantum-hardware" id="large-scale-manufacturing-of-quantum-hardware">Large scale manufacturing of “Quantum hardware”</h3>

<p>There are many low value press releases disguised as papers lately claiming that some company has been able to start mass manufacturing quantum computers, they are essentially all the same, with the same lies-by-omission, so I will just pick one and you can apply the same analysis to all of them.</p>

<blockquote><p><a href="https://arxiv.org/abs/2404.17570">A manufacturable platform for photonic quantum computing</a></p></blockquote>

<p>As with all “progress reports” in this category, they focus heavily on component-level performance, but avoid system-level benchmarks because they have no ability to actually create a real quantum computer by linking together these components.</p>

<table>
<thead>
<tr>
<th align="center"><img src="https://i.snap.as/3DrjWjtt.png" alt="quantum modules"/></th>
</tr>
</thead>

<tbody>
<tr>
<td align="center"><em>This illustration, like all the others in this paper (and all other papers of this nature), shows high fidelity across single-hops, with idealized examples of clean/trivial quantum states. They state “if in the simplest case, the photon actually makes it through, it does so with high fidelity” but omit information how many attempts are required to get one photon to make it through and actually be detected. This only gets worse with scale.</em></td>
</tr>
</tbody>
</table>

<p>For example, this is a photonics platform based on <a href="https://en.wikipedia.org/wiki/Four-wave_mixing">SFWM</a> (Spontaneous Four-Wave Mixing). The paper handwaves away the unreliable nature of their photon sources. SFWM is inherently probabilistic, which means many attempts are needed to successfully generate the required photons. This interferes with actually linking these modules together in any meaningful way. There&#39;s some mention of BTO (Barium Titanate) electro-optic switches for multiplexing, but it&#39;s not enough to make any kind of meaningful computation possible.</p>

<p>Scaling this system requires immense resources across many verticals. <a href="https://arxiv.org/abs/2101.09310">FBQC</a> (Fusion Based Quantum Computation) needs enormous numbers of photons, and there&#39;s no concrete estimates stated of how many physical components would be needed for practically useful algorithms.</p>

<p>In the next section, I&#39;ll use this paper as an example of how hard it would actually be to scale.</p>

<h2 id="scale-of-requirements-for-a-real-useful-quantum-computer" id="scale-of-requirements-for-a-real-useful-quantum-computer">Scale of requirements for a real, useful quantum computer</h2>

<p>If we do some rough math, for running <a href="https://stackoverflow.com/questions/41397576/how-many-qubits-do-i-need-to-factor-15-using-shors-algorithm#:~:text=In%20total%20you%20need%204n,3%20qubits%20for%20general%20number.">Shor on a 256-bit ECC key</a> (such as those used in bitcoin), we can see how ridiculous it gets. Shor requires at least 1,000 logical qubits to break such keys, so let&#39;s use that as the baseline.</p>

<p>For a realistic error rate, each logical qubit might require a <a href="https://quantumcomputing.stackexchange.com/questions/29516/physical-qubit-estimates-when-using-surface-codes">code with distance ~15-25</a>. Using a surface code or similar, this translates to ~1,000-2,500 physical qubits per logical qubit.</p>

<p>Each resource <a href="https://pubs.aip.org/aip/rsi/article/91/4/041101/1030773/Single-photon-sources-Approaching-the-ideal">state preparation is probabilistic</a>, success probability of fusion operations in the paper is ~99% for toy setups, but multiple attempts are needed per fusion operation in a real setup. As such, success probability per photon source is low (~1-10%). For reliable operation, system needs ~10-100x multiplexing.</p>

<p>So our requirements for this system are:</p>
<ul><li>1,000 logical qubits × 2,000 physical qubits per logical qubit = 2,000,000 physical qubits</li>
<li>Multiplexing overhead (10-100x): 20-200 million photon sources</li>
<li>Number of detectors: Similar to number of sources, 20-200 million</li>
<li>Number of switches and other optical components: Roughly 10x the number of sources, so 200 million to 2 billion</li></ul>

<p><br/></p>

<p>The chip in the paper has at most a few dozen active components (sources, detectors, switches). While for a real quantum computer, we need hundreds of millions to billions. That&#39;s 8-9 orders of magnitude from this proof of concept.</p>

<table>
<thead>
<tr>
<th align="center"><img src="https://i.snap.as/Y9JYwQsB.webp" alt="real quantum computer"/></th>
</tr>
</thead>

<tbody>
<tr>
<td align="center"><em>This is probably around the size of this photonics computer if you were to actually try to realize it with current technology (and probably still fail)</em></td>
</tr>
</tbody>
</table>

<p>That&#39;s not even counting the added complexity for the required <a href="https://arxiv.org/abs/1409.2202">control systems</a> and needing to handle extremely fast and sophisticated classical feedback to keep up with the quantum error‐correction cycles, which would easily be another bottleneck (the system is bound by the “reaction limit”[^3]), or source of further instability, as it would be a huge amount of computation that would create heat and other noise, and is just an extremely non-trivial classical component with its own physics and engineering challenges that have not been solved at all at scale.</p>

<p>Manufacturing hundreds of millions to billions of these components with high yield / low defect rate, connecting them in a computationally useful way, and keeping signal loss &amp; error rates from multiplying exponentially, while also having to keep the entire massive system at 2 kelvin, is not something humanity is capable of with our current technology and understanding of physics.</p>

<p>[3]: “Reaction limit” refers to the minimum runtime imposed by the need to measure qubits, process those measurement results classically (including error‐decoding), and then feed the outcome back into the quantum circuit. Many fault‐tolerant gates in a surface‐code architecture require this real‐time classical feedback to proceed. Even if the physical qubits and gates can operate at very high speed, no step that depends on measurement results can finish sooner than the classical processing latency. Thus, if the algorithm has a certain “reaction depth” (number of measurement‐and‐feedforward layers), and each layer requires (say) 10 µs of classical processing, then you cannot run the computation in under reaction_depth x 10 µs, no matter how fast the hardware otherwise is.</p>

<h2 id="conclusion" id="conclusion">Conclusion</h2>

<p>We are nowhere near being able to realize useful quantum computation. It is not coming in 10 or 20 years. If we have figured out how to create real, stable topological qubits in 10-20 years, the progress can <em>start</em> then. Until that point, you should disregard pretty much all news in this category.</p>

<p>Our current state of the art is as far off from real quantum computing, as using a magnifying glass and the sun to mark wood is to <a href="https://en.wikipedia.org/wiki/Extreme_ultraviolet_lithography">EUV lithography</a>.</p>

<p>— <a href="https://x.com/matthewdif">Matthew Di Ferrante</a></p>

<p><br/></p>


]]></content:encoded>
      <guid>https://mattdf.xyz/why-quantum-computing-will-take-another-50-years</guid>
      <pubDate>Thu, 06 Mar 2025 23:28:41 +0000</pubDate>
    </item>
  </channel>
</rss>