Where AI Compute Goes When It Comes Home

In my last post I argued that once model progress levels off, inference moves toward the people using it. The closest person using it is sitting in your living room. So I went and counted the hardware built to put a model there: 21 desktop AI boxes announced or shipped in the last twelve months, from 16 companies. Most launched somewhere between $2,000 and $4,000.

Twenty-One Boxes in a Year

Nvidia announced Project DIGITS at CES in January 2025: 128GB of unified memory for “$3,000,” due in May. Apple shipped first. Two months later the Mac Studio could be configured with up to 512GB, which Apple called “the most unified memory ever in a personal computer,” and it pitched the machine as able to run models “with over 600 billion parameters entirely in memory.” DIGITS finally went on sale in October as the DGX Spark, at $3,999.

Then everyone piled in. Seven PC makers sell their own version of the Spark’s GB10 board. AMD’s Ryzen AI Max+ 395, the chip everyone calls Strix Halo, ended up in mini PCs from Framework and a long list of smaller brands, and this summer AMD started selling its own box through Micro Center. On August 25 Apple announced the M5 Mac Studio, shipping September 22.

Platform Sold by Max memory Memory bandwidth Price
Nvidia GB10 Nvidia, Acer, ASUS, Dell, Gigabyte, HP, Lenovo, MSI 128GB 273GB/s DGX Spark: $3,999 at launch, $4,699 since February
AMD Ryzen AI Max+ 395 AMD, Framework, many mini PC brands 128GB 256GB/s AMD Ryzen AI Halo: $3,999.99
Apple M5 Max Apple (Mac Studio) 128GB 614GB/s From $2,499
Apple M5 Ultra Apple (Mac Studio) 512GB 1.2TB/s From $5,499; 512GB config due late October

Then Memory Got Expensive

What makes these boxes useful is a big pool of fast memory, and memory is the part that blew up. TrendForce says conventional DRAM contract prices rose roughly 93% to 98% in the first quarter of 2026 alone, with another 58% to 63% forecast for the second. It puts the blame on AI servers soaking up general-purpose memory. The same build-out I wrote about last time is buying the same chips.

Device Launch price Price now
Raspberry Pi 5, 16GB $120 (January 2025) $305
Framework Desktop, 128GB $1,999 (February 2025) $3,449, out of stock
Mac Studio, base $1,999 (March 2025) $2,499 (June 2026)
Mac Studio, M3 Ultra $3,999 (March 2025) $5,299 (June 2026)
Nvidia DGX Spark $3,999 (October 2025) $4,699 (February 2026)

Tim Cook called it a “hundred-year flood” when Apple raised Mac prices in June. Raspberry Pi says the LPDDR4 on its boards went up seven-fold in a year. That one matters more than the Mac, and I’ll come back to it.

The Box Next to the Router

Here’s the world I think these boxes point to. Every house has a small, quiet machine on the shelf beside the Wi-Fi router. It holds the family’s mail, photos, documents, and calendar, and it runs a model good enough to answer questions about all of it. Nothing leaves the house. There’s no per-token bill, and no status page to refresh when a provider has a bad afternoon.

The worry is already there. In Pew’s February 2026 survey, roughly seven in ten Americans said AI will make their personal information less secure. Apple is selling the new Mac Studio on exactly that: its launch copy says you can “run massive models entirely on device with complete privacy.” The biggest home assistant went the other way. In March 2025 Amazon removed the “Do Not Send Voice Recordings” option from several Echo devices, because its generative Alexa features “rely on the processing power of Amazon’s secure cloud.”

Power is a practical problem. A Netgear Orbi router idles at 7.4W. ServeTheHome measured a DGX Spark idling at 40 to 45W and drawing 60 to 90W during LLM inference. That’s fine on a developer’s desk. It’s a harder sell for something that runs all day in a hallway closet.

Price is the bigger one. Nobody puts a $4,699 box next to their router. The home version has to cost about what a good router costs, so it can’t depend on a big GPU or a giant pool of premium unified memory. It has to run on ordinary hardware. That sounds out of reach today. I don’t think it is.

Memory Bandwidth Sets the Speed Limit

Generating text on a local model is mostly a memory problem. For every token, the machine reads the model’s active weights out of memory, so the ceiling on speed is roughly how fast it can move bytes, divided by how many bytes each token needs.

tokens/sec ceiling  ~  memory bandwidth / bytes read per token

dense 70B, 4-bit            ~40 GB per token
  Strix Halo @ 256 GB/s     ~6 tok/s ceiling       measured: 5.0

MoE, 3B active, 4-bit       ~2 GB per token
  Strix Halo @ 256 GB/s     ~130 tok/s ceiling     measured: 72.0

Apple’s ML team showed this cleanly when it tested the M5. Memory bandwidth went from 120GB/s on the M4 to 153GB/s, a 28% bump, and token generation got 19% to 27% faster. Time to first token is compute-bound, and it improved 3.3x to 4x.

Here’s where the hardware lands.

Hardware Memory bandwidth
Desktop, dual-channel DDR5-5600 ~90GB/s on paper
AMD Ryzen AI Max+ 395 256GB/s
Nvidia GB10 273GB/s
Apple M4 Max 410 to 546GB/s
Apple M5 Max 614GB/s
Apple M3 Ultra 819GB/s
Apple M5 Ultra 1.2TB/s

An ordinary desktop gets you about a third of a Spark, and the top Mac Studio is more than four times past the Spark. On bandwidth alone, cheap hardware loses badly.

Mixture of Experts Changed the Math

A mixture-of-experts (MoE) model splits its weights into many small expert networks and only runs a few of them per token. Qwen3.6-35B-A3B has 35B parameters in total and activates 3B. Total parameters decide how much memory you need. Active parameters decide how fast it runs. That split is the whole case for cheap hardware, because ordinary DDR5 is slow but you can put a lot of it in an ordinary machine.

Hardware Dense model Speed MoE model Speed
Nvidia DGX Spark Llama 3.1 70B, FP8 2.7 tok/s gpt-oss-120b, MXFP4 58.7 tok/s
AMD Strix Halo Llama 3 70B fine-tune, Q4_K_M 5.0 tok/s Qwen3-30B-A3B, Q4 72.0 tok/s
Laptop, dual-channel DDR5-5600, CPU only Qwen2.5-Coder 32B 3.5 tok/s Qwen3-Coder-Next 80B-A3B 7.7 tok/s
Raspberry Pi 5, 16GB Qwen3-30B-A3B, 2.7 bits per weight 8.0 tok/s

These numbers come from different people using different tools, so don’t read them too precisely. The gaps are too big to be noise, though: the same Spark runs a 117B MoE model more than 20 times faster than a dense 70B.

The laptop row is the one I care about. It’s a CPU with ordinary RAM, and an 80B MoE model runs twice as fast on it as a dense 32B. The person who posted those numbers says the MoE result is still 3 to 4 times slower than the bandwidth math predicts, so the software has room left. Then there’s the Pi: a $305 board running a 30B model at 8 tokens a second, at a quantization that keeps about 94% of full-precision quality.

The software is closing that gap quickly. llama.cpp added --cpu-moe in August 2025, which keeps the expert weights in system RAM and puts the rest on whatever GPU you have. Running Qwen3.6-35B-A3B that way is one line:

llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M -ngl 99 --cpu-moe

One user runs a community fine-tune of that model this way on a Ryzen 7 5800X with a 6GB GTX 1660 Super, and gets almost 16 tokens a second. That’s a gaming PC from 2020.

Multi-token prediction landed in llama.cpp in May 2026. On an RTX 3090, Qwen3.6-27B went from 23 to 42 tokens a second from a software update. Apple’s M4-to-M5 jump bought 19% to 27%.

The models keep shrinking for the same capability, too. The Densing Law paper estimates the capability density of LLMs doubles roughly every three months. Epoch AI found a single RTX 5090 runs models that matched the frontier 6 to 12 months earlier, and it notes its method is biased against sparse models, so MoE likely shortens that lag.

What Still Doesn’t Work on Cheap Hardware

Reading the prompt. Generation is bound by bandwidth, but prompt processing is bound by compute, and CPUs are slow at it. On a 48-core EPYC with 12 channels of DDR5, CPU only, gpt-oss-120b processes prompts at about 109 tokens a second. A 32K-token prompt means at least five minutes before the first word comes back. The Spark’s GPU does the same model at about 2,400. For a chat, that’s fine. For “summarize my whole inbox,” a CPU box has to do the reading ahead of time, overnight, while nobody’s waiting. For something that sits next to the router all day, that’s a reasonable design.

New architectures. On that same EPYC, Qwen3-Next-80B-A3B, which uses a newer hybrid attention design, ran at 11.8 tokens a second. Qwen3-30B-A3B, with about the same active size, ran at 63.1. The software catches up, but the cheapest hardware tends to get the fast path last.

Quality. The Pi result keeps 94% of full-precision quality, which also means it gives up 6%. For a house assistant answering questions about the calendar, I’d take that trade. For hard reasoning, I wouldn’t.

Memory prices. The cheap-hardware argument depends on plenty of ordinary RAM, and the LPDDR4 on that Pi is up seven-fold in a year. The Pi 5 16GB launched at $120 in January 2025. It’s $305 now.

My bet is that the home AI box ends up being a cheap machine with a lot of ordinary memory, running a MoE model and doing its heavy reading while the house sleeps. The $4,000 developer boxes are proving the demand and paying for the software work that gets there. The silicon exists. The models are getting there on a three-month doubling. What’s holding it back in 2026 is the price of DRAM, and that’s a supply problem.

When supply catches up, I expect the box next to the router to cost about what the router did.

The 200-Day Certificate Is Already Here

Most of the coverage of shrinking certificate lifetimes is written as a warning about 2029. That framing is five months out of date. The first step already happened: since March 15, 2026, publicly trusted TLS certificates have been capped at 200 days. If your renewal process still assumes an annual cadence, it is already wrong.

The Schedule

CA/Browser Forum ballot SC-081v3 passed in April 2025 and set out a staged reduction rather than a single cliff:

before 2026-03-15    398 days
from   2026-03-15    200 days     <- in force now
from   2027-03-15    100 days
from   2029-03-15     47 days

Domain validation data reuse shrinks alongside it, from 398 days down to 10 days for SAN validation by the 2029 milestone. That second number gets less attention and matters more for anyone with a manual approval step, because it means the validation itself stops being something you do once a year and file away.

The values are deliberately awkward. 200, 100, and 47 are not round, and the reasoning is that a number you cannot map onto "twice a year" or "once a quarter" discourages anyone from building a manual calendar reminder around it. Whether that works as intended, it does communicate the intent clearly enough.

What Actually Breaks

ACME-managed web servers are fine. If nginx or Apache is fronted by certbot or an equivalent, the cadence change is invisible. The breakage is concentrated in the places nobody automated, and in my experience it is the same short list every time.

Appliances with no ACME client. Load balancer management interfaces, older firewalls, IPMI and out-of-band controllers, storage arrays, VPN concentrators. Many of these accept a certificate only through a web form. Some accept only a specific bundle format. At 398 days that was an annual annoyance somebody remembered. At 100 days it is quarterly, and at 47 days it is not survivable by hand.

Non-HTTPS services. Mail is the big one, because a mail server presents a certificate on SMTP, submission, IMAP, and POP3, and renewing the web certificate does nothing for any of them unless something copies the file and reloads the daemon. The reload is the step people forget. A renewed certificate sitting on disk while the running process holds the old one in memory is an outage waiting for the expiry date.

Pinned certificates and manually distributed trust. Anything where a partner has your specific certificate configured on their end now needs that coordination three to seven times a year instead of once.

What to Do About It

Inventory first. You cannot automate endpoints you do not know about, and every environment I have looked at has certificates nobody remembered issuing. Scan your own address space by port rather than trusting a spreadsheet.

Automate what can be automated, then deal honestly with what cannot. For the appliance that genuinely has no API, the answer is a documented runbook with an owner and a calendar entry, not an intention. Write down who renews it, how, and what breaks if they do not.

Then monitor the endpoint rather than the issuance. This is the part people skip. Certificate management tools tell you what they issued, and they cannot tell you what a client actually receives, which is where incomplete chains, stale files, and un-reloaded daemons live. A quick manual check of what a server is presenting:

openssl s_client -connect mail.example.com:465 -servername mail.example.com </dev/null 2>/dev/null \
    | openssl x509 -noout -subject -dates

For a one-off look at a public endpoint, including the chain, the SSL check on mrdns.com does the same thing without the command line. For getting the underlying configuration right on whatever you are running, goodtls.com has per-application guides, including the mail servers and databases that tend to be the ones running on expired certificates.

The one thing I would push back on is treating 2029 as the deadline. March 2027 is the milestone that hurts, because 100 days is where quarterly manual renewal stops being merely tedious and starts producing outages. That is seven months away.

Net_DNS2 v2: Breaking a Decade of Backwards Compatibility

I have maintained Net_DNS2 since 2010. It started as a cleanup of the old PEAR Net_DNS library, and for most of its life the guiding rule was simple: do not break anyone. That rule held through PHP 5.3, 5.4, 7.x, and into 8.x. With v2.0 I broke it deliberately, and I think it was overdue.

What PEAR-Era Naming Costs You

Net_DNS2 predates PSR-4, namespaces, and any modern autoloading convention. The class naming reflected that:

$r = new Net_DNS2_Resolver(['nameservers' => ['8.8.8.8']]);
$result = $r->query('example.com', 'MX');

Underscores as a pseudo-namespace worked, and it left the library carrying an autoloader that mapped underscores to directory separators, class names that grew to Net_DNS2_RR_OPENPGPKEY, and no way to use any of the type machinery PHP had spent a decade adding. Every resource record type was a loosely typed bag of public properties. Every enumerated value was an integer or a bare string, validated by convention.

The practical cost was in the bug reports. A meaningful share were people passing the wrong type into something, getting no error, and finding out later when the wire format came out malformed. The library could not tell them, because it had no way to say what it expected.

v2.0 Requires PHP 8.1

The v2.0 rewrite moved to PSR-4 and real namespaces, with the parallel rename you would expect:

$r = new \NetDNS2\Resolver(['nameservers' => ['8.8.8.8']]);
$result = $r->query('example.com', 'MX');

The floor is PHP 8.1, which is what enums require. That version choice is the entire reason the break was worth making. DNS is a protocol built almost entirely out of small enumerated sets: record types, classes, opcodes, response codes, DNSSEC algorithms, digest types. Modelling those as integer constants means every function that accepts one accepts any integer. Modelling them as backed enums means the type system rejects nonsense before a packet is ever assembled.

Most class, method, and property names survived the move. Someone upgrading is mostly changing Net_DNS2_Thing to \NetDNS2\Thing and raising their PHP requirement, which is a mechanical change a search and replace handles. That was the design constraint I set for myself: break the naming, keep the shape.

Deciding to Break Compatibility

The argument against was straightforward. Net_DNS2 has a long tail of users on old PHP, often embedded in something they inherited and do not want to touch. A hard PHP 8.1 floor strands all of them.

What changed my mind was noticing that I was writing new code to work around the absence of types, then writing tests to catch the bugs that the absent types would have caught, then answering issues from people who hit those bugs anyway. The library was subsidising PHP 5 compatibility with permanent complexity, and the people paying that tax were mostly not the people benefiting from it.

The compromise is that v1.x still exists and still gets fixes. That is the part I would recommend to anyone in the same position. A hard break is much easier to justify when the old branch does not disappear the same day, and tagging a final v1 release that people can pin to costs almost nothing.

What I Would Do Differently

I would have done it sooner. The signal was there for years: every feature request that involved better validation ran into the same wall, and I kept routing around it. Deprecation cycles have a cost too, and carrying a compatibility promise you have quietly stopped believing in is worse than announcing the break.

I would also have been more aggressive about the enum conversion in one pass. Doing it incrementally meant a stretch where some values were enums and some were still integers, and the mixed state was more confusing than either endpoint. If you are going to break, break cleanly and finish.

If you use the library and are still on v1, there is no urgency. If you are starting something new on PHP 8.1 or later, start on v2. The DNS lookup tool on mrdns.com runs on this code, so it gets exercised against real-world responses continuously, which has caught more parsing edge cases than my test suite ever did.

Fanning Out 140 DNS Queries per Host Without Melting the Server

A blacklist check is not one DNS query. Checking a single IP against our source list means somewhere north of 140 DNS lookups, and the checker runs continuously across a large number of hosts. The naive implementation of that is a loop calling getaddrinfo(), and it is catastrophically slow for a reason that has nothing to do with CPU.

DNS Is Almost Entirely Waiting

A DNSBL lookup is a small UDP packet out and a small UDP packet back. The work is negligible. What dominates is latency, and the distribution is ugly: most zones answer in tens of milliseconds, a few take hundreds, and some fraction never answer at all and burn your full timeout. Serialized, 140 lookups at an average of 60ms is over eight seconds for one host, and the tail is far worse than the average.

The instinct is a thread per query. That works until it doesn’t. Threads are expensive relative to the work being done, and a thread blocked on a 5 second DNS timeout is a thread doing nothing while holding a stack. Scale that across concurrent hosts and you are context-switching more than resolving.

One Socket, Many Queries in Flight

DNS over UDP is already designed for multiplexing. Every query carries a 16-bit ID, and the response echoes it back. That gives you everything you need to run hundreds of queries over a single socket and demultiplex the answers as they arrive:

//
// send all queries first, then collect
//
for (auto const &source : this->m_sources)
{
    uint16_t id = this->_next_query_id();

    this->m_pending[id] = { source.id, now_ms() + source.timeout_ms };
    this->_send_query(id, source, _host);
}

Then a single epoll_wait() loop reads whatever comes back, looks up the ID, and retires that entry. The whole batch costs one socket and one thread, and wall clock becomes the slowest response rather than the sum of all of them.

Two details make this correct rather than merely fast. The query ID has to be random, not sequential, because a predictable ID plus a known source port is the cache-poisoning setup from 2008. And you have to verify that the response actually matches the question you asked, since the socket is unconnected and will happily hand you a packet from anyone.

Timeouts Are the Hard Part

Nothing tells you a UDP query failed. The absence of a response is the signal, which means every in-flight query needs a deadline and something has to sweep for expired ones. Checking all 140 pending entries on every loop iteration is fine at this scale. At larger scale, keep them in a structure ordered by deadline so the sweep only touches what has actually expired.

int64_t now = now_ms();

for (auto it = this->m_pending.begin(); it != this->m_pending.end(); )
{
    if (it->second.deadline <= now)
    {
        this->_record_timeout(it->second.source_id);
        it = this->m_pending.erase(it);
    } else
    {
        ++it;
    }
}

The timeout value itself is a judgment call with real consequences. Too short and you record false negatives for slow-but-working zones, which quietly degrades your results. Too long and one dead source sets the floor for every check. Per-source timeouts, tuned from that source’s observed behavior, beat one global number.

The Failure Modes That Only Show Up in Production

Source port exhaustion is the first one. If you open a socket per query, you will run out of ephemeral ports under load, and the symptom is intermittent bind() failures that look like nothing else. A small pool of sockets with many queries multiplexed over each avoids it entirely.

Rate limiting is the second. Public DNSBL operators will throttle or block you, and they are right to. Anything running at volume belongs on a paid data feed or a local mirror, and the checker needs per-source concurrency caps so one aggressive batch cannot trip a limit that then poisons results for every subsequent check.

The third one took me longest to appreciate. A source that returns NXDOMAIN and a source that never answers look similar in a naive implementation, and they mean completely different things. The first is a confirmed “not listed”. The second is “no data”, and recording it as “not listed” turns an outage at one provider into a silent gap in your results. Track them as distinct outcomes from the start, because retrofitting that distinction after the fact means every historical row is ambiguous.

Where AI Compute Goes When the Models Stop Getting Better

Almost every AI infrastructure bet I see rests on two assumptions: that the best models keep getting meaningfully better, and that you keep renting them from three or four companies. Both are weaker than they look, and I think they break sooner than the roadmaps imply. Here’s the chain of reasoning, in the order I expect it to play out.

The Frontier Is Getting Crowded From Below

The distance between the best closed model and the best open one is now measured in months. Epoch AI puts the lag at roughly four months, about one point release, and Stanford’s 2026 AI Index shows four of the top ten public Arena models are now open weight, clustered tight on Elo.

The Frontier Is Getting Crowded From Below

The Frontier Is Getting Crowded From Below

Alibaba’s Qwen3.8-Max matches the closed frontier on general knowledge in its own benchmarks, and its open-weight sibling fits on a single GPU.

The frontier is still closed. But “a few percent behind and you can download the weights” is a different market than “only three companies can do this,” and everything below follows from that.

Nobody Needs a Frontier Model to Read a License Plate

Most production AI is narrow: translate a field, pull a total off an invoice, read a license plate, classify a ticket. Aiming a frontier model at those is like renting a rack to run cron. The cheaper purpose-tuned model that clears the bar at a fraction of the cost is the right call, so architectures are going multi-model: route each request to the cheapest model that passes, keep the expensive one for the genuinely hard reasoning.

The clearest signal of where this is headed is that Stripe just agreed to acquire OpenRouter (announced August 19, terms undisclosed, reported north of $7B by Bloomberg and TechCrunch), the router that sits in front of 400+ models and meters the tokens. When a payments company pays billions for the thing that routes and bills model calls, tokens have become a commodity you buy by the unit.

Commodities sold through a broker that shops for the cheapest passing option only get cheaper. If your business model assumes today’s token prices, plan for them falling by an order of magnitude.

The Plateau Comes Sooner Than the Roadmaps Say

Model progress will plateau, and given how fast the curve has moved, it will be sooner than you think. When the frontier stops jumping, the game stops being about the “smartest model” and becomes “same quality for the least money, power, and space,” which is a hardware question.

Today the answer is Nvidia H100 and H200 GPUs, and the challenge is two things: programmability (CUDA, a chip that runs whatever you invent next quarter) and networking (NVLink and InfiniBand lash thousands of GPUs into one fabric). But a frozen architecture is exactly what you burn into silicon, and plenty of companies are already betting on that.

Companies Already Building Fixed-Function AI Silicon

Companies Already Building Fixed-Function AI Silicon

Etched is the purest bet: Sohu hard-codes the transformer into silicon and runs nothing else. Its throughput claims are marketing until someone benchmarks them independently, but the thesis holds. If the transformer is the architecture for the next decade, a chip that does only transformers wins on cost per token by a margin a flexible chip can’t answer.

We’ve seen this arc in crypto mining: CPU, then GPU, then ASIC, each stage wiping out the last. Litecoin’s Scrypt was deliberately designed to be “ASIC-resistant” to keep mining on GPUs. It didn’t work; Bitmain shipped the Antminer L3, a Scrypt ASIC, and the holdouts were finished.

AI is harder than a hash, so this runs slower and messier, and Nvidia’s networking outlasts its CUDA one. But the people betting GPUs stay central forever are standing where the Scrypt holdouts stood. It’s a question of when, and the when is closer than the capex suggests.

What Happens to the Buildings

Meanwhile the industry is pouring concrete for a frontier that keeps advancing and a GPU that stays central.

What's Been Announced (proposed data centers)

What’s Been Announced (proposed data centers)

I don’t believe most of this gets built: Morgan Stanley pegs the data-center financing gap through 2028 at around $1.5T, Bain figures the industry is roughly $800B a year short on the revenue to fund it, and Satya Nadella has already said “there will be an overbuild.” Microsoft has been quietly canceling leases, which TD Cowen read as oversupply.

Now layer on the hardware shift. These campuses sit where power is cheap and land is empty, far from people, which is fine for training because training doesn’t care about latency. Inference does; it wants to sit near users.

So picture the end state: progress plateaus, the workload tilts to efficient inference on purpose-built silicon, and that inference wants a metro. What happens to a two-gigawatt training barn in rural Louisiana, built for GPUs doing a job that moved somewhere else? My honest guess is that some become the abandoned malls of this decade, stranded too far from anyone to repurpose. The ones near grid capacity, fiber, and people convert fine. The rest hold hardware that lost on cost per token, in a place nobody needs.

This is the ordinary shape of an infrastructure boom, and AI keeps working fine right through it. The first wave overbuilds general-purpose capacity chasing a moving frontier; the workload commoditizes and specializes; the hardware goes from flexible to fixed; and the winners are whoever got cost per useful token lowest, not whoever had the biggest model.

We ran this loop in mining, and in the fiber glut before it. Worth positioning for the back half now, while everyone else builds for the front.