Tech Spindle Weekly Newsletter 07.18.2026: Netflix goes deep on LLMs, China drops a 2.8T monster

Harika: Okay, so Netflix decided to host their own LLMs instead of just paying OpenAI or Anthropic for API calls. That's a huge operational lift — you need GPU infrastructure, serving layers, the whole deployment pipeline.

Rohan: Right, but this is what I love — they're not just throwing models on servers. They integrated LLM inference into their existing unified serving system, so it's sitting right next to all their traditional ML models with the same A/B testing and logging infrastructure.

Harika: Which makes total sense at Netflix scale! When you're serving millions of members, API costs add up fast, and you probably want tighter control over latency and model behavior anyway.

Rohan: Plus they've got this cool hybrid setup where small CPU models run in-process but bigger GPU models get delegated to their Model Scoring Service. It's like... architectural efficiency meets practicality, which honestly, chef's kiss.

Harika: Okay, so a Chinese lab just dropped a model that's basically neck-and-neck with GPT-5 and Claude, except it's open weights and costs a fraction of the price. Like, developers who've been locked into the big APIs now have a legitimate alternative that doesn't break the bank.

Rohan: And the architecture is wild! They're running 896 experts but only activating 16 per token — that's insane sparsity. Most MoE models don't go anywhere near that ratio, so they're squeezing way more capability out of way less compute.

Harika: Right, and with a million-token context window and native vision, this isn't some research toy. This is production-ready, and it ships in like ten days.

Rohan: Plus the cache hit pricing is genius — ninety percent of your coding tokens cost thirty cents per million instead of three bucks. That's... honestly kind of a game changer for anyone building agents or long-context apps.

Harika: Yeah, I mean, if the benchmarks hold up in the real world, this is the first time an open model has actually caught up to the frontier. Not 'almost there' — actually there.

Rohan: Exactly. And now every other lab has to explain why their closed model costs ten times more. Good luck with that!

Rohan: Okay this is genuinely cool — someone built a local workspace that runs Claude Code, opencode, and Kimi side-by-side, each in its own git worktree so they don't stomp on each other. It's like finally having a proper IDE for your AI agents instead of juggling terminal windows.

Harika: The practical win here is huge though — you can queue up prompts while one agent is working, fork conversations, and literally undo the files a prompt touched if the AI messes up. No more babysitting one agent at a time or losing your main branch to a bad suggestion.

Rohan: And it's all local! No API keys going to some third-party service, just a Rust binary that spawns the CLIs you already have and translates their output into one UI. The nerd in me loves that they skipped vendor SDKs entirely.

Harika: I mean, if you're already wrangling multiple AI coding tools this is basically a no-brainer — it's open source, MIT licensed, and solves the exact workflow chaos everyone's dealing with. This is going straight into my toolkit.

Rohan: Okay this is actually clever — Brex is basically putting a bouncer at the network layer instead of trying to teach every AI agent the rules. Every time an agent tries to call an API, their proxy intercepts it and uses another LLM to be like 'yeah that's cool' or 'nope, blocked.'

Harika: Right, and the reason this matters is that traditional permission systems just can't keep up when you've got agents spinning up requests to dozens of different APIs. You can't manually configure guardrails for every possible thing an agent might try to do — you need something watching in real time.

Rohan: The LLM-as-a-judge part is wild though — you're using an AI to police an AI. I'm just imagining them having philosophical debates about whether a Slack message is appropriate.

Harika: Ha! But honestly, if this works at Brex's scale, it's a template every company deploying agents is going to need. You can't give agents real credentials and just hope for the best.

Rohan: Okay, this is legitimately wild — they're running encrypted image classification in 200 milliseconds. Like, the data never gets decrypted, the model never sees the actual images, and it's still fast enough to actually use!

Harika: Right, and that's the whole game-changer here. Before this, homomorphic encryption was so slow it was basically academic — now you could actually build a real product where, say, a hospital sends you encrypted medical images and you run AI on them without ever seeing the patient data.

Rohan: The trick is they're using this newer CKKS scheme with a bunch of optimizations — batching operations, approximating activations. It's like, finally someone figured out how to make the math not completely destroy performance.

Harika: Yeah, but let's be real — CIFAR-10 is tiny images, simple model. The question is whether this scales to actual production workloads or if we're still a few years out from that. Still, 200ms is... that's honestly impressive.

Rohan: Oh for sure, it's a proof of concept, but it's the first one where I'm like 'okay, I can see the path from here to there.' That's huge!