podcast

Dex Horthy - What Actually Gets You 2-3x With AI Coding

You can't get 10x with AI coding. Not today. Dex Horthy on context engineering, 2-3x without shipping slop, and why skipping quality burns a codebase into ash.

Listen on

Spotify

Apple Podcasts

YouTube

Jan-Niklas Wortmann

13 Aug 2026 • 78 min read

Episode 21 · August 13, 2026 · Dex Horthy

Key Takeaways

Dex's ceiling for production work is 2-3x, not 10x: 99% of human-quality code, written as if you had typed every character, just faster.
A perfect spec still produces garbage if you give the model permission to skip quality.
PR review is the bottleneck. Spend the human time on design and breakdown so review is 20 minutes, not six hours.
A spec should give leverage so you can re-steer a short markdown file instead of thousands of lines of code.
Use the smartest model until it works, then make it cheaper. Kill a session when the model starts flailing on tests.
Vibe-coding a side project and keeping a 10-year enterprise system alive share almost no constraints.

Chapters

Mentioned In This Episode

Pull Quotes

You can get 99% of human-quality code, like very good code as if you had written every character by hand, but two to three times faster. You can't get 10x. It can't be done. Not today.

I don't give two damns how your spec is shaped. It should give you leverage.

I think abandoning code quality and system quality, giving engineers permission to ship slop, I don't think that's correct. I think that's going to collapse your codebase into ash much faster than you think.

If you're a manager and you are not trying to help your people adopt AI, you are failing them.

A developer vibe coding a side project a dozen people will ever run and a team keeping a 10-year-old enterprise system alive for another quarter share almost no constraints worth naming.

Guest Bio

Dex Horthy is the CEO and co-founder of HumanLayer, an AI context lab helping production software teams ship faster with coding agents without descending into slop. He coined "context engineering" and is known for Research / Plan / Implement workflows, 12-factor agents, and the "dumb zone" of long context. Before HumanLayer he spent years at Replicated shipping Kubernetes products into customer environments.

Full Transcript

Read the full transcript

# Transcript

Episode 21 — Dex Horthy

**Dex** (00:00:00)

I don't give two damns how your spec is shaped. It should give you leverage. Because you can give it a really good architecture doc, and the model can follow it to the letter. Everything you asked for in your system design, Mermaid charts, here's the modules, here's the new endpoints, is exactly as specified. But somehow the code is still garbage, and you can do a lot of steering, and you can get 99% of human-quality code, like very good code as if you had written every character by hand, but two to three times faster. You can't get 10x. It can't be done. Not today.

**Jan-Niklas** (00:00:35)

That's Dex Horthy. He's the founder of Human Layer and godfather of context engineering. So not just prompting models, but deliberately designing what information goes into an agent, when humans should steer it, and how teams can ship with AI without watching their codebase burn to the ground.

**Dex** (00:00:52)

I think abandoning code quality and system quality, giving engineers permission to ship slop, I don't think that's correct. I think that's going to collapse your codebase into ash much faster than you think. I have not seen someone nail the abstraction yet to a point where I would use that instead of hand crafting my own memory so I can control every single token that goes into the context window for every prompt in every agent of my system.

**Jan-Niklas** (00:01:19)

So if you're trying to use AI for real production software, not just site project slop, this one is for you.

**Dex** (00:01:27)

We're out here shipping value and helping users and helping people ship value today while everyone who is like bitter lesson pilled is basically just like yoloing prompts into the best model they can find sitting around waiting for GPT-7 to come out because they're just like oh it's not worth doing anything because the models are just going to get smarter lots of people are out there telling you code is free and build every single thing you can and it's

**Jan-Niklas** (00:01:50)

It's not free.

**Dex** (00:01:51)

It's not because if you don't care about it you will be throwing it out in six months. If you're a manager and you are not trying to help your people adopt AI, you are failing them.

**Jan-Niklas** (00:02:03)

In this episode, we talk about why software engineering is not going away, why code review might actually become more important, why just let the agent run is not a strategy, and how serious teams should think about planning, context windows, subagents, cheaper models, and AI adoption.

**Dex** (00:02:19)

I think the core insight behind context engineering is that Um, the AI, like the space of people building on LLMs There are a lot of buzzwords and concepts and ideas around memory and context graphs and rag and agentic systems and multi-agent and sub-agents and cross-it. There's all these ideas. And everyone wants to sell you a product or sell you an open source thing that kind of abstracts away something. And the idea behind context engineering is like it's not that thick a layer and you should really just understand that you're assembling everything is assembling context windows under the hood and do stuff yourself before you go reach for abstractions because i don't think we've figured i haven't seen a single all the memory companies are really smart people doing really interesting stuff they're getting really good results on benchmarks i have not seen someone nail the abstraction yet to a point where i would use that instead of hand crafting my own memory so i can control every single token that goes into the context window for every prompt in every agent of my system

**Jan-Niklas** (00:03:29)

So you brought up benchmarks and i gotta say this is such a pain point for me because they're all kind of trash to be super honest in general around ai.

**Dex** (00:03:39)

And they get worse over time Well,

**Jan-Niklas** (00:03:42)

Over time, agents, models, whatever, are just gaming the system where it's like, oh, yeah, I have like 5% point more than the last model. I'm so cool. That doesn't mean jack shit at this point.

**Dex** (00:03:53)

Depending on where we get to today, I have two benchmarks that I'm actually interested in that we can dig into, but...

**Jan-Niklas** (00:03:59)

I do think the DeepSWE looks, it at least seems realistic enough with my own experience where I have like a certain sense of trust in that.

**Dex** (00:04:11)

DeepSWE, this is the, I actually haven't gone too deep on DeepSWE. I pulled up SWE-Marathon and FrontierCode right now is the ones I've been digging into, but I have to check out DeepSWE.

**Jan-Niklas** (00:04:22)

So like, if you look at like, what is it called? Terminal-Bench Pro is something that a GPT-5.5 and a GPT-5.4 Mini are like 2 percentage points far from. It's completely delusional. Completely delusional. I don't understand how people are like, oh, yeah, this is a great model. I mean, it is a great model for its purpose. Don't get me wrong. But those benchmarks are completely useless at this point.

**Dex** (00:04:46)

Yeah, people are accidentally training on test. And then you have specifically with Terminal-Bench, you have people straight up cheating at all the problems by like putting secret things in the system prompt. So it's even worse than we thought.

**Jan-Niklas** (00:04:58)

It's so bad. Okay, sorry, I got off on a rant. So context engineering.

**Dex** (00:05:03)

Yes.

**Jan-Niklas** (00:05:04)

I have a somewhat maybe provocative question, but do you think, as I fully agree, it is very relevant right now that you intentionally steal the context and be very mindful of what you put in, what you expect to get out of it, etc. Do you think this will be a thing in the somewhat near future? Do you think LLMs and agents are just getting good enough to do this themselves entirely autonomously.

**Dex** (00:05:27)

They keep getting better. Can I share two slides? This is from a talk I gave at AI Engineer Miami. But basically it's like you have some task and you have the default model out of the box has some ability with naive prompting. If you just YOLO the prompts in, there's some set of tasks and it can do those tasks at some quality. And then you can do a little bit of context engineering for the tasks you care about and make it a little bit better at those tasks. Right. Right. This is the, and then the bitter lesson basically says, uh, or bitter lesson or whatever you want to call it. Um, um, Is that eventually a new model will come along. And so you'll have this and then a new model will come and it will just blow all your, it will make half of your work irrelevant. But then you can immediately turn around and do more context engineering. And like the goal is like, hey, we have to make these models solve these tasks. I mean, this is engineering at its core. It's like, how do we get smart about boundaries and testing and evals and all this stuff so that we can take the off the shelf thing and make it perform better than what somebody, and we spend weeks and or months of time making the off-the-shelf model perform better than what you can get just with naive prompting, and people are willing to pay for that. And so if you're building products, you're building systems, you're solving problems for yourself, I think it will always be relevant. I mean, there may be a world where GPT-9 is so smart that it's 100% on every benchmark we as humans can ever come up with, but I don't see the trajectory going there anytime soon and I think there's a lot of interesting problems to solve and a lot of value to be created in investing and making these models better and in pushing the frontier on the tasks that we care about and I think anyone seriously building AI and building AI for enterprise is doing this and thinking in this way You

**Jan-Niklas** (00:07:14)

Touched on something and I just want to be 100% clear because again, I'm terminally online and therefore you easily get into this narrative oh software engineering is completely solved, we don't basically need software engineers or you have like i i don't know who mediocre CEO it was it was like talking about 100x engineers.

**Dex** (00:07:34)

I remember who it was but i won't i won't i won't say who it was it was probably more than one anyways

**Jan-Niklas** (00:07:41)

Do you think software engineering as a craft is going to disappear?

**Dex** (00:07:48)

Nope. Not anytime soon.

**Jan-Niklas** (00:07:51)

I do want to preface this because I also didn't expect it a year ago to be where we are right now. So the speed of change this way is tremendous. At the same time, software engineering has never about writing code from my perspective. I i was never the best coder i i'm i'm good at writing software don't get me wrong but um, the skills that may be valuable to the companies i worked at were usually more soft skills like i can facilitate i can coordinate work i have like the bigger picture i can architect systems like that is what companies value in my skills not that i can crank out 100 words a minute or whatever.

**Dex** (00:08:34)

Or you can nail a bubble sort algorithm at first pass or whatever it is.

**Jan-Niklas** (00:08:39)

I'm completely useless with lead code. I was very fortunate. So I grew up in Germany where this whiteboard test is not as much of a thing. So I was always like, I would fail every one of these.

**Dex** (00:08:53)

Must be nice.

**Jan-Niklas** (00:08:54)

It's not bad, not going to lie.

**Dex** (00:08:56)

Yeah, there's a couple layers to this in terms of, like, is software engineering dead? And I think about this in terms of, like, the new software factory and the old software factory, which was before we had AI and before we had the notion of lights off. We still had software factories. If you go back to, like, 2021, and this isn't even when the term came out, but, like, the Department of Defense, there's this guy, Nicolas Chaillan, the, like, chief software officer for the Air Force. And he said, he came in, he said, we need to start building software like every other enterprise. And, like, not the, like, big banks of the world, but he's looking at, like, what we call, like, these, like, hipster enterprise, right? Your Airbnbs, your Ubers, your Instacarts, whatever it is. And they have really high security. They have really high throughput. But they have incredibly high quality. They deal in crazy systems. They ship hundreds of times a day. And they're using this whole stack of tools of Jenkins and code scanning and security and like all of these things that like, there's like a hundred tools that touch every single release that goes out. And it's like, we need to do this so that we can move faster. Whatever the reasoning is, but like this was the concept of Software Factory before AI. So AI is just a new layer on top of it. And I'm going down a little bit of a tangent here. So bring me back to what the question was, and we could go deeper on that, maybe.

**Jan-Niklas** (00:10:17)

Well, actually, now you got me down there, so I'm right there with you.

**Dex** (00:10:21)

Yeah, okay, let's go.

**Jan-Niklas** (00:10:22)

So in Germany, we have this concept of dual studies program. So I studied something that is kind of like a mix between economics and computer science. So nothing really. I know jack shit, but like a little bit of everything. And there we went into production. And I think, honestly, like looking into like how a Toyota or something structures their production sites and factories is super interesting. And software engineering as a craft can learn so much from that. As a whole.

**Dex** (00:10:48)

I remember being like a year or two into my software career and I was in my boss's office for a one-on-one, um, incredible manager. And, uh, he, uh, I was like, we have this inefficiency, like, you know, it just, it was something with like the QA process and we kept having regressions and like, and he's like, okay, Dex, there's a thing that we're going to have to learn right now. I'm going to give you a book to read. It's called the goal by Eli Goldratt. And the big takeaway is there will be inefficiencies that are not bottlenecks. And if the inefficiencies are not blocking the main through line, then you just have to hold your nose and be okay with them. And the mistake we're making right now with the loops and the token maxing is like, we're all I forget who said this first. It was on Twitter, like a couple months ago, it was like, we are doing the primary mistake of like what factories were doing in the 60s and 70s, which is they would bring in these MBAs. And the NBA's job was to pick like one station in the factory of the hundred different stations to make a part. And they would optimize the hell out of it and make sure it was always at full utilization. We're trying to saturate utilization of the key parts of the factory rather than focusing on the end to end and fixing the bottlenecks.

**Jan-Niklas** (00:11:58)

I think this is interesting because on the one hand, I very much think it is a good thing that right now we're still a little bit in like this infinite pot of gold for where companies just like throw money at AI. hoping to see like that 10x return on investment. I highly doubt there is a 10x return on investment, but different conversation. Where I'm going with this is I think this like exploration phase, like getting the engineers into the state of AI psychosis and out of that as quickly as possible is super important.

**Dex** (00:12:27)

Super important. You have thoughts. You force people into AI psychosis or like you encourage them to go crazy with it. And then you encourage them to kind of pull back.

**Jan-Niklas** (00:12:38)

I think it's a natural, like in software engineering, we tend to do this thing where we overcorrect. On the last episode of this podcast, I was talking to David Cramer and we were also talking about like microservices, like 2016, 17, 18. Like everyone was like, oh, we need microservices for everything. Figured out like, oh, microservices are maybe not the best way. Uh maybe some parts where we want scalability or something should be microsoft etc so we have this like swinging and as an industry we keep doing the same stupid thing where it's like oh ai all the things oh maybe, uh or uh who was it now, like one company was basically like, uh yeah it was Uber who spent their entire ai budget in like the first three four months.

**Dex** (00:13:20)

Yeah a million dollars lots that's like every company i talked to who got ai pilled in December they built their budget in September before Opus 4.5, so they had no idea that suddenly every engineer was gonna be addicted to this fucking crack cocaine that is Claude Code, basically.

**Jan-Niklas** (00:13:35)

That is such an interesting word. I never thought about it.

**Dex** (00:13:39)

That's not my, I read that on Twitter too, but I think that's 100% correct.

**Jan-Niklas** (00:13:43)

You need to have that exposure to figure out this is what it's good at because there are things that AI is fantastic at and I don't want to lose that. But there are also parts where AI is absolutely trash for.

**Dex** (00:13:52)

Yep. And I don't know, my current evolving thesis on all of this is basically like some people are, there's like levels to this. There's the lights off factory where you say, cool we're just going to write specs this was the strong dm thing that came out in like january or february other companies have posted about this of like we're going to stop reading the code it's the bottleneck we're just going to throw more tokens at the problem and like yes throwing more tokens at the problem does give you better results but it does not guarantee to give you correct results and i think there's like level one is you care about the specs and you're like okay i define the behavior really really well and i define like okay if if it is working these things will be true and hope that the model will find a way to test and assert those things are true, right? And modern models are pretty smart about this. You give it a browser, you give it a bash shell. It can do a lot of poking from the outside and make sure it does what you want. The next level is like architecture. And a lot of people are like down this path now. People who used to be full vibe coders never read the code are now like, well, I understand the system and the layout and the architecture. This is like Peter Steinberger talks about this a lot as well. It's just like, well, I don't read every line of code, but I know how the systems work and I understand kind of the interfaces between them. And I think a lot of people there, I think what we have learned in the last six months working with customers and working internally is that's like actually not enough because you can give it a really good architecture doc and the model can follow it to the letter. Everything you asked for in your system design Mermaid charts, here's the modules, here's the new endpoints is exactly as specified, but somehow the code is still garbage. Somehow there's still like leaky abstractions, tramp data everywhere. And like, I'll get into why I think this is, but like the next step that we need to do is actually get into like program design, like not system design, but actually like, what are the interfaces? What are the test seams? Where are we doing dependency injection? Like how do these things, this is especially crazy on the front end, which is weird because as a backend engineer, my whole career, I always thought that what front end engineers, it's actually one of the most complex things. Models got really good at writing api endpoints and crud systems and even like really complicated algorithms and write-ahead logs and sophisticated storage mechanisms they still can't do reason about React very well

**Jan-Niklas** (00:16:08)

That cracks me up every time when i see like oh yeah front-end development is cooked and i'm like i have never gotten good results with front-end development and part of that is probably also the way i work uh with llms but we will touch on that in a bit i, depending on At least it's looking like that. So I'm always wondering when I see these things on Twitter and something like a Peter Steinberger saying, I'm not looking at code anymore, whatever, if that is just a glimpse in the future because he's, a little bit ahead of the curve in that sense, or if that is just like his way of working and we're more drifting into establishing different ways of working the same way, like a couple of years back, people were like discussing TDD versus TDD. Not TDD, right? Like, are we more talking about like different approaches for different problems, different constraints, whatever? Or is this really, is this where the industry is heading? From my perspective, I think it's more different workflows. I think there are, if we're talking about spectrum development, for instance, I think they're very valid scenarios and very characters where this works really well. I'm personally not such a character because I'm not as structured in my approach. I'm more of like a fuck around and find out person. Yes. Yes. Very small chunks. That's why I was always also skeptical of like, oh, yeah, we wrote the C compiler with a two-week running long-term agent. I'm like, I don't know. I don't know. I don't know how I feel about this.

**Dex** (00:17:41)

Well, compilers themselves is like the one thing that is very well suited to hands-off running agents. like lights off, you know, just let it go. This is Ralph Wiggum. Like a year ago, you say, I didn't know. Uh, I didn't know a year ago that we would be here. I got a peek actually, uh, a year ago tomorrow is, uh, is the day that I met Geoff Huntley in San Francisco. And he showed us all the like early demo of Ralph Wiggum and the programming language he created for $5,000 by running Sonnet in a loop for six weeks. Uh, and it works in and in all of this. And it's just like, so program managers themselves are actually like incredibly verifiable. So it's very easy for the model to just like go through a checklist. But yeah, I think the zooming out again, like I think the thing to think about the most is leverage. Our job is to ship software end to end that is good, which means shipping code that's going to pass review. It means, in my opinion, reviewing the code. You can throw more tokens at the problem and, you know, BugBot and Codex reviewing Claude's code or vice versa. A you will find some things and you will raise the floor but like you're not gonna get if you're shipping really high quality production code and you're not reading it and caring about it or you are reading it and caring about it but you haven't like cultivated a deep intuition about like what and i don't want to gatekeep here but the bottom line is like you know a bad pattern because you debugged it at three in the morning this is jake from netflix and a couple other people and so it's like if you don't have the sense of what good looks like it's really really hard to build systems that are going to last.

**Jan-Niklas** (00:19:15)

Okay, there's so many different paths right now where I kind of want to go down with you. Let's talk about that review piece a little bit. Because I look at any kind of automation that we can integrate in a review, like even if it's just like automating tests and stuff, those are signals that tell me a certain amount of, like a bigger picture piece about the quality. Are those fully extensive signals? Absolutely not, right? Like code can be garbage even though the tests pass. And vice versa, the code can be great even though the tests fail. So and this is like looking at the most deterministic tests can also be flaky but fundamentally, you know what i mean um and the same way i also think if you have this spending budget on saying hey let's run Codex on every pr to get like that signal get like a first summary of the review what the code quality is absolutely would i treat this as a strict quality gate these need to be fix everything this needs to be included in everything this is this is the, the base or i treat it more as a baseline, or like a starting point for a human to look at oh okay this might be something we should explore more something.

**Dex** (00:20:23)

And it's a little blurry it is like yeah you shouldn't spend really valuable human hours on catching like small issues like if Codex can catch it let Codex catch it and fix it for sure

**Jan-Niklas** (00:20:33)

I see these stories and maybe this is just the environments that I worked with, but I've seen these stories where people are like, oh, I'm now just reviewing code where I'm like, well, if you're just spending hours reviewing code, you're doing something wrong. That's my thesis. There are absolutely difficult pieces of code where you need to carve out like 30 minutes or something to digest it properly, grab yourself a cup of coffee and zone everything out. But if you need that much time to review a PR, I don't know what level of review you're doing I've reviewed PRs with hundreds of files and never spend I don't, my wife is working for a medium sized company and they, are so distracted by PR reviews where they're effectively saying we don't do them. And I'm like, A, if you're distracted by them, you're not understanding why they're valuable. B, the code quality aspect is nice. I'm still, my biggest aspect of PR reviews is more of like the knowledge transfer, but again, different conversation. But I don't understand this notion of like, oh, there's not so much code, we cannot review it anymore. Like, what are you doing? Like, are you, it's not a novel.

**Dex** (00:21:49)

Like yeah and this is Dax from OpenCode has said this and i forget who else notable said it recently but it's basically like there is a ton of things you could build and the fact that you can just like prompt a feature into existence and like they basically my take is like lots of people are out there telling you code is free and build every single thing you can and it's

**Jan-Niklas** (00:22:13)

It's not free.

**Dex** (00:22:13)

It's not because if you don't care about it you will be throwing it out in six months i Like I said, maybe GPT-7 or GPT-9 will be able to solve the current model's problems, but if you want to have a functioning company, if you're 0-1 and you're trying to find stuff that works and you're throwing shit against the wall, you're basically in prototyping mode, amazing. Don't read the code. If you're a fintech with hundreds of engineers and you get fined millions of dollars if something is done incorrectly, you have to read the code. You literally like it is existential to your company, bordering, it's borderline existential to your company to make sure that things are correct. And bugs still happen and humans make mistakes too, but they don't make the types of mistakes that models make.

**Jan-Niklas** (00:22:54)

And this is probably the thing that bothers me the most in this online conversation because it is more nuanced and Twitter is not great for nuance. It's actually horrible. But there are scenarios where wipe coding is completely valid and I would probably even, I do it myself, I would encourage.

**Dex** (00:23:08)

I do it all the time.

**Jan-Niklas** (00:23:09)

If I write a tool myself for my little, I want to track how well this podcast is doing, why should I pay a company if I have some clock tokens floating around, right?

**Dex** (00:23:20)

Did you watch for a launch video?

**Jan-Niklas** (00:23:22)

The laundry was fantastic.

**Dex** (00:23:25)

So every single animation in there, all of the motion graphics, vibe-coded, like Opus 4.7, it's like 2,000 line React files. I have not read a single line of them, completely vibe-coded.

**Jan-Niklas** (00:23:38)

But that is also something that is easily verifiable. You put it in the video, figure out, oh, yeah, it's great, and you're good to go.

**Dex** (00:23:45)

Yep.

**Jan-Niklas** (00:23:47)

If we have software that affects users, maybe even on a very regular basis, like daily, you should absolutely care about what you're shipping to them.

**Dex** (00:23:55)

Yeah. And so here, I can, can we do some drawing? Yeah. Hell yeah. Okay.

**Jan-Niklas** (00:24:02)

So this will also be an audio version, so I'll try my best to describe it very accurately.

**Dex** (00:24:09)

I can put the end picture somewhere. Oh, sweet. Yeah, let's do that. And we can post it with the show notes or something. Yes. Um, but like the like classic, like software factory SDLC is you have actually, let me get a classic, uh, SDLC. There's like, it's like the infinity sign with like the eight nodes or whatever. Right. All right. We can just copy this one. Uh, so you have, yeah. Planning analysis, design, implementation, testing, maintenance, some, some, some, this is where we were like 20 years ago. No one, no one does this. I hope you have like ticket ticket. This is like, you know, what are we going to build? I'll call it like PRDs, right? And then at some point you have like doing the code and then you have, you know, code review and maybe you have, I'll put code review and like testing in one block. It's just like, make sure it's correct. And then it goes to prod,

**Jan-Niklas** (00:25:07)

Right? Sounds reasonable. Yeah.

**Dex** (00:25:10)

And then customers use it.

**Jan-Niklas** (00:25:13)

Hopefully.

**Dex** (00:25:15)

Hopefully. Hopefully you have users. And then by some process, this makes it back into your team, their feedback and whatever it is. And you make new PRDs or bug reports or whatever it is, right? But like requests for changes in the product, right? And this is how a lot of teams go. And then as you get a little bit bigger, you kind of introduce two stages here. Most teams start doing this pretty early. We have kind of like a design meeting, not like visual design, but I mean, like, how are we going to build this? Usually this happens like you have design meeting and then you have like break it down into tickets

**Jan-Niklas** (00:25:47)

Yep yep yep yep yep.

**Dex** (00:25:49)

And then those are what go into the coding and so like we've all been in here uh sit in a room full of four or five software engineers you look at the things we want to build this week you say cool how are we gonna you align on how we're gonna do it and then you break it down into small stories and then everyone just goes and gets to work you create the tickets out the backlog um What a lot of and so like this is this is the old way the things that change here for the AI software factory is basically like you have, you know, some tech in here, which is like sandbox orchestration. Uh, you maybe have like a, what we call like an outer harness, which is like giving it like feedback and testing and a browser and stuff like this. Um, and then you have your like inner harness, which is, you know, Claude Code, Codex, app server, whatever it is. Uh, any of the model. And this is going to like push things into your code review and testing loop. Maybe you have like manual human testing if no one's reading the code i hope someone's at least trying it and seeing if it works before we merge it uh you may have automated testing i'm gonna assume you're gonna do as much automated testing as possible here and so you're gonna catch all the things that automated testing could catch so you might have manual human testing you probably have like ai code review bots Um, and the question is, is like, can we skip the like actual, like important thing of like, you know, human code review? Uh, and if you just throw AI in the places we have it so far, essentially what happens is you get, does it make sense so far?

**Jan-Niklas** (00:27:20)

Yes, yes, yes, yes, absolutely. Yeah.

**Dex** (00:27:22)

So you can throw AI in here and then this thing is like, cool, you're going to spend all, literally this is the only human step. So you're going to spend six hours a day reviewing code, basically. Or let's say you're going to spend, it's a giant PR with six features or whatever it is, because the AI is on goal mode and it's shipping the entire thing. And you could go read the code for six hours. I don't think anyone can do a good job of reviewing code for six hours straight. And I don't think we should ask software engineers to do that. And I don't think anyone wants this. Uh and so like the thing that we've seen working in a lot of really good teams is actually like using ai plus human to do design and do like breakdown breakdown and ordering right this is this is kind of our bread and butter is like hey if you do this really well and you spend 20 minutes here or maybe let's say we spend an hour here to do the design to break it down into steps where it's going to be verifiable along the way uh maybe you even have a human for really complex things you have like a human like spot checking in between because you've ordered things correctly then your code review goes from six hours to like 20 minutes because like the prs that take a long time to review are the ones that are bad a perfect pr or a pr that is like okay i gotta change some variable names and maybe move things across files it's easy it's almost a joy to review uh especially if you already went through this process and you understand it.

**Jan-Niklas** (00:28:50)

So that brings me kind of to a somewhat related question, because I've read that at Human Layer, you'd kind of change the way you're developing by focusing more on having a very extensive spec. And you might, I think you have a, you call it a little bit different. It's not quite a spec. So we can talk about where you differentiate there, but you have a very extensive, research and plan that is then getting reviewed before it's, going into the coding part of things. Yeah. Is that a fair summary? Okay.

**Dex** (00:29:22)

Fair enough. I mean, I actually, I think the word spec is pretty, like, what is the word? Yeah, it's just, it means too many things to too many different people. And for some people, it's like writing a detailed ticket. And for some people, it's writing 30 Markdown files with, like, 80 architectural decision records. And, like, everything is numbered. You have acceptance criteria and all this stuff. And like, I don't, the, the goal of all of this, I don't give, I don't give two damns how your spec is shaped. It should give you leverage. It should let you read a 200 line markdown file and re-steer rather than having to read 2000 lines of code and re-steer later where it's like more work for you to build it, load it into context, more work for the model to debug or make changes. Cause it's already kind of committed down one path. And so we actually have, yeah, these two phases where you have like, okay, what is the overall system architecture? And then what is the program? If you care a lot about program design, what is the program design going to look like? You want to basically give the model every opportunity to give you a zoomed out version of the code so that you can resteer that. Because the more detailed the thing is, the harder it is to resteer. And so the more, if you can do a pass at the 50,000 foot view and at the 25,000 foot view and at the 10,000 foot view, you save yourself a lot of time at the live view on the ground reviewing the code itself.

**Jan-Niklas** (00:30:47)

No, I appreciate that. How do you make sure that... Because, I mean, like, effectively, models are not deterministic. Agents don't make this slightly better by having tools to make it a little bit more deterministic. But fundamentally, still, building on non-deterministic software doesn't help with making it deterministic. So where I'm going with this, you can have the same prompt, perform it twice, and based on the time of the day, you will get different results. Yeah. how how do you make sure then that you have a plan whatever you want to call it in place that is detailed enough to prevent that drift, but also not over detail where you're basically like exceeding the context window with just one markdown file that is 5,000.

**Dex** (00:31:32)

Lines long right and this is the i forget there's some haskell person posted this and um Mario Zechner the pi agent guy, he talks about this all the time. It's like, people say, oh, but I have a very detailed spec. And it's like, if you are at a level of detail that you are guaranteeing that every line of code is written in a particular way, you haven't written a spec. You've written code. A program is a detailed spec.

**Jan-Niklas** (00:31:55)

So the last episode of this podcast went out with Mario and we talked about spec-driven development and enterprise-wide coding. So, still very fresh in my mind. Yeah.

**Dex** (00:32:07)

So again, We're looking for leverage, right? I coach people, actually, the first time people start spectrum development, they kind of, like, have it in their head. It's like, oh, if I just get this perfect, then I won't have to recode ever again. I won't have to do this. It will just, the model will just handle it. We're all looking for, like, an easy button or a way out of, like, thinking about the systems and designing the code and, like, cultivating taste. And it's right. It's like the friction in the building is where you learn, and people try to avoid friction. And so for me, I see people try to get, they spend an hour getting a spec perfect. And I'm like, no, spend, spend 10 minutes, get it like 80% of the way there, get it close enough that when you zoom into the next level, it is easy to re-steer if you left things out or if you forgot things or you want to change things. And so that's from the system design down to the program design down to the actual code. It's all about how do I make this directionally correct enough so that when I get down into the weeds, the likelihood that I'm making big changes is really small and that I can quickly recover in one session.

**Jan-Niklas** (00:33:11)

So I assume these markdown files that you prepare as part of that are getting checked into the code as well as part of the project or what you're working on or how you're handling this.

**Dex** (00:33:21)

The thing we landed on like a year ago was that these shouldn't be checked into the code because they they work separately because we don't maintain specs in the sense of like there's this there's this school of spec driven development where people say like, cool, we maintain a set of specs that describes the program state. And then we change the specs. And as we change the code, we update the specs to match. I am on a GitHub issue thread somehow i'm subscribed to that is a year old that is people on the spec kit repo like complaining that this is a really keeping these things in sync is really really hard

**Jan-Niklas** (00:33:57)

And and that's where i was going like we have so nuanced bugs sometimes that are like part of maybe several specs and like how do you reflect that properly that's kind of where i was going with this oh.

**Dex** (00:34:07)

Yeah but okay and you have bugs that have nothing to do with the specification of the software they're about the internals and so how do you capture that in a spec no well you have to document the program design but if you try to document it's the same thing it's like you try to document in comments and in function names and you have developer docs of how does this module work and it's it's out of date immediately and so we actually treat all of our specs we we do store them but we store them the original version was it was a symlinked repo into your current repo that every time a file was written we would sync it to a separate GitHub repo so just like no commits no merge it was technically a git repo but we were treating git basically like google docs or like s3 where every time someone makes a change you push a new version uh and so they were accessible and you could pull them in and you could link them in but they're really like tactical docs for the tasks that i'm on and when i'm done shipping whatever feature or bug fix i'm shipping they kind of get archived and like we rarely go pull them back out

**Jan-Niklas** (00:35:03)

Okay that makes a lot more sense at least thinking about the way that i work and i mean we talked about this where I'm like more explorative where it's like, oh, Let's start here and then find my way around the software, right? That makes a lot more sense because then you can, like, start with a good enough plan, as you described it, and take it from there. Whereas, like, otherwise, if you check it in, then it feels like, to some extent, more final, approved. This is what's going to happen. Yeah.

**Dex** (00:35:32)

Now I'm committed to this. And if I get halfway down the implementation, I'm like, oh, that's not going to work because we forgot about this thing. Like, I can just finish. If it's close enough, I can just fix it and re-steer because we're going to throw that dock out. But if we change and I own that dock and other people are going to be using it, now I have to change my code and then I have to remember to go update the plan. And you're actually just giving yourself more work. Yes.

**Jan-Niklas** (00:35:56)

And that's the way that I, with this traditional approach of spectrum development, how I always felt about it. Because effectively, again, what the software developers most of the time are doing is finding out what they need to build to some extent. We have a good enough ticket that we know vaguely what we need to build, but how it exactly looks, what exactly are the constraints. I've never yet seen a ticket that has all the level or all the details that I needed to know to just be like, oh yeah, let me throw this at GPT-5.5. It will take care of this. Completely delusional from my perspective. But like, you need to have enough details, some kind of acceptance criteria that you as an engineer are confident, okay, I can build this. Once you have that level of confidence, You'll figure out the roadblocks along the way. That's at least how I've been working. And this approach never really clicked with like a traditional spectrum approach. So I always looked at it more as like, okay, what are people doing here? Why are they thinking this is good? But I like your approach a lot more. It seems kind of like a middle ground in that sense.

**Dex** (00:36:58)

Yeah. You got to be comfortable rewinding. You got to be comfortable throwing stuff out. You got to become same thing with code. I don't know. I used to do this thing when I was a software engineer writing code all day. It's crazy to think of this. If I was writing a PR and I didn't like the design, I did all this work. I spent two hours on this. But it's like, I tried to cultivate the habit of like, look, the hard part is actually understanding. I'm going to get reset, start over and build the feature from scratch again. And it would always be better code. It would take a little bit longer, but I'd be so much happier with it once you've gone through and figured out what all the constraints are. And so like you have to leave space for surprises and you have to basically like the spec is not about getting it perfect. It's about optimizing like optimizing your chance of success and the amount of changes you have to do while not getting too attached so that you can be flexible and squishy and kind of like re-steer as you go, as you learn things.

**Jan-Niklas** (00:38:01)

I think this attachment to code is one thing that is a little bit weird to me. And I think also where a lot of this anti-AI sentiment is coming from. I absolutely enjoyed coding as an activity and stuff. It's fun. It's a challenge in some way. So it's like very stimulating in that sense. But at the same time, the overall goal is to, is always to, I mean, like this traditional Silicon Valley thing, make lives better, right? Like whatever it is, like at the end of the day, you want to build something that is used by people in some way and makes them more productive, makes it something easier for them, gives them some kind of value in that sense. And code is just a means for that value. I mean, like in the same, I like DIY woodworking, right? if I could, code in that sense it's just the same thing as like if i build a bird house and my daughters love that easy win.

**Dex** (00:38:58)

I could buy hands sometimes it's it's great but i do it on on saturday and it's like a side project where i'm like i just want to play with interfaces and do do the do the red green tdd loop by hand and like i don't know i so i read a book when i was 22 i'm sure you've heard of it is uh Bob Martin Clean Code uh and i had been using at that point i've been using JetBrains IDEs for like a year and I had just got started to get really handy with like a lot of the more advanced refactoring tools and I don't know I'm not gonna pull up the quote there's like extract function rename variable like inline method of like you weren't really thinking at the level of the individual characters anymore you were like moving things around and you were like restructuring the the program design and and Bob wrote this quote he was using re-sharper when he wrote this but it's like visual studio whatever uh and he's just like this is the best tool it is like it knows exactly what you want and everyone is saying for 40 years that we're going to have drag and drop things and code is dead and people software engineers are just going to use like WYSIWYG things to design their programs and like we're kind of getting there but it's not a serious thing that anyone uses for serious work uh But uh I I this spoke to me so much of just like the the the the joy and the flow state of like molding the clay at like a higher level um as i did not really answer your question but yeah this idea of like there is a there's there's a lot of like joy to writing the things by hand and designing the system

**Jan-Niklas** (00:40:30)

Absolutely um at the same time i think there is also okay do you think, it is reasonable at this point for a company to require ai or mandate ai usage for writing software.

**Dex** (00:40:47)

I think from a like radical candor perspective, if you love your employees and you want them to grow and be prospective, sorry, if you love your employees and you want them to grow and be successful in this new world and they really, really don't want to do AI, you should find a way to get them to learn. It's like, it's like the, the engineer who's stuck in text edit and they're really good engineer. And you're like, look, I know you don't want to learn Vim key bindings, or I know you don't want to like fire up this clunky IDE every time you start to work. But I promise you, you will, if you, if you stick with it and you do the like two to three works of two to three weeks of pain, and you commit to learning more about this every week, you will be, you will have superpowers and you will thank me. And it's it's hard to do but i think it's a man if you're a manager and you are not trying to help your people adopt ai you are failing them

**Jan-Niklas** (00:41:45)

I i'm a little bit torn because if i if i take like a couple steps away then ai is just a tool right if you can guarantee me the same level of performance quality etc without that tool at the end of the day i don't care, at this point someone to make that statement i think that is a very bold statement to say i can deliver the same quality in the same time without using AI. Yes.

**Dex** (00:42:08)

But I mean, this is a really good point about, because even if it's using AI, well, which AI, which system, which spectrum development kit, which skills, which coding agent, which harness, which IDE, all of these things. And it's very easy to become a very qualitative discussion. And I think like, if you want to drive meaningful AI adoption in your org, you kind of need like, you need two things. You need a metric and you need like social proof. You need to be able to point to a team over there that is like, hey, those people over there, we all agree, they're shipping like crazy and they're not descending into slop. They're not having more bugs. Their project is on track. Everyone thinks in general the code quality is good and they're crushing the metric because then you can go around. You have some hypothesis or you have six. We're going to have this team over here do Cursor and this and this and this. And metrics are hard. We can't measure engineering productivity. We've been trying for 50 years.

**Jan-Niklas** (00:43:02)

Okay, that's where I wanted to go with this, because every engineering metric that I've seen around productivity, I'm like, that's garbage, that's garbage, that's garbage.

**Dex** (00:43:11)

You don't like weighted PRs per dollar token spend as a metric? No, it's really hard. I mean, I think you could pick a metric that's directionally correct, where it's like, okay, yes, you can cheat this and game this. But if that person over there has double this number than that person over there, I would reasonably assume they're shipping more and they're more productive.

**Jan-Niklas** (00:43:34)

That's true. That's fair. That's a fair point.

**Dex** (00:43:36)

But then you get to have the conversation with everyone on your team, right? Once you have that social proof and you have a metric that is like at least defensible, then you get to go and like talk to other teams and you say, look, I don't care how you do it. I don't care what tools you use. I'm not telling you to use an IDE. I'm not even telling you to use AI, but you have to hit that level. They have proven it can be done while maintaining a high quality bar and they have proven they can double your numbers if you want to go if you figure out you can deliver more shareholder value by running three gas towns in the corner and throwing more polecats at the problem great but you have to get there and we're going to help everybody get to that bar or get close to it

**Jan-Niklas** (00:44:18)

So one thing that I see all the time is that level of usage with AI is very different. If we look at total token consumption within companies, the top 10 spenders have 90% of token consumption and the rest of the company usually share the rest of that. Do you think... I mean, part of that is we're now starting in this phase where there is an AI budget that is strict. Do you think... Because a lot of these workflows like Gas Town, Loops, etc. Are based on the premise that AI tokens are infinite and companies don't care how much you spend on that. How do you see this going to change?

**Dex** (00:45:01)

I mean, my thesis is basically like the best thing you can do today if you need to maintain high quality and you're shipping to production, all the things we've talked about is you can probably get to like two to three X for most types of work. And 10 X or 100 X is like reserved for very specific, very verifiable domains. Where you know like the the the the bun rewrite is like okay cool they already had you know tens of thousands if not hundreds of thousands of unit tests very verifiable if it's working or not And i i think two to three x is great and i think people are i've already talked to teams who measure if you know dx that company that got bought by atlassian they they do a like they have a kind of proprietary productivity metric and you can just look at that metric or you can look at it divided by dollars spent on tokens and i think that's going to become an interesting thing and like again i hate metrics but at the end of the day like the things that motivate engineers are like building beautiful code and probably getting paid and so if like if the company has agreed that oh this metric is directly correct and if you do well at it you are going to get promoted you are going to get a raise whatever it is like you as a you as a leader have to figure out how do we create pull towards new ways of working um and i think probably like using tokens efficiently using tokens as many tokens as you can is a great way to get to push people into ai psychosis and get them completely obsessed with it and then pull them back out and it's like cool now figure out how to do this cheaply or we'll call it efficiently more efficiently

**Jan-Niklas** (00:46:36)

We're slowly now starting to look at like the ri and like the obviously infinite tokens are nowhere near a reasonable RRI because I can use infinite tokens. I can have 15 goal Codex loops running, doing jack shit, right? But having like this, Okay, on that, do you think being more selective with the model is going to be one way that developers are going to tweak that? Because if I look at my workflow, I use usually the most frontier model. Fable was fantastic.

**Dex** (00:47:14)

I don't have time to fix Sonnet's work. It's faster and better use of my time to just use the smart one, right?

**Jan-Niklas** (00:47:20)

That's exactly. I was trying to give it the benefit of it the other day. I used Sonnet for something and I was like, I had to multiple times do it where I'm like, you're factually wrong here. And not even like just off a little bit, you're literally wrong. And that happens with Opus too, but like proportionally, nowhere near that. So I usually just, I don't even mess dramatically with reasoning levels, if I be honest with myself. I set it to high or extra high and call it a day. It's slower, but I usually do other things while the AI is running anyway, because it helps my adhd to keep going so give me one last answer what is the one mistake companies are making right now with ai's usage biggest mistake.

**Dex** (00:48:03)

I think abandoning code quality and system quality giving engineers permission to ship slop i don't think that's correct i think that's going to i think that's going to uh collapse your codebase into ash much faster than you think

**Jan-Niklas** (00:48:17)

We actually had to stop the first recording right there, which was super annoying because we had just gotten into the practical part. When should you spend money on the frontier model? When does it make sense to use something cheaper? And what does planning even mean once agents are doing more of the implementation? So a few days later, Dex came back and we picked it up exactly there. We had to cut it off last time a little bit short, so I'm glad we are able to continue it here. And we were talking about cheaper models and that it's kind of tedious to fix the mistakes that they make and instead usually, at least both of us, we're kind of resorting to just using a bigger, more frontier model. But you also said there are a couple nuances to that where you would resort to using something like a solid model. So let's continue there.

**Dex** (00:49:06)

So this comes down to, I think, a concept that I've been shipping back and forth with developers for a decade now which is make it run, make it right, make it fast and maybe make it cheap you know I think designers taught me this The idea of, like, prototyping and getting so, like, I don't know, we talked about this on our podcast a lot of, like, before you go try to optimize, because there's some techniques you could do. This was a year ago, so the landscape was different, but at the time, it was, like, 01 or 03 had just come out. It was a really smart, beefy reasoning model, but it was flow and it was expensive. And the advice basically came down to, like, cool, get it working with the smartest, best model you can. Figure out what the frontier can do. And then as you need to, if your volume goes way up or you need to optimize for latency or price or whatever it is, then go try to make it work on GPT-4o. Then go try to make it work on 4o Mini or whatever it is. Does that make sense?

**Jan-Niklas** (00:50:11)

Yes. And like the Fable release, that was the first time where I was, actually the first time was probably Opus 4.7. I think they changed the tokenizer where they said like oh the model is technically at the same price but it was consuming 30 more tokens so that was the first time where i started to be really like, okay do i really get more bang for this for the buck here or not Fable then kind of elevated that where it was like okay it's twice as expensive as Opus now do i really always need to spend the big dollars, or but like most of the time i still think like and for most developers i talk to it's like okay using something like Opus GPT-5.6, um is for most developers i know at the way they work at least like the average developer they get fine with the budgets that they have in that sense i'm not talking about like running crazy loops parallelizing things like crazy crazy. But right now, I don't see this anywhere near. At the same time, though, tokens are getting so much more expensive that I, at the same time, also think companies will eventually put in restrictions like that.

**Dex** (00:51:24)

Oh, I've talked to enterprise customers who have gotten guidance from leadership of like, okay, we're going to use Opus for planning, but then I want you to switch to Sonnet for implementation.

**Jan-Niklas** (00:51:35)

Is that something where you see negligible quality difference, if that makes sense?

**Dex** (00:51:43)

I think it depends. I mean, I think, number one, there's a huge question here of like, are you on a subscription or are you paying per token?

**Jan-Niklas** (00:51:53)

Fair enough.

**Dex** (00:51:55)

And for the subscription people i will say if you were maxing out your subscription on Claude and you're not using Fable obviously Fable ate through it really fast but if you're just using Opus for everything and you're maxing out your subscription like you're probably doing too much um and again there's two categories right there's the vibe coders and then there's the people building production software and like i vibe code all the time but there's a time and a place for it and yes you can vibe code the entire universe of software if you get good at paralyzing but i don't think it's actually like productive in terms of like creating value in the world

**Jan-Niklas** (00:52:33)

I that's how i like to separate i think there's like a huge area where web coding is fun even also productive like writing an internal tool or something that just you and a couple colleagues are using completely viable to just vibe code it not look at it.

**Dex** (00:52:46)

As long as we call this a couple times call this fight slop with slop you vibe code the tools that help you like improve the code quality of your actual code

**Jan-Niklas** (00:52:56)

I i do that a lot for the for partners here for like post-processing and stuff like as long as the result looks good i don't really care for the code quality as long as it works i'm the only one using it so totally cool but, I'm always wondering, okay, because the plan is for me the part where I want to like involve more of my personal thinking. So I'm always wondering, okay, and that's kind of where my original question was coming from. Am I getting a better, or do you see getting better results of using an, expensive model for planning and a cheap model for the execution, where my thinking would more be the vice versa having like, a cheap model kind of act more as like my sparing partner to like encourage me think through a couple edge cases and stuff use that for planning and as long as i green light the plan i would want an expensive model to cover the edge cases properly.

**Dex** (00:53:53)

It depends. What you mean by plan, I think. I think there are...

**Jan-Niklas** (00:53:59)

That's a great question because I already remember from last time that I think we had a slight different differentiation or like way of looking at plan. And a lot of the things that you're doing at Human Layer, also you're constantly describing this concept of plan or questions.

**Dex** (00:54:15)

Research, design, structure, plan, implement, work tree, whatever. I don't, yeah, there's... I think I'm done with acronyms. The point is is you're building a prompt like you're building a pipeline of prompts and you're being in the loop like a plan is just a prompt and so your prompt can be every single line of code that should be changed which feels like overkill or your prompt can be the two sentences you typed in in the first place and there's this whole spectrum between i rambled about a thing i want and a highly like detailed technical doc and this like process of refining various levels of detail in those you go from two sentences to a page you go from a page to a like you know essay the three-page essay you go from three pages to like a detailed outline it's just like writing anything right

**Jan-Niklas** (00:55:05)

I think for me the sweet spot is usually where i feel i have a good enough understanding where i don't hit too many edge case or like uncover too many edge cases but also have, enough understanding that i can share with an agent or whatever in form of verification steps that these edge cases could be uncovered while the agent is.

**Dex** (00:55:26)

Running yeah i think i think it's not about getting a perfect plan that perfectly documents your intent that's kind of like I don't know what the analogy is, but it's like, you don't need to get it perfect. You need to get it good enough that you'll be able to address any deviations without having to do a lot of manual work to rebuild context or start a new session or whatever it is. It's like you build this intuition for how much can I do in a context window? How much can I do in a context window where I'm using sub agents? And then you say, cool, the plan has to be good enough that if it's 20% wrong, I know I'll be able to recover, but if it's 50% wrong, I'm probably, it's probably going to be easier for me to throw it all out and fix the plan. And if the, if the plan is so it'll come back and I realized the plan was actually 50% wrong or 80% wrong, then it's like, okay, cool. I'm also going to throw out the plan and I'm going to rewind even further. And so it's like, you're constantly getting, the more comfortable you can get with zooming in and out of these levels of abstraction the more comfortable you can get with okay this is close enough that i'll be able to recover and there's only a 10 chance that i'll have to rewind it's like all these like stacking of probabilities in your head of like okay i don't this is why i think like have you heard this thing that like people who play starcraft are really good startup founders or like rts games Is because you're basically taking a bunch of incomplete information. There's fog of war. Matt Pocock talks about the fog of war and the frontier. There's stuff that you don't know. You only know what you've seen. And so there's like a 30% chance this is going to happen and a 10% chance this is going to happen. And how do I get more information at some level of abstraction to help me recalculate the probabilities of how the thing works today and how it's going to be built when I build it? And then you're like, I don't know. I think this is what people talk about, LLM intuition. It's like being able to, without thinking about it, naturally hold all these probabilities in your head and then combine them into what's the best path.

**Jan-Niklas** (00:57:28)

One thing that I've seen more and more being talked about on social media is that people interact with different models differently. So I think Theo is talking about that a lot, that he's using Opus fundamentally different than GPT-5.6, more on a prompt level than anything else. And I have this level of intuition where... I experiment with these tools way too much so that I say like, okay, I get better results in this area by using that model and better results by using this model. That is the level of intuition that I have. I don't think, because usually I share my thinking with the model in a way, in the prompt, I don't think I drastically, change the shape of the prompt based on the model. Is that something you're doing?

**Dex** (00:58:11)

I think Calvin French-Owen, who is one of the people on the Codex launch, and now he's doing like some EIR thing but he was the found he was the founder of Segment like he's been building shit for a decade plus uh a decade and a half probably and he his his take was like the at the highest level is like GPT is more literal Opus is more expansive GPT will take your instructions and do them Opus is a little more likely to be like okay you said this but based on what i know and based on everything i've seen you probably want this and this whereas GPT will like just do the thing. So as far as prompting them differently, it's like the energy and the conversation I think is different. And we actually had to update a bunch of our prompts. So we primarily support Claude Code, but we have Codex now and like it's now in GA, but we've had to adapt our system because Claude naturally writes docs in the way that we want them. And Codex in our research phase, it would literally just give you a bulleted list of 200 individual bullets of like this happens in this file on this line, this happens in this file on this line. It was like, wasn't readable for a human, which is if you think of these as, Oh, I'm building the prompt for the next session. And sometimes I want to be in the loop and iterating. And sometimes I don't even read it. Then, uh, then yeah you see you see what i'm saying we have to update all our prompts to make Codex write like Opus basically

**Jan-Niklas** (00:59:30)

I totally get what you're saying because i had this huge moment of revelation with GPT-5 when it came out first where it was like, very to the point whereas Opus or like Claude at that time in general felt more like flowery and like describing so and still to this day for like creative writing i like Claude models Anthropic models in general so much more they are just way more natural to read, but like for my German way of communicating GPT is just like exactly what i need.

**Dex** (01:00:01)

GPT is very German i haven't said this before and i haven't heard it before but you're you're absolutely right

**Jan-Niklas** (01:00:08)

I was immediately like oh i i feel like talking to one of my people here. It's fantastic. You touched on managing context windows a little bit a second ago with different techniques of how much you can fit into a context window, when do you want to use subagents. So I have huge reservations with 1 million token context window. I don't think it dramatically improved anything. I still have the same reservation of going somewhere across the 60% a very imaginary bar but it's for me a bar so i usually do this thing where i get near this have like created have the agent create a summary of some kind, usually in a markdown format and then either start start a new, agent entirely or before if i imagine okay this is a bigger scope uh spawn have it spawn separate separate sub agents how do you feel about this whole thing of okay one million, context window and the techniques that we have in place right now to manage that. Plus compaction. How do you feel about compaction?

**Dex** (01:01:17)

So we used to say the dumb zone was like 40% of your context window. For a lot of models, that was like 80 to 100K tokens. I think it has gotten better, but I still, you know, we build a product that tries to help you do better context engineering. And if you're using a 200K token like model context window, then we will give a warning around 100K tokens. Like, hey, start thinking about, you're not, it's not going to break and become really dumb, But like start thinking about wrapping it up. And there's certain scenarios where I will happily go to 300K for the million context window, by the way, our warning is at 200K. So for slightly smarter models, better models, we know they have more data to train on long context. So it's getting a little bit better. We still warn you around 200K is like, OK, you're getting into a place where like your mileage may vary. I have regularly gone 300 plus, but that's like your intuition and your decisions and based on what's happening. The biggest tell for me when it's like, all right, kill it now and just go do a new context window is if the model is struggling to get tests to pass. If it's like, oh, I got to do this. Okay, let me try doing this. Oh, I got to do an end because that's what it's going to start trying weird, crazy stuff. It doesn't have as much intelligence going to forget things that have happened before. It might even forget how it got the test running. 100,000 tokens ago. So depending on what you're doing, also compaction has gotten a lot better. So your idea of like, hey, I write to a markdown file and then I use that to resume my next session. Compaction can now well capture most of that. And I know the Claude compaction now also points it to the conversation history. So the model can go grep through the old one if it needs to. But you're not in the loop. The nice thing about the markdown file you can go edit it and I'm what do you think is missing when you do like compaction I guess my question is like you keep some information but what do you lose

**Jan-Niklas** (01:03:15)

It is a huge black box to me. That is the thing that is difficult for me to, deal with because at the point of compaction, I lose the information, okay, what information are now in the context window or not. Till the point of compaction, it's pretty much like I assume, it's also not completely realistic, but I assume everything of that is in the context window and will be considered.

**Dex** (01:03:36)

Well, so if you build a custom wrapper, a custom client, you do get a user message event out that has what was in the compaction. It's fair. So if you want to know what was in your compaction, you can always go look in the JSONL file and you can see the user message that it created. And that's how I know that compaction is getting better because now it has summary of what's happened and that it actually includes verbatim every user message that has been sent in the entire conversation, which is why now in compaction, if you give it some instructions pre-compaction and then it compacts, the chances a year ago that it was going to bring those instructions through verbatim was very, very low. And now it's going to see every single time you re-steered it or said, no, run the test like this, that all gets preserved now, which is, I think, a big improvement.

**Jan-Niklas** (01:04:23)

I think we also talked about that last time briefly. How do you feel about sub-agents? I think we talked more about like using sub-agents for different roles, which sounds fancier than it actually is. But I think we didn't really talk about the aspect of like being able to utilize different context windows.

**Dex** (01:04:40)

I think of sub-agents, I think the best metaphor I heard is like sub-agents are map reduce. If you have something that is paralyzable and usually like read only where I like, I want to go read a million tokens of codebase context and then turn it into, combine it into, you know, 50,000 tokens of codebase context. Sub-agents are amazing. And then the other use case that I think they're really good for is we've been exploring what we call like an RLM mode. So we built our own custom harness and we just added one more depth of sub-agent. Claude Code, you can now configure this too, where you can allow sub-agents to call sub-agents. I think the main reason why for the last year you've only had two layers you had main agent and sub agents and that was it is mostly because uh it got very like the results just like couldn't be guaranteed to be good uh but i think adding one more layer is actually interesting where we actually have a or even just two layer rlm where the parent model can call sub agents but you prompt it to do everything through sub agents because you know the worst i'm i'm optimizing our cicd right now we have a pipeline the tests are slowly creeping up as they do i've got up to like seven minutes to run all the tests for the repo and this is a great use case for sub agents because you're like okay you have to go read a bunch of code you have to make a change you have to commit it you have to push it you have to kick off a ci run in GitHub Actions and then you have to sit in a loop and pull it and watch it and then all of that can be isolated to a context window and come back up to the main model and you basically i prompt the main model you are only allowed to use the agent tool you're not allowed to read any files you're not allowed to do any it's a little bit slower but you can get better results if you're we use this for debugging too if we're going to be reading like CloudWatch logs and Datadog data and Sentry traces and like user data like this is all we like we find this works much better with sub agents if there's if if the work is context intensive

**Jan-Niklas** (01:06:35)

Do you think these, But workflows that are based on exposure and experience are going fade or just disappear because models are getting smarter that they can figure it out themselves.

**Dex** (01:06:50)

Listen, man. Okay. So here's like the biggest thing that I keep saying over and over again is like, there is a future where this is solved. Like, I'm want to focus on like, like all of this might get bitter lesson. You might just get a model with actual, like, infinite context, and you don't have to think about it, and, like, compaction is invisible and instantaneous and high fidelity. This might all get bitter-lessened. But, like, we're out here shipping value and helping users and helping people ship value today, while everyone who is, like, bitter-lessened-pilled is basically just, like, YOLOing prompts into the best model they can find sitting around waiting for GPT-7 to come out. Because they're just like, oh, it's not worth doing anything because the models are just going to get smarter. And no, the thing that gives you an edge as a product engineer is how do you get the model to be 20% better at solving a particular task? How do you find the thing that is, we said this a year ago, how do you find the thing that's right at the boundary of what the model can do and get it right consistently every single time? This is what made Notebook LM great. This is actually a, um, a, uh, a quote from one of the, the, the notebook alum guys that did late in space like a year ago. There's like the way you create magical experiences in AI is you make it better than what the YOLO prompter can do. And this is where all of the value unlock. This is where all of the interesting things you can do. And so like, yes, there is a future in two years or five years where this is all solved. And if you want to be a part of that future and contribute to it you have to fucking excuse me you have to go down a level and understand how this stuff works and build the intuition so that you can like push the frontier yourself

**Jan-Niklas** (01:08:32)

So I'll cut this out but on cursing Apple always marks my podcast as explicit because I'm cursing. Before this there will be an episode with David Cramer I will, It won't be a lot too quick. Oh, it's a fantastic episode, but I think a couple people at JetBrains will be a little salty at me.

**Dex** (01:08:52)

Well, now the gauntlet has been cast. I'm going to have to catch up to David. I don't know how much more time we have, but we can certainly turn up the expletives.

**Jan-Niklas** (01:09:01)

One thing I'm wondering is, because right now it feels pretty much like if we're looking at the big providers, right? With Anthropic, OpenAI, Google a little bit behind, and then xAI are probably a little bit more behind depending on how you look at things. But do you think there is a future where these difference will be negligible and kind of like right now, where honestly, like I prefer using a Mac, but I would also use like a framework computer or a good Dell machine. I would probably put Linux on it, but different conversation. So like these difference are getting negligible at some point. Do you think we'll look at these model providers in the near future where these differences are getting less relevant and companies are just, oh, I got a good deal with OpenAI because I know one of their salespeople. So our company is using OpenAI, whereas you switch jobs, you start to work for another company, and they're caring maybe a little bit more about security, and therefore they have their Claude deal or something. Do you think that is something? Because right now what I see, most companies I talk to have several subscription parallels, and you can use most of that, which you like, which I think is important right now because we're still in this exploration phase.

**Dex** (01:10:13)

Everybody's got a $200 plan, right? And you just pick which one you like. And if they're all the same, then yeah, pick the one that your buddy works at, right? Yeah, I don't know.

**Jan-Niklas** (01:10:24)

Yeah.

**Dex** (01:10:26)

It would be interesting world. I mean, it doesn't seem like we're getting on that trajectory. The models are all good, but they're good in different ways. I would imagine that with the amount of relevance to national security that these models pose, it's unlikely that too much is going to get leaked and that people like labs will be able to maintain their moat because they have to keep everything super locked down either way. Uh so i don't know if i would like bet money on a world where all the models are exactly the same we'll see everyone says this is the year where open source models are going to catch up glm 5.2 is incredible it would be nice to see an open model win or at least be one of the options um because i think that gives people the ability and the freedom to open the box and play with it and change it and repair it and like all the all this stuff that tinkerers love to do and that like Like if I don't take an ethical stance on this, but like it is it is in a world where like, yes, you can go see the model and mess with it and change it and retrain it and all this stuff in a world where you can. Obviously, I want the world where I have that option because it could be interesting. It could be fun. It could yield different dimensions of kind of like wins as far as like what you can what you can do from a product perspective.

**Jan-Niklas** (01:11:37)

I think this is also for so many aspects, super important, even if it's just like commodity to keep these big players also to some extent in check. At the same time, the convenience that you have by not hosting your own model is so big that most companies are not going to do this any time near, I assume at least, unless you have like crazy security concerns, which even like, I still talk to a lot of like German customers and even those at OpenAI or Anthropic, whatever. So I want, as you said, like for various reasons, I want open source models to succeed, be competitive in that space. But I mean, also every year since I'm in the industry, I've heard, oh, this is the year of Linux for desktop. It was every year and still it's nowhere near as popular as Mac OS or something, but different conversation.

**Dex** (01:12:29)

Yeah, I did Linux on the desktop a while ago. I was just riffing with somebody about window managers yesterday of like, hey, look, like actually it's a little bit better now than it was. Like with AI, it's like, cool, Claude can fix my Tmux config, Claude can debug my Wi-Fi thing. I don't have to go spend all day reading Stack Overflow once every two weeks because some random update broke my webcam or something.

**Jan-Niklas** (01:12:49)

Getting Bluetooth and graphic card drivers working on Linux, at least the last time that I really messed with it, which was just at the beginning of the pandemic, was still utter trash.

**Dex** (01:12:59)

Yep, I believe it. Yeah, I saw Jeff posted there at some hacker house in Mexico doing a Nix hackathon and someone's like, oh yeah, I vibe coded a Chromecast plugin to cast my Ghostty tab to the TV from Linux. And it's like, no one would have, like, that would have been your only project and you would have been the sole maintainer and you would have spent 20 plus hours a week on that just to be the one guy who figured out how to Chromecast Ghostty to a smart TV from Linux.

**Jan-Niklas** (01:13:28)

Coming back to this workflow of running 10 agents in parallel, doing like several side projects at the same time is, and to be precise, like Peter Steinberger from OpenClaw is very actively talking about that and not reading code a whole lot anymore. And I'm wondering how much of that is just a science experiment or like a look in the future, because they're very different constraints than an open claw has than what... An enterprise codebase deals with.

**Dex** (01:14:01)

Can I read a post from Addy Osmani? You probably saw it. I'm just going to read one quote from it that I really liked. It was from the Loop Engineering article. Here we can see this, agentic code review. So, A developer vibe coding a side project a dozen people will ever run and a team keeping a 10-year-old enterprise system alive for another quarter share almost no constraints worth naming. And most of the advice in circulation is really just one of those two people telling the other how to live that's that's my take is there's two very different worlds and like vibe coding is great if you want to vibe code 100 side projects that's fine but don't pretend you know what it's like to maintain a million line Kotlin codebase for a bank and if you maintain a million line Kotlin codebase for a bank and you have to be very careful and specific and you can't just yolo coat that's great but also like there's no reason to throw shade and shout at people who are vibe coding 100 projects or think you're freaking smarter than everybody because you tell people like there's this like i don't know even my content i think sometimes comes off as a little bit like gatekeeping and i'm trying to like like shave the edges off of that because it's like i am focused on a very specific group of people which who produce incredible value in the world which is like software developers in companies building tools that are used by people that do tens or hundreds or hundreds of millions in revenue or billions in revenue, and The stakes are too high to risk things going wrong. And how in that world with that constraint, can you still move two to three times faster with AI?

**Jan-Niklas** (01:15:41)

That leads me to a very interesting question. What do you think has the highest, net benefit in terms of like AI workflow.

**Dex** (01:15:49)

Like like products like end-to-end product throughput for like commercial applications and production that need to last is your question yes it comes down to one word man it's leverage it's kind of what we were talking about of like you start with two sentences and go to a page and get the page right and then you get the three pages right and then you get the 10 pages right and then you probably go ship it or you break it down into pieces you check it along the way but It's like, how do you, I think the problem is when you're back and forth with an AI model, it's really hard to, you're kind of like, you're being synchronous for things that don't need to be asynchronous. And then you're, you know what I mean? It's like, the model's really good at going and reading. Like, you should send an agent off to spend 10 minutes reading code and thinking about the problem. And then you should come back and riff with it. And then you should disappear for 10 minutes and let it go, like build the first part of it. And so it's like, okay, how do you pull yourself? How do you design your workflows? So humans are pulled into the parts where their time is the most valuable. And if you're optimizing for that and you really think about it and you get really good into LLM intuition, you can carve off enough time that you can do a little bit of paralyzing and you can do a lot of steering and you can get 99% of human quality code, like very good code as if you had written every character by hand but two to three times faster you can't get 10x it can't be done not today

**Jan-Niklas** (01:17:17)

We talked about that i, don't see this 10x or 100x narrative anywhere realistic and that is just CEOs being delusional or gaming a system.

**Dex** (01:17:28)

Or burning their codebase to the ground and they're just they're not going to find out for three months

**Jan-Niklas** (01:17:32)

Yeah most engineers that i know that work in like more traditional as you said like banks or something they have like very dedicated systems and they often lack like the time and resources to learn these things really. From my perspective, exposure is the best way to kind of say, okay, David in the last episode said that he's not writing a single line of code right now himself. He always uses an LM like that, that kind of forces him to get this intuition. What do you think? How can people build that intuition?

**Dex** (01:18:05)

You just got to use the thing all day and you got to watch other people use the thing. I think pairing with AI, like pair programming is actually really underrated right now. Uh, or it's, it's gotten a lot more powerful, um, for a number of reasons. Like we're all learning new ways of working. Like, I mean, look at JetBrains, for example, like I got a lot better at JetBrains when I sat with a senior engineer and watched them use all the shortcuts and all the refactoring tools and all of this stuff. It's, it's, it's, it's a thing where it's like a large complex tool. You can learn it by searching the web. You can learn it by sitting there, but most of times, like you don't know what questions to ask. There are unknown unknowns. And if you take 10 people who are all learning on their own trajectories and you have them mix and match around, you're going to spread that learning and everyone's going to move faster, faster. And so like, even though like, there's a couple of different ways you could work with, you could have one person working on one thing at a time and another person working on another thing. You could have, if that person gets really good, you can do two or three things in parallel, right? One person working two or three things in parallel, another person working two or three things in parallel. What I actually love is two people doing two things in parallel. Because you have enough work that you're mostly doing the work. Because when one thing is blocked, you jump to the other thing. And then when there's more downtime, you don't go check Twitter or check email or multitask or start more work. You sit there next in person with the person you're talking to and you engage with the problem. You whiteboard. You look ahead. You think about what's coming next. You like discuss. And I think that's just powerful on its own. Layer onto that the fact that you're both going to learn each other's ai tricks as you go like oh check out this prompt that i do i think i think that's the the answer is you have to try it and you have to try things and Simon Willison is always saying like every now and then you should try a thing that ai cannot do you know it you've tried it 10 times try it again when a new model comes out have your like personal eval that like helps you understand how much better is this model than the last time we tried this three months ago. So that's part of it. But if you can make a way for those things to merge and people to share all their versions of that while doing actual work, then you accelerate it. But the short answer is if it's just you, just use it as much as you can and pay attention and be thoughtful about what's working and what's not and build that intuition.

**Jan-Niklas** (01:20:30)

The line to vibe coding and not reading the thing is very blurry right now because it can already do so much and you get very easily distracted of like, oh yeah, this seems to work fine. I don't need to bother about it. But I think as long as you're in this exploration stage and even afterwards, the degree just changes. But in that exploration stage, I would check, obviously every line of code, but also every text that is output, every line of thinking, any line of reasoning, anything that is going on to build that intuition and understanding. This is how the model operates. This is how... This is what I would have expected from the other model that I used last week to act here and build that understanding. It is so easy to be like, oh, this works. That's probably also the biggest concern that I have right now for people coming into the industry, being like, I learned software development by going through the motion, or as you said, pairing with a senior who taught me the tricks and I adapted them.

**Dex** (01:21:32)

One one more thing of like how do you learn software engineering is you read the code of really good software engineers yes when i was when i was stuck on a problem and i had to chew on it i would we used all these google libraries from like the guava library is this java library from google it's like really good collection and stuff and so when i was like stuck on a problem and i didn't want to think about it i would just click into the methods in the libraries and i would read oh here's how google writes their code oh here's how meta writes their code like reading other people's really good code. And there's not really a version of that for AI. You can't just go read Simon Willison's Claude traces.

**Jan-Niklas** (01:22:06)

My concern is even more that from a junior perspective, the AI takes that role of the senior engineer kind of and like shape. But like realistically, if you don't steer AI properly, it just puts out garbage. So the only thing that you're creating is like a multiplier of garbage.

**Dex** (01:22:25)

Yes. Slop cannons, we call them.

**Jan-Niklas** (01:22:27)

And that's what I'm really concerned about coming into the industry right now. Just to be clear, I'm not saying software engineering is going away. I don't think this at all. I'd rather think we're just going through a phase where companies are making questionable decisions. I think probably software engineering is super valuable right now. But how this coming into the industry how do we build now how do we ramp up juniors to build these necessary skills i have absolutely no idea and i would love to figure this out.

**Dex** (01:22:57)

Yeah i don't have an answer for you um Our current take in hiring is we would much rather hire someone with really good software engineering, fundamental systems, distributed systems, that's much harder to teach and, like, help them get really good at using AI than, I think you can get really good at using AI in a couple months. Or you can get to, like, 99th percentile. If you're thoughtful about it and you're consuming a lot of content, you're watching other people work and you're learning from people who know how to do it well, you can learn it in a couple months. You cannot get a cs undergrad degree in three months unless you're like a super genius i that's not an answer to your question no really really hard

**Jan-Niklas** (01:23:36)

It's interesting because from that perspective i think mid-level and senior engineer, got to some extent more valuable for a company because they have that force multiplier now, and there's a perception issue because junior engineers are going to be perceived as more productive in a codebase because they can ship features but they're fundamentally lacking that understanding what they ship and then it's a matter of diligence to really be like, have that learning cycle so i i don't know i don't i wouldn't want to be in that position.

**Dex** (01:24:06)

Yeah i want one one way i frame this sometimes is like as as as leaders the the old like good pr review versus bad pr review was like the bad pr review was, this is wrong. Here's a code snippet on how I want it to look. The good PR review was, well, this isn't quite right. Can you go find how we do it over here and do it that way? The problem is that PR comment used to be good before AI because it made the person think and understand and transform the patterns and really learn how both parts of the codebase work and learn, okay, the next time I do this, I'm going to do it this way. But now if I put that comment on your PR, you're just going to paste it into Claude and Claude's going to do it for you and And you're not going to have to learn. And so I've seen Mitchell Hashimoto on comment threads on on Ghostty basically say, like, go ask Claude why this is wrong. It's not here's the here's where to look or here's what I would like it to look like. It's just like this is wrong. Go figure out why. And that's, and our challenge as like mentors and senior engineers and coaches and managers is now we have two jobs. We don't just have to teach people software engineering fundamentals and good program design. We also have to teach people good AI usage. And the two things are almost can like feel in conflict sometimes.

**Jan-Niklas** (01:25:21)

Yeah, see what you're saying.

**Dex** (01:25:23)

And I don't know. I don't know how you develop that instinct for like, hey, this, this, this function has 40 variables. That's too many. I mean, you can enforce some of this with linters, but a lot of it, you can't.

**Jan-Niklas** (01:25:32)

Have we struggled with like expressing code quality and metrics and stuff for, ever no one has really figured it out.

**Dex** (01:25:40)

For 30 years

**Jan-Niklas** (01:25:41)

Yeah i absolutely i'm the biggest fan of linters you if you can express something as a linter you absolutely should even more so now because then agents can fix it themselves yes but at the same time some things are just well this worked bad in the past let's not do this again.

**Dex** (01:25:56)

If your plan yeah i mean the only the only way to learn that this is really bad is you were stuck at 3 a.m. Debugging it because a pager went off and somebody it's like oh my god why did we do it this way i'm never doing this way again i'm never letting anyone else do it this way again etc we used to have this like wall we had a mono repo at my first job at sprout social in chicago and there was a there was a like a note on the wall in the in the repo README this was Bitbucket we didn't we weren't even on GitHub back then we had mercurial we were like hipster vcs or whatever in uh 2013 2014 i had this list of rules and one of the rules was like when you're reviewing a pr you have to ask all these questions and one of them is like would you want to review would you want to debug this code at two in the morning and like an agent just can't decide that because at the end of the day like there will be problems that an agent can't solve and then a human's gonna have to deal with it and like It's hard to know unless you've had that pain before.

**Jan-Niklas** (01:26:55)

The amount of conversation that I had with people on Twitter where I'm like, if you open a PR before someone else sees it, you should look through it yourself. And people were like devastated by that is mind blowing to me.

**Dex** (01:27:10)

That's crazy.

**Jan-Niklas** (01:27:11)

The audacity that you say, oh, I don't bother reading my own code or the code, but I expect you.

**Dex** (01:27:17)

But I'm going to ask someone else to spend an hour reading it.

**Jan-Niklas** (01:27:19)

Yeah, I'm still shocked by it. To me, to some extent, this goes kind of in my mentality when you're late to a meeting, you're taking away everyone else's time. That is kind of the same mentality. If you're not looking through the code and finding the easy things yourself, then you're not respecting everyone else's work properly. I have another question for you, and that is going to be somewhat off topic. It's not super off topic, but in your Twitter bio, you say you're working on the post-IDE IDE. And for various reasons, I have a mutual interest in that. So I would love to hear everything you're thinking about that so that I can see that. I'm just kidding. Tell me your, because products like that need to have a fundamental vision. Tell me your vision for a Human Layer.

**Dex** (01:28:06)

The fundamental vision is basically that like the text pane should not be the primary editor experience. And we've been doing this for a year ago. I think Cursor 3 just did this too. It's like, you should have to work really hard to find where the file browser. I still haven't found it. I have a Cursor set as my editor for editing markdown files, and I had to change it to Zed because I'm like, I can't find the file editor. And I spent about two minutes. I spent two minutes. I was like, this is annoying. I need to edit a file. I'm quitting this and switching to Zed because that still opens a freaking text pane. But the primary interaction with the codebase is not going to be through files. You're still going to want to see files. You're still going to want to read diffs. You still might want to open code and edit it by hand. But what would happen if that was a, the same way that agents were bolted on to the IDEs of the pass in a sidebar, what if the agent experience was primary and The file interaction experience was kind of secondary or tertiary? And so build an app from the ground up for managing agents. And then the bigger piece is like everything should be collaborative. It shouldn't matter where the code is. It shouldn't matter where the agent is running. And it shouldn't matter like basically... We have these like, even the modern super agile SDLC is very waterfall. It's like, okay, we plan the thing. The ticket is ready. Someone goes and builds it for two hours or two days. And then we review what they did. And that's very discreet. And our thesis is like, that should all be spread out and continuous. I posted something yesterday. I was like, hey, should we kill the PR? I want to kill the pull request. I don't want people to stop reviewing code. I want it to be more continuous. And as the thing is being written, as the steps are there, the earlier you can get in the more you can shift left on sharing understanding of what's happening at the plan level at the design level at the code level the uh you're gonna have advantages i think it's it's the more you the more you remove synchronization points and the more you make it async but fluid and anyone can come and see what your agent is doing at any point you can And that, like, it should feel like Slack. Like, the thing that made Slack better than email was that everything happened in channels. And so even if you weren't in a conversation, it was happening and you could see it and you could see discussions that were happening. And so it's, like, this idea of, like, how do we help development work be more in the open and not stuck on somebody's workstation or even their remote dev box? How do we make working on code feel more like, I hate Slack, it's chaos. But like they did get this like sharing of information and like basically the sound of the woods at night. You have all these channels lighting up and kind of check on what's happening. You can decide to ignore it. But how do we basically put that in everybody's view? So when you're not working on your thing, you kind of see what else is happening and you can pull context in before the, whether it's a 20 line pull request or a 2000 line pull request. When you know what other people on your team are doing and you can see it and you can jump in and you say like, oh, I'm working on this over here and here's my thing and here's a link to my session and my diff where I'm solving this like this. I think it's the future. And so there's a lot of infrastructure. There's a lot of cloud and streaming and sync and all these like modern like database concepts. Yeah.

**Jan-Niklas** (01:31:23)

Okay, so you're basically, the, I don't mean this in any dismissive way, but so therefore the IDE is more of a visualization layer over these information that you're having from your perspective. Yeah. Kind of like a command center pulling all the strings together.

**Dex** (01:31:40)

Yeah, and the core things are like, you have artifacts, which are like docs meant for humans, plans, designs, etc. You have agent sessions, which are like, kind of just like traces. You have like some grouping and organization projects, tasks that group those two things. And then you have like streaming code diffs of every time an agent makes a change, that's a data point. And then how do you create a good interface to do, to explore and understand all of that happening in real time across everybody in your team?

**Jan-Niklas** (01:32:11)

For the IntelliJ codebase, we have, I think right now, the last time I checked, roughly 500 committers in the Mono repository. Don't you think that would create like a shit ton of noise so if i um if i not if i look at my Slack just in the matter of this conversation that we recorded here i have like 20 messages pop up on my left side.

**Dex** (01:32:34)

Yeah i mean organizing that this is the google mission right it's like how do you take all that that that information and organize it and make it useful and how do i scope to just my team how can we use ai to kind of like instead of i have to go check a big inbox and decide what's worth looking at, something more generative can tell me, hey, okay, cool, you're working over here just so you know that person's working over there. You might want to take a look.

**Jan-Niklas** (01:32:58)

Maybe not the single source. At least most companies I know have also all data docs and stuff. If you move further away, how do you tackle building an understanding of architecture, decisions made, patterns that live throughout the codebase? How do you visualize that? Because I fundamentally agree with you that we're, and we've seen this, that we're very much moving away from this code writing experience more towards a code reading experience. Most companies, including us, haven't focused as much on code reading so far, because the code writing was the painful part. Not painful, but the time-consuming part for most people. And that just more or less solved itself. So now there's still this problem, okay, how do we build understanding? Particularly for people that join a project that is running for a couple of years. Like the the things with agent are super easy if you start on a greenfield project everything is, green and there's nothing there so you just start you you maybe have like an agents mb that say hey block every decision here in this architecture decision record, um you can be very diligent about documentation yada yada yada i have not seen a codebase in my 15 years of experience that is has all these information. So how do you build that understanding when the code reading or the code itself is a secondary artifact?

**Dex** (01:34:26)

I'll zoom out a little bit. I think Guillermo Rauch said something about this, which is like coding is different than shipping. Like coding is writing the actual code shipping is testing it deploying it like monitoring it fixing it take talking to users like shipping is like this whole big equation and this is like part of the software factor how do we get it so the models can ship not just code and they can read support tickets and go change stuff and they can read Sentry crashes and they can go change stuff and like continue evolving it um but basically the way i said it is like i don't think of it as like reading code versus writing code. I think the software engineer's job has evolved to write working code, uh, to produce working code. And so whether you write it by hand, whether the model writes it, however you test it, however you vet it, however you get it, like vetted by users, whatever it is, is like yours job is still to produce good software. Um, and so I, uh, I think the, the answer to your question of like, how do people onboard into it and like what there's so much truth beyond the docs and stuff is like Yes there is there is ways to take all these data points get diffs the current codebase agentic traces documents human comments on documents human comments on agentic traces and distill that into various levels of uh of of abstract things like okay the very high level one your CLAUDE.md or whatever it is is just like here's what this is and how it works the very basic is like cool that's not going to change that much and then you have layers of like the low the more detailed you get the more churn you're going to have and so you kind of like there's this like optimization problem of like how often are you streaming this like base set of data points into these views on the data and how often do those views need to be updated interesting okay

**Jan-Niklas** (01:36:20)

So I don't know.

**Dex** (01:36:21)

We're getting into the weird part, I guess.

**Jan-Niklas** (01:36:23)

No, I'm very interested in that because, I mean, we talked about this a lot, that the way that software developers work is changing and the tools are a fundamental part of that. So most people, from my perspective, are mostly just looking at like, okay, I use, I can now use a terminal and problem solved. This takes us to a whole nother level. So that's why I'm kind of excited to see what you're building there.

**Dex** (01:36:48)

Yeah.

**Jan-Niklas** (01:36:50)

Last question for you. How or what tip would you give someone who's working for 20 years at the same company, maybe is still, right, like the level of like how they're educating building skills is much slower than the people that are like on Twitter every day. What is the single advice you would give this like very traditional engineer right now?

**Dex** (01:37:14)

Three things. One, find ways to use it as much as possible. Even if you throw out what you did, learn where the boundaries are, you will eventually be surprised of like, oh, this thing I used to hate doing it now can do. And like, so like build that intuition just by using AI as much as you can. Number two, find people that you respect that are three to six months ahead of you and talk to them as much as you can. I mean, I got pulled forward from where I was 18 months ago pretty quickly by being in like two very special group chats on special to me. They're just group chats where people bullshit about AI all day. But it's like find find a community that feels more people are way more willing to share their stuff in private. Everything you see posted on the TL, there's like 90 percent of it is like hype and slop and stuff. So like find a small group of like three to thirty to maybe a hundred people that you respect and trust and talk to them you know talk to your peers who are in the same space and find people who are a little bit ahead uh and talk about what really works with them um and i think private works better because people are more willing to share their secrets and then number three is uh I hate to say this, but like figure out the loops thing. Start trying to engineer like small loops into your system, like learn back pressure. Basically, that's the number one thing is like figure out how you can make it as easy as possible for the model to check its own work. That's the only way that you can leave things unattended for longer and actually get good results.

**Jan-Niklas** (01:38:47)

Thank you so much for joining me twice for this. I've I had a great time talking to you. I very much appreciate you taking the time for this.

**Dex** (01:38:53)

Thank you so much. This is super fun, dude. I think you we have this there's like yeah it's almost like find people who are as skeptical as you are or live in the same like skepticism circle because it's a lot more fun because you all we all see the same problems in the world

**Jan-Niklas** (01:39:09)

Awesome thank you so much and see you next time.

**Dex** (01:39:11)

Yeah and this was great thank you

Dex Horthy - What Actually Gets You 2-3x With AI Coding

Jan-Niklas Wortmann

Key Takeaways

Chapters

Mentioned In This Episode

Pull Quotes

Guest Bio

Full Transcript

How To Ship Real Code With AI (Not Junk) ft. David Cramer

Code Isn't Free - Mario Zechner on the Hard Truths of Coding With AI (creator of Pi)

A Google Engineer's Honest Take on AI, Angular, and the Future of Your Career

Key Takeaways

Chapters

Mentioned In This Episode

Pull Quotes

Guest Bio

Full Transcript

Related Episodes

How To Ship Real Code With AI (Not Junk) ft. David Cramer

Code Isn't Free - Mario Zechner on the Hard Truths of Coding With AI (creator of Pi)

A Google Engineer's Honest Take on AI, Angular, and the Future of Your Career