podcast Featured

How To Ship Real Code With AI (Not Junk) ft. David Cramer

Sentry co-founder David Cramer on why 100x AI productivity is overhyped, how Warden found 100+ vulnerabilities in production code, and where LLMs actually help working developers.

Listen on

Spotify

Apple Podcasts

YouTube

Jan-Niklas Wortmann

30 Jun 2026 • 61 min read

Episode 20 · June 30, 2026 · David Cramer

Key Takeaways

The 100x developer narrative breaks down once you optimize for quality, security, and maintainability instead of raw output.
David built Warden inside Sentry and used it to find more than 100 previously unknown vulnerabilities in production code, including auth bypasses.
Search and internal knowledge tools may be the highest-leverage LLM use case inside a company today, not autonomous code generation.
MCP is useful when it gives agents structured access to real workflows, not when it is treated as a thin API wrapper.
Open source sustainability and AI training economics are colliding, and the financial models behind many AI companies do not add up.

Chapters

Mentioned In This Episode

Pull Quotes

So this 100X thing is BS. The only way you get more done is when you generate junk that you don't need.

LLMs are not making it faster for me to build software. Despite what the internet would like to say, I'll sit here all day long on a single patch.

If I ship something that has massive vulnerabilities in Sentry, that could cause the company to disappear.

There is nobody that is credible that says software engineering as a craft is completely changing.

Guest Bio

David Cramer is a software engineer by trade, and the co-founder and CTO of Sentry, an open source application monitoring platform used by nearly 100,000 technology companies. Prior to Sentry, he focused on infrastructure and developer experience at companies like Dropbox and Disqus, and is a prolific contributor to several open source ecosystems.

Full Transcript

Read the full transcript

# Transcript

Because we went from like tab complete to instantly we just don't write code anymore. And it's like, maybe we should have stopped somewhere in between. So this 100X thing is BS. But the only way you get more done is when you generate junk that you don't need. This is David Cramer, co-founder of Sentry,

and one of the few people on a C-level who's actually using AI to ship production software at Sentry. But the reality is like, if I ship something that has massive vulnerabilities in Sentry, that could cause the company to disappear. You want to flex that you can generate all of your code and have hundreds of

things going in parallel? I will flex and show you how broken the code is 100% of the time. If you hang out there, you see like, oh, software engineering as a craft is completely changing, yada, yada, yada.

There is nobody that is credible that says that. LLMs are not making it faster for me to build software. Despite what the internet would like to say, I'll sit here all day long on a single patch. This is David Cramer on how to ship real code with AI.

I built a scanner inside of Sentry that identified a ton of security vulnerabilities in our active code base, which is a little bit borderline hobby project, but it's like real-world production software that we use every day now. And then I went and I patched some of those security vulnerabilities for us.

And so, you know, I'd like to think that I have enough experience in this space, plus I'm pretty grounded about my experience. Plus, I do not care at all that I'm willing to share my opinion about what I see. So one thing, and I'm glad you bring up that security scanner because I read

the blog post that I think you published like February or something like that. I'll link it in the show notes. And like in one of the first introduction sentence, you said something like, we literally have virtually infinite money for AI. Something along the line. Is that something that you still stick to?

Because I see a lot of people like, oh, companies, Uber recently introducing an AI limit that walked back on statements like that. I would say it's not infinite, but for reasonable use, it's very easy to waste a lot of money. You see this from the labs.

The labs just spend money because whatever. I don't know. That's got to stop at some point, right? But it's really hard to do something useful and spend, say, $1,000 a week on a coding agent or more than that, let alone. And stuff does add up, but as soon as stuff adds up, you quickly realize you're

doing things in an inefficient way. Like, LLMs are not the solution to inefficiency. And so, I don't know, here's my favorite analogy that I think people still don't worry about, even though they should. If you're going to build a web crawler in 2026, you're going to use an LLM.

But the bad way to do it would be having an LLM run through every single page that it gets, right? The right way to do it would be to have an LLM, when the pages change, generate a new script that can parse the page, and then just run that script over and over until it stops working, right?

Because one's much more efficient than the other. And I think that's true in a lot of things, you know? It can't be true in everything, obviously. Like our scanner tool, for example, I think there are ways to bring determinism to it and make it more

efficient, but not necessarily true. But when we look at, like, that particular example because it's finding security vulnerabilities, we look at, like, our bug bounty program and how much that costs us whenever anybody finds any of these.

And I think the first run, or one of the first runs was like $1,500 to scan one of the core code bases or a part of the core code base. And we found two high-sev vulnerabilities in that that more than covered, I think, the monthly spend of that thing that month.

And so... Well, and that doesn't even include the fact that someone could have exploited that. Yeah, yeah. Brand damage and all those things. But I do think the money has to come from somewhere, I would say. So I think people shouldn't be foolish with it and they should have rational expectations.

But at least what we have seen is right now, to be fair, like $1,000 a week is still quite a lot from a budget point of view. But right now we feel like it's somewhat okay. And we expect like the baseline to get a little bit cheaper in time. Maybe not a lot, but a little bit.

It's funny that you say that in the week where Opus, which is such like ridiculously expensive, came out. But I get what you're saying. On that, how do you treat AI budget within Sentry, though, like more on an IC level? Because that is a thing that I think is really difficult.

And we see this also at JetBrains, where we have people that, just burn through like tokens ridiculously and then some people just use it very moderately and it's very hard we're nowhere near yet really evaluating like an ROI on something where i'm very curious to see how companies are doing

that but how is that from your perspective, yeah honestly like people ask this a lot and there's a lot of companies trying to build solutions around this, it's too early so like we look at it we set a budget that we thought was reasonable,

and i think it's it's not a thousand dollars a week to be clear it's much less um but we set it, roughly what it amounted to was $15,000 a year a developer, which, math that out, it's actually close to Uber. It's close to $1,500 a month. But what you find is that's blended across the team.

And if you look at engineering or any job function, you're going to have people that are high throughput, high volume, and people that are not. And some of that is like a performance of individuals thing. And some of that is just the way they work. Some of that is what they're working on, all these things.

I don't think $1,500 a month is enough, at least right now. But when we look at it, we're like, we're okay with that spend. And we're okay with that spend in the same way we are okay doing R&D work that might not be successful, right? And so we think it will be valuable in the future.

And actually, I think it's pretty obvious that you do have to learn how to use LLMs. And you see this from people that are on different cycles of the learning curve. and they're, I don't know, they still think they're magic or something like this. And so we're mostly just okay with it. I think that's like year one. Yeah.

But it just wasn't, I mean, we're like 200 engineers. It wasn't that expensive at the end of the day. It's not nothing, right? Like money, money is like, there's not an infinite budget for everything if you just keep stacking it. But I'm like, I could definitely delete some tools from the company that don't

cost that much, unfortunately. But it felt justified. And from an ROI point of view, we just don't care. Like the ROI. And so this is my decision-making process. If I believe in it, that's it. Like I don't need any other evidence beyond that.

And I think this is also where me being sort of in the weeds to some degree or to a large degree helps a lot because I can make those judgment calls where, and this wasn't, I didn't even come up with this budget item, our CTO did, and I'm like, done. Three million sounds fine.

But I see the incremental value and I see some things, not everything, some things that you definitely get done much more easily or much more quickly, And then other things, what we're probably wasting our time and wasting our money right now, to be honest with you.

But it's probably not like crazy efficiencies. Like it's probably on average. I bet it's like low 10% kind of like boost in actual output. You're not expecting 100x developers? How come? No. I mean, you know what's insane? I was talking to somebody about this.

I have the kind of personality that's really good at context switching. ADD or something. I don't know. Who knows? but I'm usually pretty good about jumping around and I'm usually, I'm pretty impatient and I'm pretty fast about doing things. And you know what, what's really annoying these days is it's not,

like LLMs are not making it faster for me to build software. Despite what the internet would like to say, I'll sit here all day long on a single patch. Now, I might work on five different single patches at the same time, but to get one quality piece of code in any sufficiently complicated software,

it's not any easier than it used to be. And in fact, in some cases, it's harder because. I forced myself to stop writing code by hand almost a year ago at this point. And that constraint, whether useful or not, it's just harder, right?

And so I'd like to believe I could get more done. So this 100x thing is BS. But the only way you get more done is when you generate junk that you don't need, when it's like brand new projects or something like this, which is almost always something that was not actually going to add that much value.

On that i i am curious what i what is your opinion on like uh peter steinberger's way of developing because as an engineer i have a lot of respect and i also appreciate him being like, very much on the forefront i still think like no company anywhere near with

that kind of money on development efforts so i'm curious to hear your opinion on that i think peter is a sharp guy and he's certainly insane um i mean i, you are not generating the amount of output they are generating for real-world software that has liabilities.

Like, I mean, it's impressive what they've done, don't get me wrong, but it's a damn chatbot where they've decided to accept all risk. And they try to de-risk it. Like, again, they're not just intentionally doing bad things, but they basically accepted that it's junk code.

And I think there's a, and I don't know, I haven't talked to Peter about this personally, but like, I think there's a type of person, and I can understand this, that inherently believes that LLMs will get better enough that they will go back and fix this stuff, that it will be able to clean up all the junk that's

been stacked up along the way. I don't think that's true. But the reality is, like, if I ship something that has massive vulnerabilities in Sentry, that is a very serious dilemma that could cause the company to disappear. If you ship something that fucks up the chatbot, what happens? Nothing.

Like, a bunch of hobbyists maybe get mad? You know, I don't know. So it's just like, it's a science experiment. It's interesting. It's not worth a seven-figure compute bill, that's for sure. But yes, I don't know. But nobody does that in enterprise, if you will.

I don't like the word enterprise, but you get what I mean. I do get what you mean. At the same time, Anthropic seems to be operating on a similar scale. Again, also a very similar circumstance with working on the lab where inference is. Actually free for them more or less so i'm wondering if this is as you said

a science experiment which i i like the metaphor or just like a glimpse on the future and i'm definitely more leaning on the science experiment i don't think it's, reasonable so i think it's a science experiment and this is no shade i like a lot of people at Anthropic but the software is broken all the time and so

you want to flex that you can generate all of your code and have hundreds of things going in parallel, i will flex and show you how broken the code is 100% of the time and how the entire industry has made a meme and a mockery out of that and that's like it's

not a flex i'm like that's cool but like that's not the world i live in i want like quality software and so do a lot of people so do our customers and i as a customer of Anthropic i mean i stopped using Claude Code more than a month ago

I haven't touched, I haven't touched Opus. I'm like yeah it's probably interesting to see i just don't care because i'm like i'm gonna be annoyed about all the random and things that keep breaking daily in it. And they seem to, it's like forest from the trees. I'm like,

people do not see the community sentiment here. That's like, hey, slow down. And software verification is the hardest problem. It's always been one of the hardest problems. We've not made it any better. We have code review bots, and they help a little bit. But if you're honest with yourself, we're spending inference to fix all the

other stuff that was from inference. And I mean, that's fine, but it's really expensive. And it doesn't actually fix it because it's non-deterministic. And so I don't know. So I think a little bit of this, yes, is probably the future.

I don't think to the degree that everybody would like to believe it is, unless some new technology comes into existence. I don't think transformers are. Um, and I think it's mostly a science experiment. I think it's mostly living in their bubble, kind of forgetting,

but it's easy to let success go to your head. You know, they say PMF, product- market fit, fixes everything. Well, Claude Code has great product-market fit. It fixes the fact that the software is broken all the time. That is wildly inefficient. That, you know, I don't know. There's a, it's, there's no causation is what I would tell you.

I think it's purely correlation that they generate all the like massive amounts of code and that the product is successful, to be fair also like i i'm still on the fence whether Anthropic's marketing is just entirely next level or pure luck all the time it's insane they're,

they they keep being successful with their marketing for no obvious reason and it's completely infuriating but. You mentioned one thing that LLMs are not a great solution for a lot of things. Where do you think LLMs are a great solution for? I think search is the greatest, greatest thing.

And I think it's still under leveraged. Like the most valuable thing I've done at our company this year is build this Slack bot, which has been very fun to build because what it did was it took sort of, there's this thing, and I'm guilty of this too, where,

you will ask the question of your peers because you're being lazy instead of going to answer it yourself. And I like, this is my model in life. Like if somebody else will solve the problem faster for you, then why solve it yourself kind of thing.

So I do this too, but at scale, it's actually like a big time waster. And with Slack, it actually makes it worse because people just broadcast these questions all over the place. And so I'm like, why are people still not using LLMs? And the sort of answer

was like, the interfaces are not great for some of this. And so we built the Slack bot, we gave it access to all the stuff like a bunch of people have at this point. And people now ask the bot or tag the bot in to answer these questions and it

does it so damn well because it searches so much information. And when you see that, you're like, that is genuinely like an improvement. It's like a behavioral improvement. It's an efficiency improvement. Now, I don't know how much it is, obviously.

But I think it's like still undervalued and underinvested in. And then, and I think there's lots of versions of that, right? Like summarization, all these other techniques are basically the same. I think code generation is the harder, like one of the hardest problems we could

pick and somehow we've decided that's the most important problem or the most valuable problem even if it's unsolvable um. But then beyond that, I think, I don't know, like a lot of things are not, they're not the solution. Like for verification of software,

for example, I'd be hard pressed to say LLMs are like the right tool for the job just because of the nature of what they are. But and then, yeah, I don't know. I think we get surprised in where you can leverage them to augment other mechanisms, right?

I use them a lot for like parallelizing things, as you also kind of said, of like jumping around those kind of things or i have a meeting so i send an LLM on like a quest to find information prepare, some work or something you wrote the Sentry MCP is that right yeah,

how do you feel about MCP these days it's great tell me how you really feel i'm gonna i'm gonna publish something later actually because like there's still nothing that compares i think there's flaws like the protocol is a pain in the ass like it should have been stateless i think it's still not fully stateless but it's going to be

but it should have been stateless by default it should have just been HTTP. It isn't. We are where we are. It's not going away. There's some other things that are kind of annoying, but most of the things you can work around if you just use critical thinking.

But one of the problems... So I'll give you... The reason I like it is because I care about user experience. I care about what is the lowest friction, best user experience I can create. And when I have effectively a plugin that drops in, has native authentication,

which means it can do a lot of things. It can do like scoped authentication and all these things which reduces friction makes it more secure it can do all that hypothetically it has controls around which tools, permission systems, all this stuff which is very valuable and I mean

this in the contrast with something like arbitrary shell commands, CLI, etc. don't um, And when it's baked into the interpreter or the agent where I can actually steer the agent a little bit, like reverse steer it, you actually can create a really,

really good experience. And so I like it for all those things. It has its challenges with like, I don't know, I'm forever trying to figure out why our MCP server gets deauthenticated all the time. Because it's like, you know. I had that with Notion too, if that makes it feel better.

I think it's common. And I still can't figure out if there's a bug. And I'm going to like, I'm actually going to write a new transport for it to see if the bug goes away. Because you have no idea how many hours I've spent trying to get rid of this behavior, and I'm not sure if it's the clients or if it's us.

But you'll hit all these things, and people don't consider you can just work around them. So... We always constrain the tools, for example, because you don't want to cause too much pollution in the context environment. That said, Sentry is the only MCP I use in my coding agents because it's the

only one I use daily with them, and there's no point in keeping stuff loaded. But, for example, we would constrain the tools because you don't want to overload it and you need steering and all this stuff. We shipped something earlier this month that just puts, like if you're familiar

with code mode, it has the concept as a search and execute function that does progressive discovery, and then it writes code. But if you ignore the code part, all we did was add a search and execute tool to our MCP. And we buried a bunch of other tools behind it.

And it still works exceedingly well. We didn't do it for everything. But we're like, oh, now we have another solution. Now we can expose more once again. And we can increase the amount of behavior and all that stuff. And so I think people wrongly wrote it off because they just simply don't understand

what the market looks like or what the value of these things are. Or they didn't understand how to build a good product in this space where when I saw it, I'm like, this is clearly going to be a useful way to get Sentry into agents, you know, in a way where otherwise we can't, you know.

Like a CLI is not the solution to me to that problem because you still have to like bundle some other stuff, get them to install this, you know, etc. And it's been really good for us. Like we have a lot of daily active users on the MCP, even with them getting logged out all the time, which is absurd to me.

And I think that tells you just how like useful it is. And I dog food it all the time. I use it myself. And so I recognize some of that value. But yeah, I think it's very, it's the first time in my career that I've seen like a reverse integration like this where you actually can build a plugin and

it's not, you know, designed for like a single partner. It's just like you build it and like it just works for all future partners kind of thing. And then, I don't know, it ends up being like very, very high value. Do you think the protocol was published too early as a standard though?

Because some of the problems you mentioned, particularly around OAuth, If you have a concrete use case in mind, which, I mean, enterprise software, is kind of the obvious for those kind of things usually, you should think about Auth at least. The version of Auth they're using, obviously, most of it's still the DCR,

the Dynamic Client Registration or whatever it's called. I had never heard of before MCP, which I'm sure is true for the entire internet. It's complicated. It's not super secure in the sense of, it's not insecure, but security includes a lot of things. It's not just like, is it logically secure?

It's also like, does it prevent fraud and abuse and all this stuff? And so it doesn't really do that. But I don't know that there's a solution to the problem, to be honest with you. And so, I don't know. I think it works okay.

I'm glad it's OAuth, if you will. And I'm glad that because it's OAuth, you can implement it natively in a lot of flows. But yeah, it does feel rough. I actually think the OAuth is less of an issue than sort of the way the transport mechanism works.

But I do agree that it was like a standard right off the bat. And whenever you do this, and especially when there's a lot of money involved in sort of, AI is one of those phases. All the big companies are like, yeah, standards. We love spending time on standards.

Let's get a committee together and go argue about standards. And I will say, to MCP's credit, it at least is making progress. Where most of those go, like, I love to rag on OpenTelemetry. So, like, most of them get stuck, you know?

I was just about to say that, like, for a standard, it's still very fast moving. Other than I think VS Code, no one has really, is even aiming to be, like, 100% spec compliant. Which tells a lot about the pace of the product or the protocol.

Or the value of some of the protocol, to be fair. That's also true. Most of the things, like resources, they were never used. Why did they exist? Prompts have been dead a long time. They didn't get implemented everywhere. Tools are great, don't get me wrong.

I think skills are coming, but they're not there yet. I didn't even know that. I think that your UI part is interesting, but I cannot quite foresee the use case of that other than like visualizing a diagram or something, which most of the, at least those chat clients can do these days anyway.

So I'm not quite sure how I feel about that. It's my analogy for the UI. I agree. I think it's interesting, but I don't know if it's practical. Like we have UIs in all this and it can work, right? But Sentry has a

lot of data, right? And so we have traces, which you could render visually much more interestingly if you could embed like a React component, right? Okay, is that a big deal? No. We have video replay, session replays. Okay. Okay, that one is like wildly different

if you could embed like an interactable component. And so do you need that though? I don't know, you can click the link and go into the UI. It's not actually that useful. So I think, and this is how all these things go. There's a lot of, this is neat, not this had market demand and thus we built it kind of thing.

And so, yeah, I don't know. Maybe it'll become more valuable, but we actually haven't shipped any MCP UI apps, I think is the core of the spec because it just doesn't work in most places. And it's kind of, it's like. It has trade-offs to ship it. You end up bundling more information over the

wire, even if it's not used. And so I don't know. I have mixed feelings on the whole thing right now. But it does feel like there's a bunch of this stuff that probably doesn't survive in the spec. I think that is a good thing. I think.

Kind of weird for a spec, but still. How do you feel about the... So if you're... I spend way too much time online. But if you are online, there you see like, oh, software engineering as a craft is completely changing, yada, yada, yada. Do you think that? Or somebody says that, I click their bio, and I'm like, okay.

And that gives you all the information you need. There is nobody that is credible, that says that. I've not found a single human. Like, obviously, the way we're writing code is a little bit different. But software engineering, the way you design systems, the knowledge you need,

all these things, it's not gotten any easier. The LLM's not magically going to give you, like, decisions are not a math equation in things like software and products. It may not be a creative expression, but it's somewhere in between. There's no binary right answer for a lot of things.

And even on top of that, they're not that capable as machines. And I think any of us that actually use them day to day see this. You give them a hard test, and I guess they go haywire. And that's even on binary problems. Because, yeah, you can run them in a loop

and verify to some degree on some problems, but not most things. They can't even write CSS today. They can't do anything in UI without it being wrong all the time, right? And so I feel like sometimes you have to be like remind yourself that a lot

of stuff is people attempting to do marketing but not understanding that one, that's what they're doing and then two, they're doing it poorly. And Twitter incentivizes the rage bait shit. And so I know a lot of people that are like.

Sort of like just tired of these conversations at this point and they're like i'm just like checking out of twitter because it's just like echo chamber of people that aren't actually even in the industry, talking about things that aren't true trying to get like clicks you know things like this and

and it's like you can ignore it but it's frustrating to ignore it at least for like a lot of us right it drains a lot of energy also that's because you're if you spend too much time in this echo chamber, you start to lose perspective to some extent. Yeah, and I think it's actually really unfortunate. It's the same with startups and whatnot.

I'll post something on the internet and I'm never looking for somebody to sell me a thing. I'll have a genuine technology interest about something. I'm like, oh, I wonder if anybody else has done this at all or X, Y, Z. And I'll just get like 20, they're not even startups, they're like little slapped-

together projects that kind of look like a startup in yesterday's world. And I'm like, what is going on here? And I think just people are out of touch with a lot of things. And because there's so much noise, it's really hard to find signal in it as like somebody consuming it. Like, like I can keep grounded in my opinions of

like what I'm doing and what is good and what isn't good. But knowing what is good from other people that I just like is sort of beyond my domain or I have an experience. I'm like, I don't know what's real and what's not real. And, and you're sort of back to the same, which is good.

You're back to this thing where there's like reputational value where, well. There's a handful of people I know. I recognize their names because I've interacted with them a bunch. I know what they do. I know they're real. They're not just like one of these randoms on the internet producing slop.

And so I'm like, well, if they're talking about something, I will probably like pay more attention now. But the mental load of like keeping this context and then figuring out who's who and all this, it's a lot. I don't know. I think it's also difficult because you get a lot of shitty noise

from like these big companies, like Facebook still having like their token leaderboard where I'm just like, well, this is the dumbest idea if I ever heard. Yeah or like the nvidia guy who who was like yeah an engineer making 500 000 should at least spend 250 000 a year in tokens like,

where are we talking about the value that this produced i can easily spend that much money on tokens no problem i don't know some of this is and like. There are a lot of people in high profile so i i believe in a sort of moral duty thing if you if you have power you have a responsibility,

full stop. I just don't care. And I don't have a lot of power, but I have enough, like influence, if you will. I have enough influence that I feel like more responsibility to be like that is bullshit is, you know, that's not necessarily what I mean.

But I do feel like I have a duty to not just feed people BS. And a lot of people... I don't know. Some of it I think people don't know. They're so out of touch. Like CEOs and stuff, they're so out of touch.

And then other people, I think they know what they're doing and they choose to do it anyway. And I'm like, all of you are to blame. And you'll have somebody. So I think Jensen Huang is a good example of somebody who's generally pretty grounded. He's not running around being like, AI's replacing all jobs or anything.

He'll say stuff once in a while that's a little absurd to me. There was that quote from him, like, you know, engineers should be spending like a quarter mil a year in inference or something. I'm like, eh, no. So the most funny is but then you hear all these other people especially like

Dario it's like the most absurd shit I was just about to say that I was just about to say, he is just frankly he's bad at public comms he should not do it that'd be my honest advice if he ever heard me or if I ever saw him be like you should stop posting anything in public ever because you don't know what you're doing and

that's just simply it's the truth, and I think it's turning out okay because they have such good product market fit but in any other situation and they're just burning goodwill of customers. And it's like, why? What do you gain out of burning good? Like, you know, it doesn't make sense.

The thing is like at the start when Anthropic got big, they primarily targeted developer for their marketing and then saying like, oh, in six months, developers won't write code anymore. Who thought this was a good idea? Besides the fact that it's wildly untrue, it's just...

And I'm sure he means well, but in that role, you've got to step back and be like, okay, I actually do have to consider the message I'm putting out. I'm not just like sort of having to think piece with my friends or anything. I'm like, oh, will it maybe look like that or something?

If you go to a journalist and you say that, what do you think happens? It's bad. It's bad for everybody. And I just wish people would like check themselves a little bit. And a lot of people do this. It's not just like people that are high profile

like him. that are people that are like my peers that also constantly are doing things like this. And they know what it does. And I find it quite frustrating. But there's an increasingly, there's a growing population, I think, also of people. That are kind of like bothered by that, that are also in positions like mine

that are sort of being more grounded and voices of reason, you know, trying to, literally trying to just be grounded, not be like super negative or like overly optimistic about things. And I think that is like really valuable. But again, signal to noise is still a really hard problem.

One thing I would like to discuss because I got into the industry, what, 15 years ago, something like that, where the easiest way to learn for me was like just like facing a problem heads on and struggling with it for days that's that's how i learned how do you think

people will get into the industry these days where the, bar to entry to some extent is so much lower because you can just outsource your problem and not go through the motions of like learning by repetition learning by, i like to call it learning by pain just struggling with problems for days yeah

this i actually don't know because i agree like you know. I think one so Well, I think two things. I think academics are still important because they give you theory. Even if it's not applied most of the time, knowing that theory is actually still useful.

And I say this as somebody who's like, I'm a high school dropout, but I recognize, I saw the value later career where these things would come into play and whatnot. So I do think that still helps, but I don't know how you learn if you don't actually, if you're not doing the thing.

Because you need, at minimum, even if the LLMs or whatever, even if technology can generate code and verify and some stuff like this, you still need systems design. And like, the problem is there's like an impatience that I think is amplified with these things. It's like gambling.

And are you going to read the output that says this is a bad design and actually consume it and then guide it correctly next time? You're probably not. You're probably going to go in and do the same thing where you're like, I want to build this thing.

You're not going to give it to the design of the system. You're going to let it do whatever it's going to do. And it will often do it wrong anyways, which doesn't help the situation. And I don't actually know.

I don't have an instinct on what the solution is because I just don't think you're going to be able to learn that way. And then as a solution, you shouldn't be using LLMs. That doesn't seem practical either. So I don't actually know.

It seems complicated, to be honest. I don't think this saying of like, oh yeah, while learning, you shouldn't use LLMs. That is not realistic. That's not how you're going to work long term. At the same time, I think this accountability piece of like,

at the end of the day, it's still your name tied to that piece of code. If there's a bug causing a production outage, you're to some extent accountable for that. I'm not trying to establish finger-pointing culture by any means, but at the same time, that's how things work if I cause a production outage.

Yeah, I think there's got to be accountability. And I want to believe, because we're not in a world, we're not like, I don't think we ever get to this, to be quite honest with you. Humans are going to stay in the loop. If nothing else, because there has to

be accountability, there has to be liability. And so at the very minimum, there's code review. And good engineering practices are exactly the same as they were before. So like smaller change sets, all this. The problem is everybody wants to go fast now and you can't necessarily go faster.

And if you still practice all those things, which I think will continue to enforce, maybe you learn more through like that peer review process and stuff than you would before because you did learn a lot through that in the past, or through pre-design, sort of whether it's building specs or some other systems

design exercise. But yeah, I'm not sure. One thing you brought up that is kind of insane to me is like this whole conversation of like, oh, we need to push out more code faster, always just assumes that writing code was the bottleneck where I'm like, in every company I've worked at,

writing code is never the issue. I can write code all day if I didn't have meetings, for instance, which I try to prioritize important meetings, for instance. Like that whole conversation doesn't make sense. And I'm not reading code just

for the sake of reading and because it's such a fun activity because it actually prevents issues. What are we even discussing here? It's interesting because I don't know. If you kind of step back and you look... If you really look at things and try to be like very critical,

things don't look like they've changed that much in terms of how we produce software. Like all the mechanisms look exactly the same. And so, but there's these other things that are kind of breaking with it. I'll give you mine. I'm very much like, I am super high throughput. Like if you went back in time,

like the 10X engineer persona, nobody would question that that is me. And it's mostly because my output is so, so extreme. and it still is with LLMs but one challenge I have now and this might get solved for what it's worth is a lot of, learning if you will but it wasn't necessarily learning how to code but,

it is connected I would build a first version of something I didn't build a spec, I would build the software and I would iterate on it as my idea got more refined as it got more solid and sometimes I'd come up with better technology choices in that idea and I think that's pretty common for folks um,

And that was everything. And that's actually quite hard to do with LLMs. One, because they're slow. And I actually want to run an experiment of using Composer 2 regularly. I have a feeling I'm not going to be happy because I'm going to have to course correct it too often.

But I think it's also because you're not. One of the things I would do regularly is you kind of refactor as you grow. It's this constant cleanup thing. And nobody's found a solution for this. And I don't know if there is one when using agents and whatnot.

But a lot of that is subjective. it's like very hard to have an objective like this is definitely correct because you have sort of a vision in your head or, a set of things you've built on top of that you believe in and they've worked well for you or, you know, something like this.

And you also develop those things by sitting in there and going through the problems and whatnot. And we still have some of that same thing. And so there's one version that you could believe in that's like, well, doing that now will become refinement on the tools on top of agents rather than the code itself and getting them to output the right code.

But another version says, well, there's still language models and they're still going to be non-deterministic and that might not work. And we're going to have to do something else differently. Like there's a lot of people I know that have tried different experiments around this.

Like, oh, maybe they'll by hand draft out the interface or something like just pseudocode or something and then have the LLM fill it in. Or they'll try to do some crazy spec thing or something, which I don't find works at all for what it's worth because the spec's never correct.

But um yeah I don't know I kind of wonder I was talking to some other folks about this who are you know rational and very good um. And then kind of the shared sentiment was like, did we just jump to put like a pretending that the end state was already here or something like this?

Because we went from like tab complete to instantly we just don't write code anymore. And it's like maybe we should have stopped somewhere in between. That's very much where I'm leaning towards. Like, yes, LLMs are good at generating the necessary characters that make up code.

It's also a lot faster than if I would write type it by hand. If I don't know exactly what I need, right? Like there's explorative work. But at the same time, like you still need to read code. You still need to maintain systems. Usually you're also

not working on a system yourself where everything just lives in your head. Like if I look at IntelliJ, that is 20 years of institutional knowledge that lives somewhere. It's certainly not in my head. One thing you brought up that I would like to ask you, do you think Vibe coding will replace no-code solutions?

At the very least, I think those solutions are going to change, but. Not everybody's an engineer, and that the delta between generating software and using some kind of block system or something is very different. And so are they going to be able to debug the problems with it?

Do they have to now all of a sudden? I don't know. I think generative interfaces, which I would not call the same thing, probably do. Like, there's probably a lot more dynamic interfaces and things people can do. And I think some of them will probably become more true code generation,

like, without constraints. Whereas I think a lot are actually just going to be constrained interface generation. Like, I don't know, imagine a design system or something like this, and it'll be able to build lots of different blocks. I'm not totally sure.

It definitely changes. But, like, does Squarespace become lovable or something like that? I'm like, hmm, I'm not sure. Because I think the one thing I do believe in with absolute certainty is the market has expanded, that there are more people doing engineering work that

were not previously engineers and are not great engineers now, but they're better than they were, and they will probably continue to get better, but, they may stop at an entry-level engineering role.

But all they're doing is building little websites for their brick-and-mortar store or something like this, right? And so we have this conviction around market expansion because we see it in numbers, and we're trying to figure out how do we address that as a company

because we only target developers. We've never targeted, say, WordPress admins or anything like this, right? And these are sort of new developers that we feel are close enough that they are part of our audience, I guess, if you will.

And so I think that's a truth, but that's not everybody either. And so I don't know where that delta is in between, but it is interesting. Like our CFO, he has some application he's built, and it's totally reasonable. And when you realize he's a finance guy, he thinks like an engineer,

it kind of makes sense that he'd be capable of doing this. He just doesn't know the syntax, and he doesn't know systems design, to be fair, either. But he's definitely more capable than a lot of people would be um and so i don't know i don't know what that translates to though and over a long period of time,

for what it's worth i also think vibe coding is very viable if you're just doing something for yourself just like you write a tool for yourself you don't really care about the code quality or something and if it breaks you just yeet it through a prompt one more time,

i i'm cool with that and i do that too for some things but like, the minute you send like customers or friends or families on it you should maybe know a little bit what you're doing there. One thing you brought up, do you think the distinction between product work

or like product development, product management, design, is somewhat growing closer together with the use of LLMs? Because I can't see at least like a designer or product manager to create a prototype, not production ready, but at least like a prototype,

which is usually easier to discuss than a spec. I don't see anything that suggests, for example, that we're collapsing roles like that. An engineer is now also a PM. Not in any version that is different than before, if you will. Same with design.

I do see people having more access to do things, especially at a prototype stage. And some people hate this for what it's worth. Our design team hates when I, slap together some generative artwork. But it does allow me to get more prototypes out the door and stuff like that,

and so I think I think it is giving us it's a new set of tools right it's like we're still trying to solve the same problems, we've expanded tools like I my analogy here and I'm not I'm not an expert on this so take it for what it is is once upon a time there was my mom was an accountant growing up and,

she started in the era of Lotus 1-2-3, which I know nothing about but it was originally like some accounting software or something. Yeah. And I only know this story because she stopped being an accountant when she got pregnant and had me as a kid.

And there was that gap where they went from Lotus 1-2-3 to, I don't know if it was Excel, but whatever came after it, right? And it was a big technology change, right? It was a toolchain change. But accounting didn't really change. It was just like, there was new software.

And at least back then, there wasn't like this thing where you could easily pick up the new software or whatever. It was like, you actually had to learn it. This is just that to me. It's just another set of tools that are going to be used in certain ways, except right now we're pretending it's used in every single way.

And it changes... We're sort of... We is not necessarily representative, but a lot of people are sort of jumping to the conclusion that... A bunch of jobs no longer exist. They are now LLMs. And I'm like,

hmm, I haven't seen a single job. Like, there are real roles that are at risk, I think. Things in like, especially like with big companies like customer service centers and things like this, those are going to change drastically. Engineering, I don't know. I still need just as many engineers.

I need them to be just as capable designers, PMs, salespeople. Like, all this stuff, I don't know. I don't see a lot of it changing in terms of people doing the work, just the way they do the work is probably going to, some of them will change more than others, I guess, if you will.

I am curious about these things when like, I think Anthropic has like a proof of concept for Claude for lawyers where I'm like, that sounds insane to me. Utterly insane. I mean, we use, we use Claude Cowork for legal work at Sentry. Like, we get so... By a lawyer. I would hope so, at least. Oh,

yeah, yeah, by a lawyer. Oh, yeah, yeah, sorry. Yeah, yeah, yeah. I mean, again, once again, this is liability. Like, exactly. Choose if you want to go to jail. And I think if you're like a one person business. Okay, sure,

you're gonna cut corners. That's, that's always been true, right? So that doesn't change anything there. But if you're a one person business, you do not represent anything of substance in this world. And I think that's like a grounding truth people need to understand. Like, companies with,

at minimum tens, but mostly thousands of employees are actually the ones that represent the economy for the most part. And they're not going to outsource legal paperwork to a fucking robot, right? And so I don't know.

That's the disconnect I see nonstop from people. I'm like, it's like, just because you with your little hobby thing want to do this doesn't mean anybody in the sort of quote unquote real world is ever going to operate that way because we employ people to reduce liability.

Not because they're necessarily doing incremental value work all the time, you know? Do you think with LLMs and agents and you name it, the open source scene is changing? And I mean, to some extent, we see a change already in the amount of pull requests we have. Before this episode, an episode with Mario Tzechno will go out on Pi.

And he said, like, here's so many garbage pull requests now. But, and we see also a lot of open source project closing pull requests to non-known contributors. So how do you think that part of the industry is going to change? Yeah, it's definitely changing. So I don't know, it's weird because you would

always get junk pull requests. Now there's more of it. Now there's a lot of accidental pull requests and stuff like that. We get these all the time on our projects. It's like, oops, I meant to open this on my fork. And that I think is a worse signal.

And so this idea of, like, we have a shared problem. There's all these different things actually happening in open source. But it used to be, we have this shared problem. It is not our business. And there is no business around it.

We're going to open source something. And then other people had this shared problem. And software is hard to build. You'd rather, if you have the same problem, work with somebody else to solve it, right? Right. That should still be true. It is not true right now. And it is not true for

all these different reasons. One, everybody commercializes everything under the sun. We intentionally, I kid you not, I consider it my life's goal now to open source people's businesses that should not be businesses in the first place. Like this code review thing that I built. I mean, it's under FairSource.

I don't really care about that, but that's mostly because we have like a competing product. But I'm like, this is not that hard to build. It is not a product. Sure, you can sell it. That's totally fine. But what I mean is like, there is no inherent sort of technology challenge to build this thing.

There's no reason it shouldn't be open source infrastructure, right? And so there's a lot of stuff like that. So that's going on. I think there's a second thing where a lot of people are exposed to this and don't really understand it or don't value the collaboration or the community aspect.

Pi, I actually think is on the counter side where like people do value it and Mario's done a great job there. Thus, you get more of this slop. But yeah, we see very little. Everybody just DIYs their own thing now because, it's easy to get the V0 version out.

And so people do it and they sort of assume that's where it's complete. I spend this code review thing, first day, I had a prototype that worked and proved it worked, right? Three weeks I spent making the sort of interface design good and functional,

and what I would call like high taste or something like that. And I think people forget that that actually matters to the long-term health of a software project. Sure, you can slap together, but this is the same argument for SaaS. SaaS is not dead. It will never be dead because I want good software,

not some slop that I have to maintain or will just be broken or have all these other problems, right? And so. There's that. And then there's a third version, which is people sort of like theft, like IP theft, which is going to happen now more than ever,

right? It was already a thing that happened that, you know, sometimes you would see, sometimes you wouldn't see. I don't know. I want, I want somebody. So if somebody listens to this and you have courage and are okay with us wrecking your life, I want somebody to go

take an LLM, take our code base, which is licensed under the functional source license, and re-implement it and try to violate our license and claim that it's some white rule or what are they, Clean Room, Clean Room Solution. We will sue the shit out of anybody that does that, and we will win because

that's not how the law works. That's not how IP protections are going to work. And people, some... Again, people just live in this little hallucination bubble that they think things are okay because they have some conversation in their head with themselves that all of a sudden it is okay.

And there's so much of this kind of chunk. We had a company that was misled. This happens to us somewhat regularly where somebody will take our software and not attribute us. And a lot of our stuff is totally open source. Whether we like that people use it or not is a different conversation, but you can't, right?

It's MIT license or something, but you have to respect the license. Otherwise, we have lawyers, and we will enforce that license to no end. And all you got to do to respect it is put in the attribution clause. And we had somebody that didn't do it. And they tried to avoid it by like,

oh, we'll re-implement the code. I'm like, that's not how it works now. You already took the code and implemented it. You can't just re-implement it to remove the license. That's not how licenses work. Like, that's a clear breach. And I'm like, just add the attribution. What is wrong with you?

And then they did. Like, they did after like a second round of like, no, that's not good enough. But I'm like, what are you doing? Like these are legal constructs. You have to follow the law. And so I don't know. I hate it because my entire career is because, and the reason we do open source

and I still do open source, I don't care about the freedoms or any of this stuff like the FSF freedoms, but I care about the access to software and I care that doing this sort of build in public, but like this open source stuff, it gave me a lot in my career.

And so it's very valuable to me. I feel like giving back and things like this. And so it's kind of like obvious. And there's no, again, there's no business behind most of it. So it should just be. But the sort of, it does feel, it feels like it's dead.

It feels like the old version of open source, which was this, is never going to exist again. And some of that probably should die, like the little NPM packages and stuff. But I don't know, like.

Yeah. Yeah. Yeah, I don't know. That's probably like the, I'd say the least talked about most obviously something significant has happened to thing. And it's not just like the slop pull request either. It's just like the broad, again, and it's not just AI, but it's been over a course of like last decade

of like everything's venture funded now. Open core is way more present than open source and all these things. So yeah, I don't know. It's kind of sad, but my personal opinion is the old way of open source is just gone.

How's your stance on open source versus open core? You kind of alluded on that just a minute ago. I despise open core. It's not open source. It's not open source, but it's still an interesting approach. Yeah, the problem is my belief system is sort of do the thing that.

If I want to build open source, it's open source first. It's not open source second, even if I'm building a business out of it. That's not common for what it's worth, and that's an irrational way to build a business. Open core, always a business. And almost always, I refer to it as crippleware,

because almost always they're like, okay, we have to save all the good stuff for the closed source version, which you can never afford. The license is always absurd because it's some enterprise thing. And then usually the open source version kind of sucks. And they'll claim it's

like, there were all these versions of this where like oh well like you need more than one core or something it's like you have to use the the paid version or just some absurd limitations, and i just it's dumb,

it's like it's not and that's why i say it's not open source i'm like, cool you gave me junk free software and now i'm sure there's some that's that's not quite true but increasingly it becomes true as your thing becomes more successful, and anything of value is going to become more successful and you're going to

have a hard time monetizing it because it's open source already or some version of it's open source. Thus, you're going to find more ways to monetize it by making it less open source. That's how it goes 100% of the time. And we do the same. We no longer open source every day, mostly because people keep stealing our shit.

But even the core, it's like, we're like, well, how do we protect it? Because we don't want to actually water down the product you're getting. We want the product to be good, period, whether you pay us or not. But we need to be able to monetize it. And at very least, we need to protect

ourselves from somebody else monetizing it. And that's quite hard to do with open source licenses. Like, prey is not an answer, basically. And trademarks are not a solution. We tried that. And so our version was like this fair source thing, which is like delayed open

source, which is like after two years, it becomes, what I would describe as true open source, which is permissively licensed, which means effectively no restrictions other than attribution, right? And that, I think there needs to be more in that software or,

for things that are of sufficient complexity. And this is where my fear around open source, like, is Pi that? I don't know. Is it of sufficient complexity? Well, it's not simple. And it has good design. And that takes time and effort. But certainly databases are. Like, you're not going to, I mean,

people are probably going to try, but you shouldn't be, like, vibe coding a database, right? And so I've got to believe some of it has to find a way to survive and thrive again. But I do worry we're leaning way too far the other direction right now.

One thing that blows my mind is the level of entitlement that open source created in developers, where it's like, well, software is built by people that need to make a living like that parts can be altruistic, like, I don't know, some utilities or something is open source.

But at the end of the day, those are hopefully maintained by companies, because otherwise it's just an even worse form of exploitation of human labor. But I don't understand the level of cognitive dissonance where people are like, well, this should just be free.

Like, are you offering your work for free? Like, what are you talking about? Like, that doesn't make any economical sense. Yeah, I'm a very big believer, not in karma, but in some version of this thing where like, not everything needs to be an immediate reward. Like things need earned, you know?

And open source will earn your rewards. Like. I discriminate as probably the wrong liability here, but, I specifically look for people with public profiles on GitHub that have public works for a lot of reasons, not because of the open source thing,

but because it shows me a bunch of signals often that I want for people on my team, which is like, well, are they curious? Do they actually like technology? Do they want to build stuff? And you can argue not everybody can do that, and that's totally okay because

the jobs I fill, I need people that can do that. Yeah. And so that is, that's kind of what I mean. Like, you will find the rewards. Like, people value that kind of, like, every company values that, to be clear. Like, that's actually a high signal no matter what.

Now, it's a little bit harder to parse as AI slop or not these days. But I think, like, there's too many people that think you should just get, like, Everything should just be handed to you. Like things are easy, especially in tech for some reason.

And so I don't know. I worked hard for a long time in my career, did a lot of up at source. That gave me a lot of career opportunities that otherwise, there's no way I would have gotten. And here I am now. And so I'm like, see, if you just work hard and you do the

things that will build your career, you will get the reward from it. You're not going to get it tomorrow. And you don't deserve to get it tomorrow, you know? Well, it's also not hard if it's immediately rewarded. Yeah. I think Kotlin is, for me at least, a good example of how we do open source

very well. That's one of the things that I like is, yes, it helps us internally because we use Kotlin all the time and Java just wasn't great for our purpose. It also helps that we build a product around it where people that want to use Kotlin most likely want to use our product or the fork of our product in Android Studio.

So that is a good example where there's an immediate value for people. Hey, you have a much better version of Java for free. Here you go. While at the same time there's also multiple levels of motivations for us to actively work and keep investing in it so that's one example that i like and sentry is doing,

sentry is a great example for me because you have your code base completely public so every time when i need to check like a typescript quirk or performance issues i pull up the sentry code base because it's one of the biggest open source code bases out there i do wonder

like you know the other thing people don't talk about enough some people do but like, models come from human knowledge they're only possible today because of how much human knowledge has been shared but. Things don't stop like like what if nobody ever builds a framework again certainly

the ones that are out there are not perfect um especially in things like the javascript ecosystem where they're very rough at times, But, like, what if nobody ever tries to solve the problem better? Or, you know,

there's no more programming languages. You know, basically, something stops there. Like, the models are not necessarily going to create this stuff. Now, some people might try to create some of this with models.

But, like, I don't actually pay attention to a lot of technology choices now. Like, I'm not going down Google search and looking for the right library or... Even the framework, I actually just don't care. I'm like, cool, probably it's going to be React.

That's good because that's the one I would choose. But I'm like, is it going to DIY a bunch of routing? Is it going to use React Router? It's like the choice, I'm like, they're less consequential now, which I think is actually bad because it increases the brittleness of software.

And for what's worse, some runtimes have always had this problem, but my early career was Django. And the core value you get out of a fully baked framework like Django or Rails is things actually work.

And most importantly, they're implemented in a very reliable, high-quality way where I'm not having to stitch together everything or DIY everything. And LLMs are like the extreme version of this where it's like, no, don't even use libraries sometimes. Just reinvent a new pattern of it.

Now you've got to maintain that pattern and all this stuff. And I'm like, and that's always been bad in JavaScript because it was always glued together. And this is just worse. It just makes more brittle software, and... This was always a pet peeve of mine. I never understood why JavaScript operated this way.

Because the fact that people are still, like, generically across the board, DIYing some parts of authentication, and I'm not talking about using a third-party vendor. This is not a cloud service. But we still DIY auth all the time. We have some libraries that people use, of course, but the libraries aren't

just, like, dropping and they work. It's, like, dropping and then glue together lots of different things, and maybe it outsources some of the harder problems, but not all of them. Whereas like Django or Rails or something, and I'm sure this exists in the Java

communities too, it's just like, no, you just use this off adapter and it's solved all the complexity and it continues to keep the complexity solved. And I don't know, like people forget about the maintenance of software, I think. And it's serious, so.

I don't, I agree. So I grew up mostly in the JavaScript ecosystem, so I couldn't agree more. At the same time, I'm always surprised of how terrible the developer experience is with Django. I used it a couple weeks back and one of my colleagues recommended UV to me.

And I was like, okay, this is much better than what I've used before, but it's still terrible. Like, how do you live like this? Yeah, and I think it's like some of those were like, UV definitely like night and day better from what it was before.

But it's the same for like JavaScript. You'll go from like NPM to PNPM and you're like, wow, they solved a bunch of problems. But like, what if it stops there and it doesn't continue solving the problems? Like supply chain is the big one right now. Like what if nobody invests in that ever again?

Or the investment is a cloud service. And I have this big anti-cloud service thing because, again, I believe in this access to technology thing. And for core infrastructure, there needs to be open source alternatives and whatnot. And if we can't get on the same page of building, like as a community,

like collaborating on things that need solved as a community, it's just, it's bad. And I don't know. I don't know. I could go on this for days. Do you think we'll see convergence in tools and those things, though?

And we had this at the very early days of LLMs. There were at least statistics that these models generate better Java code because there's more Java code out there than Rust code, for instance. I think this got better from what I've heard. Not doing Rust a whole lot,

but do you see this being... And you said this the same, like a model would most likely recommend React these days. Do you think for that purpose, we see a convergence to a certain set of technologies where people are just like, well, that's good enough. I can roll with this.

Yes. And so I think some of it's good for what it's worth. We don't need to reinvent React yet again. But here's my thesis, because I'm dumb enough to not know any better. Um in my experience if a model has not been heavily trained on a thing like

models will only give you the right answer if you give them the right answer first right and so people try to do that via pick any jargon you want context engineering but like passing it the right stuff so like if i copy paste the right code into the model and ask it how to implement

a thing it will probably give me the right code back and obviously i'm like i'm simplifying it but, And then when you take the training set, it's just macro compression at the end of the day with a little bit of random number generator in the mix and then weights that do wacky things.

And so, and I look at all these experiences I've had. So like with those being roughly truths, because that's how LLMs were. And I look at these experiences I've had with like iterations of models, which are iterations of weights, right?

And changes in the datasets. And there's like these periods of time where like one that I always use as an example here is, no matter what, Claude models would stick the any type all over your JavaScript to work past concerns.

You could not work around it. The only way you can work around it, and I did this, is every single prompt, you would put in like, don't use the any type. And then it probably would not use it. But if you didn't put it in that prompt, in that steering prompt,

it would probably then just do it again. Like you couldn't put it in AGENTS.md or anything like this, right? I kid you not, I had like a Claude Code hook that injected this thing in every single prompt. And I think about, I use this example a lot because I'm like,

That was an issue with the weights and the training set. And the only way you could fix it, because the hook was not really a fix, the only way you generally fixed it was by fixing the model. You had to train it. And so that's one version of it. But then you look at other

things where, you know, you might look up docs, but it doesn't always want to look up docs and go to web search and stuff. It's going to use the lossy compression. It's going to use the old version of the code. Sentry is this problem.

Set up sentry logging. Good luck. Good luck trying to set up actual logs versus our error reporting. The model just goes haywire all the time. How do you fix that? You fix it by changing the weights of the model or updating the training set. And so where I'm going with this is I think if the data is not in the training

set and tuned, which is a constant chicken and egg thing, it will never work that well. And maybe fine tuning becomes an answer here. So, for example, if you had a programming language or a library that wasn't in that training set, it's just going to perform drastically worse.

And maybe I'm wrong here. Every piece of evidence I have suggests that this is truth. And where I usually bring this conversation is, I think training is part of the cost of inference. And it's not factored in the cost of inference today. Thus, the cost of inference is dramatically higher than we are led to believe. And it's already expensive.

And so I usually talk about this in the context. I'm like, what happens when these companies go public? Are we all doomed because they have to pay the bills? Because I think training has to be a constant thing for them to be reliable.

Or, I mean, fine-tuning can still be very expensive, to be fair. Or fine-tuning is going to be a solution to some of this. But that seems... I don't know enough. There's not a lot of evidence or prior art in the industry that says, like, oh, we'll have a model that's fine-tuned for Kotlin or,

you know, something like this, right? And it just seems odd. If that's the solution, I'm a little bit suspect. So we do that on a very, very small scale where we have like specially trained models for Kotlin that are really just for offline code completion.

So very niche use case where you can like provide a small model that is like capable enough to provide a user experience that users might want while working on an airplane. That is literally the scenario. Yeah. And I think they're great for that for what it's worth. There's all these like sort of,

It's an optimization at that point. It's an efficiency thing you can do. But like a macro scale, especially you think about libraries. I mean, that's never happened. You're not going to have a model for React. It's just, it's absurd, you know? The only place where I could see that would

be Google to push Angular or like Flutter. Don't give them any ideas. You brought up those companies going public, which they both publish their S1 or whatever it's called. I'm not super into the weeds of how these things. What do you think is going to change with that? Because I'm sitting here kind

of like, I'm wondering how this is going down. Well, if you're paying $200 a month for a plan and expecting to get $10,000 in compute, I mean, that bubble's already burst, except. OpenAI seems to still be targeting this growth. My theory, by the way,

on all that is they need the training data. It's not actually a pure growth technique because they're spending billions of dollars in subsidies. It's like absurd, right? And so you already should get comfortable paying token prices.

Despite popular belief, because the models are still highly problematic. They're not nearly as good as we'd like them to be. They're not going to get cheaper anytime soon, unless there's a compute architecture breakthrough. That'd be sort

of my, again, I'm dumb enough to not know any better. This is my analysis. But because I'm dumb enough, I also just look at like the signals you can visibly see versus, I don't know, pretending this thing is going to have some unknown curve or something. And so I just, I have a hard time believing, one, that the subsidies don't continue to drop.

Because the money's got to come from somewhere, right? And you can't raise unlimited money. And going public is kind of that stage where, you know, the money has to start making sense is what I would say. Now, this is an unheard of time.

So it's possible that they're able to continue to raise funds publicly by issuing more, you know, the shenanigans. But at least classically speaking, you go public when you no longer need to fundraise, generally speaking, like one more big fundraiser or something.

Now, I don't think that'll quite be true here. But either way, the costs right now to operate them are so much higher than the revenue. And again, you can claim that inference is cash flow positive, even if you're like fucking 5% margins or something.

But training isn't. And if you stop training today, your company is dead. Like the frontier, like you would not use any of their models if they stopped training. And so when you look at that, I'm like the only rational belief you can have, because math is still math and money is hypothetically finite,

is they have to get more expensive. Or we do something else, or we stop pretending they're going to solve all problems, and we focus on efficiency. But you already see this. It's kind of like this play right now where people are sort of optimizing.

Like the models, they kind of plateaued quite a while ago in terms of like sort of maximum capabilities, if you will, from running them in loops. And it feels like, Either they're going to get wildly more expensive because they're going to continue

to try to push the state-of-the-art for minor incremental gains, or there's going to be a come-up focus on efficiency plays, and they'll still get more expensive until or unless that gets solved. And I mean, it could be both, but I feel like the latter has to happen.

And I think it can't be like, well, maybe we'll figure out how to do local models better, because local models, they're not that functional. If you could choose between a local model and cloud or, OpenAI models, you would not choose the local model in any such like no matter

what like if they had the same trade-offs you would not choose the local model just performs drastically worse so so i don't know we'll see, and i know we see that's also where like people willingly buy in software because they don't want to maintain it don't want to host it etc etc etc that is the

whole thing again just that it's like 10x more complicated because no one understands how to host a local model properly. That conversation i i'm glad we're doing that kind of research because i think it's important And there are also very valid scenarios where,

again, like small use cases and stuff. But there's like, oh, yeah, I'm using, or we can fully host Qwen and don't need Opus anymore. Yeah, I mean, if you're not really doing anything, sure. But, I mean, Opus barely works.

Like, Opus is still not good enough. And I mean, I haven't used Opus. It's going to be the same outcome, though. It's still not good enough, or it's too slow or something. I think a couple months, it was a couple months ago, where Sam Altman...

Implied that the U.S. government could bail out the debts of OpenAI. What are your thoughts on that? We're in unheard of times in America with massive, visible corruption. And so, yeah, probably.

They'd be like, sure, we'll take ownership in the company and give you infinite money or some garbage like that. I don't care about these debates of, like, should things like this be public utility? I don't know. We have, like, a monopoly of, like, I don't, everything's a tradeoff,

right? I believe in parts of capitalism that force the competition, that force progress, right? And something like the government owning a thing usually does not create progress. And I think that's roughly universally true everywhere.

But at the same time, sometimes it does create more fairness. And there's always this balance between fairness and equality, if you will, and sort of innovation. And so, I don't know. I don't think that would be a good thing, though, if anything.

Look at France and Mistral. And I don't know. The company exists, but it sounds, I don't know. As a German, I want Europe to succeed in this and be successful with it. Germany does a lot of good research in that space, so I'm not shitting on I'm

sorry when you hear it. Yeah. On the counter side, though, China crushing it. Like, they're doing a really good job. It doesn't matter how they're crushing it, whether they use our data or not.

Everybody steals from everybody. It's not like, you know, my hands are cleaner than their hands, kind of thing. But they're roughly government-owned, like, if you abstract it far enough away, right? And so, but they also have a very different political ecosystem where it's like,

government-owned by the same government over a very long period of time. So, yeah. Which is not the U.S. and definitely not, I think, all of Europe. You brought up an interesting point, though, with everyone steals from everyone. Do you think there is a house?

I mean, like, no one can tell me otherwise that there are license violations in these training data. No one can convince me that that is not. Do you think that will be something that is going to be looked at, cared for, even, like, considered at some point, or is it just one of those things that we start to accept?

I think the ship has sailed and to be fair I can I probably more side with the argument of like if you had to solve for that problem these models couldn't exist I think. And I think that doesn't mean you can just do whatever and not care about IP. But I do think from the perspective of like, they do need training data.

So I'm like, I'm sort of like, yeah, forgiveness is warranted here. It's kind of messed up that they have infinite money and it's just enriching people off of that. And so there's no world where that's right. But I do think there's still, I don't know. I think there will have to be some

at least reconciliation for future concerns. I don't know. It's tough because it's really hard to license content in general. And obviously you have some industries where it's much more pronounced, like the media industry with film and whatnot.

But I mean, even for books, like if I go to Claude and be like, hey, give me a detailed walkthrough of the thought process of book XYZ, it could do that. Yeah, I think it's tough. I don't know what the, I don't think there's a solution to the problem,

to be honest with you, other than preventing them from infringing on IP as much as possible. But training, I think you're, you know, it is what it is kind of thing. You're not saying it's like, it's not morally right, but it is what it is. I have one last, maybe somewhat provocative question.

You, initially, I think you created the open source pledge. Is that a clear statement? You're at least heavily involved. Yeah, it came from like, that's my department. So yeah. So what you're doing there, I think is fantastic. How much of that is open source

goodwill versus brand marketing? It's both. It is a thing we do because we believe it's a good thing to do. And we justify it, which is mostly us just like hand-waving, pretending that there's sort of brand awareness that connects to our brand in

there or anybody that participates. Practically speaking, like a lot of this, it's like always questionable. And I think in this case. It's probably less true that there is brand awareness from open source pledge.

To be fair, we were already effectively doing it. We just wanted to codify it. And I'm like, what if we just go like social activism, like peer pressure some other people in doing it? And I wouldn't call it successful. Like there's a few million a year that are contributed open source that some

of that wasn't there before. So I think that's good. It's not the win we wanted. We were trying to get to 100 million. Maybe one day it will. Who knows? Probably not, to be honest with you. We're gonna keep doing it like we're doing a million this year um and i don't

know so so so like no matter what we'll keep doing it um and so i think from that angle this is 100 just like no we believe in doing this thing it's it's good for the world um. But you do try to rationalize why you do a thing. And we always reverse engineer the thing we want to do into a business case.

And so in this, we're like, yeah, yeah, it's some like brain marketing stuff. But like top down, we all believe in the thing. We're like, no, this is fine. It doesn't cost us that much money. Which is honestly the only way you can do it either is if it's like top down at that scale.

Coming to an end here, what would be one strong opinion about AI and development that you would go tooth and nail for with everyone? That it is useful and it also produces absolute garbage and there's no way today, today as of June 2026, there's absolutely no way to produce good software with

just prompting LLMs, no matter what, it's going to be, bloated, over-engineered, complex, Java everywhere, Java factory patterns non-stop you can't avoid it, but it just produces junk sometimes that's okay you know uh but it is certainly,

there's no uh putting the genie back in the bottle or whatever whatever the sayings are like like it has happened it is it is here to stay it will get better we'll solve some of the problems at the very least, but it doesn't necessarily mean it's a magic bullet to solve all the problems

either so i think i don't know i would go tooth and nail on this i think it's a reasonable statement but a lot of people would disagree so so i think it's you know controversial enough, do you think I think AI is a bubble. 100% a bubble.

There's a lot of bubbles. Like crypto, I think, is no longer a bubble, but might still be a bubble. Doesn't necessarily mean it pops, though, is the thing. That's true. That said, there's a lot of money in a lot of companies that there's no way

the investors get a return on. And it's unclear if there will be a, like, if something happens, say like OpenAI failed. I don't know what happens. Like, that would cascade. Or Anthropic failed. That would cascade. There's some of these smaller ones that people are not super overleveraged in,

but there is a lot of money out there that should have been debt that is venture that is done on low... Like, basically, oh, I burn a lot of money because I'm burning on inference, which is a pyramid scheme, by the way, just like a couple people make all the money.

But it's like, oh, I spend $10 million a month on inference for my, you know, $2 revenue product. Thus, I need to raise $100 million. Oh, because of the way legacy stuff works, we should take, you know, 20% ownership in the company, but they need to raise, you know,

$100 million. So it has to be valued at this. We make up an arbitrary valuation, even though it's not grounded in anything. It's just grounded in spend. And there's FOMO and competition. It's like, oh, we can't even get 20%. So we'll give the absurd valuation. We'll give it slightly more and we'll take

less because they still need the same amount of money. And so it's like, oh, we'll get 5% and we'll give them a billion-dollar valuation or something. It's like, what? There's no math involved. There's no sanity involved. And that will work for some people.

You know, obviously, Anthropic and OpenAI, hypothetically, high value, worth lots of money. A lot of them, like thinking machines. I don't know. There's a lot of money in that company that was done at very low valuations. And when you look at this, it's not how Venture has ever been done before,

at least in my lifetime, when I've been involved. And a lot of these should just be debt. Like, if all you're doing is spending money, you should have to loan that money. You shouldn't just be like, well, I only had to give up 5% of my company even though I'll never generate

revenue ever in my lifetime. It just, it's so absurd to me. And it's more absurd because of the scale of the money. And so... If something happens, it's because the financial mechanics make no sense of the industry right now. And so as an, I hypothetically am an active angel investor. I have like 100 plus investments.

I have more or less attempted to stop investing entirely because it makes no financial sense anymore. Like I cannot make a bunch of investments and possibly expect to be successful as an angel right now, even though I actually have pretty good network access,

which means I could have good deal flow. The numbers don't add up to me. And so, again, it's not my domain. I don't care about it. I don't spend my time doing it. But that's my take on the system. It just seems very unhealthy.

And it seems like the most unhealthy. You're a lot closer to that than I am. So I'm very curious to hear your perspective. Thank you. That was fantastic. I had a great time. Thank you so much for joining me. Yeah, yeah, this was fun. Yeah, thanks for having me on.

How To Ship Real Code With AI (Not Junk) ft. David Cramer

Jan-Niklas Wortmann

Key Takeaways

Chapters

Mentioned In This Episode

Pull Quotes

Guest Bio

Full Transcript

Code Isn't Free - Mario Zechner on the Hard Truths of Coding With AI (creator of Pi)

A Google Engineer's Honest Take on AI, Angular, and the Future of Your Career

The Timeline Is Gaslighting You About AI

Key Takeaways

Chapters

Mentioned In This Episode

Pull Quotes

Guest Bio

Full Transcript

Related Episodes

Code Isn't Free - Mario Zechner on the Hard Truths of Coding With AI (creator of Pi)

A Google Engineer's Honest Take on AI, Angular, and the Future of Your Career

Microfrontends: Cutting Through the Hype and Misconceptions (w/ Luca)