Event Sourcing - Talk Python Live Stream

Added: 2026-05-12

[00:00:02]Hey Chris.

[00:00:04]>> Hey there.

[00:00:04]>> How you doing? How's it going?

[00:00:06]>> I'm well. How are you?

[00:00:07]>> Oh, good. Good to hear it. I'm happy to have you back on the show.

[00:00:12]So, welcome back. Welcome back.

[00:00:14]>> Thank you.

[00:00:16]>> Yeah. Last time we had you on the show, you're part of the panel around data star and that was cool. Now, we're going to talk about event sourcing, but we'll find a way to tie it back to data star in just a a bit. I think I see it on the horizon or in the show notes.

[00:00:32]>> Sounds good.

[00:00:32]>> One of these.

[00:00:34]>> Yeah. Well, not everyone listened to every episode. We got new listeners coming all the time. So, give us the quick intro. Who are you, Chris?

[00:00:41]>> Who am I? Let's see. I am a Python developer for about 20 years and a longtime listener to the show. So, it's a a very big privilege to be on finally my own self.

[00:00:52]Um, and by the way, great job continuing going, Michael. you're constantly putting out great content. So, I really appreciate all your work.

[00:01:00]>> Um, but as far as me, I learned the program as an adult. Uh, friend suggested I learn Python. I hitched my wagon to this this engine and I've loved it ever since. Um, I uh was a technical coach for a little while. I started uh the Python group here in Richmond, Virginia, PI RVA. So, if you're local, come out. Uh, in fact, we're just two weeks away from the next meeting.

[00:01:25]Awesome. How frequently do you have meetings?

[00:01:27]>> Once a month now. In fact, if you look behind me, you can see all these blue dots are the meetings. So, >> okay. Excellent. Keep planning.

[00:01:34]>> What are some of the kind of topics you all have?

[00:01:36]>> Oh, man. We range uh pretty much whatever people we can figure out that people are interested in. Um we've had a number of AI discussions. In fact, those have been really powerful. We've kind of line up the chairs in a big circle and just have a discussion. And it's really incredible. you know, we you have people on the whole spectrum of uh of of opinions about AI. So, uh yeah, it was very very that was one of my my favorite uh uh I was going to say episodes, one of the favorite meetings.

[00:02:11]>> Um >> I think we might bookend this podcast episode with little AI at the start, little AI at the end.

[00:02:18]>> Sounds good.

[00:02:21]>> Cool. So yeah, people just go attend that. Um, and I guess probably a meetup.com is where they find you.

[00:02:28]>> It is. Yeah. And you can go to pyrva.org to uh get to the meetup page.

[00:02:33]>> Easiest way because you could find a lot of crazy stuff. You probably find people like with pet pythons that want to meet up in Virginia as well and it's not the same thing.

[00:02:41]>> Indeed. Yeah. Thankfully. And we are gosh over 10 years old. So we are one of the uh first ones that pop up just because of age I think sometimes. Uh although honestly some of the newer ones pop up ahead of us sometimes too. So you know >> yeah it's it's such a contentious topic this AI stuff you know talking about your meetup.

[00:03:02]How did that go? Were people frustrated?

[00:03:04]Were people excited? Like >> yeah it was a kind of the whole spectrum. You know I felt like it was great because everybody respected each other and there were I'd say two people on on each anchored on each of the sides. you know, two people who actually maybe there are three people who, you know, have bought their own hardware and are really going deep into AI and, you know, encouraging people to like really dig into it. And we had two people who were very AI skeptic and very, you know, worried about the environment, uh, you know, all sorts of different things. Um, and so I thought was a very healthy, very good discussion. um you know I walked away with you know a healthier respect for both sides um u some ideas that I incorporated into my work especially now that the company I work for uh gives us a mandate to to use AI to write every piece of code so that's been a very fascinating uh transition for me um but also gives me you know a little bit of my agency or a little bit permission to experiment and see what what what to do you know how to how to function in this new paradigm.

[00:04:18]>> It is engineering and we we'll talk about it more later, but it's quite wild that your company's that is the the position. I don't necessarily think it's the wrong position. I think it might be the right one, but it's I think it's got to be brought on from a from a this is a skill you need to learn, not just throw stuff at the chatbot and now that's your job. like these are not the same things, you know, but I think a lot of people do, especially when they're getting started, treat them as the same thing and then say it doesn't work and then they're frustrated.

[00:04:49]>> Totally. Yeah. Yeah. Um Yeah. I >> All right.

[00:04:54]To speak further about certain things, but yes, I agree.

[00:04:58]>> Absolutely. Absolutely. All right. Well, we'll we'll come back to that and and get there, but let's talk about a little bit of this event sourcing thing.

[00:05:07]>> Yeah.

[00:05:08]>> What exactly is event sourcing?

[00:05:12]That's a question. Okay, cool. I wasn't sure if you had more. I saw your >> No, no. I think I when you when you reached out and said, "Hey, let's talk about this." I'm like I So, let's take a step back.

[00:05:22]>> Okay.

[00:05:22]>> Design patterns, refactoring ideas, like all of these architecture. I love to talk about this stuff. I love to think about this stuff. I think also maybe that's a why one of the reasons I'm not super frustrated with the AI things because I will tell it I want you to use like you could say like I want to build a system with event sourcing and it's going to work this way like for me the fun is like oh I got to build something event sourcing it clicks together with this and it does that and like yeah I didn't have to do all the little checks and details and like the file IO but whatever like I can skip that and build just like think a little bit bigger and a little more uh more big building blocks So, I'm a big fan of design patterns and paying attention to what you're doing. I can tell that you are too.

[00:06:07]>> Yes.

[00:06:09]Especially event sourcing, I would say.

[00:06:11]Um, so I have been following the topic of event sourcing for over a decade. Uh, I listened to a podcast in with a couple uh developers in the PHP realm and they were talking about event sourcing and it just inspired me. Uh specifically the things that I really loved before I was a programmer, I was a graphic designer and so creating websites with exceptional user experiences is something that just makes me excited.

[00:06:40]And uh at the time I was working at a creative agency, building websites that would just slow get slower and slower and slower the more data we put into it or you know the more we configure the routing and all or the the navigation and whatnot.

[00:06:55]>> Yeah. And so the thought of never having a slow page load again was intoxicating to me.

[00:07:04]>> Yeah.

[00:07:04]>> Um however that was in a portion of my career where I was had so much imposter syndrome having learned to program as adult and I just felt like I couldn't suggest to my lead developer we should lean into this. Um, and even when I was a lead developer, I just didn't feel confident that I could lead a whole team into like a a re a redesign of the code.

[00:07:30]Um, but a couple years ago, I got laid off and I was like, you know what, I've been wanting to explore event sourcing for 10 years. I'm going to do it. And it turns out it was a lot easier than I thought. Um, but anyway, all this to say, let me let me define event sourcing since I've kind of danced around it.

[00:07:46]Event sourcing is how has to do with how you save data to the database.

[00:07:52]Uh the best way to contrast it is most apps are kind of CRUDbased, right? So you uh create a table like let's say you're a shopping cart uh application.

[00:08:02]You have a table called carts and you have a bunch of columns and one of the columns has a a data for the uh product ids that are in the cart. And so if you have that kind of situation and you have one um user who adds five items to the cart then removes two and then checks out and then you have another user who adds just three items to the cart and checks out. Well, if they check out at the same time and you look at the two database rows, they are very similar.

[00:08:35]You know, >> they're the same user basically. They get put in the same um cohort from your marketing side, right?

[00:08:42]>> Yeah. you know, they they both all both of them checked out exactly the same products and if you look at the database, you'd have no idea that one of them removed two items from their shopping cart.

[00:08:52]>> Mhm.

[00:08:53]>> So event sourcing um so so the reason that is is because CRUD based you know applications will mutate state in the database to ensure it's always up to date with the current state and optimized for data integrity. Event sourcing on the other on the other hand captures every change that happens within the system. So you would have an event of like cart created, Adam added to cart, Adam item added to cart. You'd have five item added to cart events and two removals. And so you could have the whole history of how each user interacts with your system in database. That is essentially the core of event sourcing.

[00:09:37]Everything else is actually adding more design patterns onto event sourcing which is really wonderful. But that is what event sourcing is >> right. So your database sort of becomes almost an audit log. It feels very uh source controlish. Feels like git, right? Like what you store is the file and then the diff then the diff then the diff.

[00:10:00]>> You get back to the file by running all the operations on it potentially.

[00:10:04]>> Yeah. Exactly. Yeah. That's exactly what it is. And and it's funny because like I don't know about you, but like when I first heard about this, I'm like, "Well, isn't it slower?" Because it just seems like you're doing so much more work.

[00:10:13]Every time you're updating the cart, you're pulling out all the events from the event store and building up the state of where it's at right now and saying, "Okay, you know, does that cart exist in the that does that item exists in the cart? Can I remove it? Okay, let's remove it." You know, and it turns out computers are fast.

[00:10:32]Um, and so it's negligibly slower depending on the query. Um, yeah.

[00:10:38]>> So I saw you and Bob Beler both talking about this on YouTube and that was one of my first thoughts too is like this is super cool but it kind of sounds a lot slower to answer questions.

[00:10:51]>> Yeah. And I think but then I thought about it and I I thought I think there's actually a decent amount of stuff that you can do to make it quite a bit faster. Right? So let me throw some ideas at you and then you tell me how they land as somebody who's actually done this stuff.

[00:11:11]>> First of all, back to your comment on computers are fast.

[00:11:15]>> Computers are so fast. They're so much faster than people realize how fast they are. And databases are fast, too. If you put indexes on them then they're so much faster. Like I just don't understand how websites are slow.

[00:11:30]It's just it's un it's not just oh I'm a little frustrated. It's like, how is it possible that somebody built this and accepted that it takes 3 seconds to load this page and yet they do and you just know that it's it's most likely that there's not a database index somewhere or maybe on an extreme situation there should be better caching, but it's just like so like when done right, you're right, it's absolutely blazes, right?

[00:11:56]>> Yeah, absolutely.

[00:11:58]>> But how how you could make it faster? So a couple of thoughts that came to mind.

[00:12:04]One was I have three. So one is you could have a operational database which has for a particular user it might have the five ads through two removals and that's their shopping cart. And the way you get that is either you query it and in code you add it up or you do some kind of aggregation thing that says get all these things and then somehow plus minus them together. Right. Yeah, >> probably in the shopping cart it feels like you probably need to actually pull them back, but still you could do that super quick and then just run that bit of code. And every time you make a change, you could also write that to a second table, second database that doesn't have the it just has the current state, right? I just have three for that user shopping cart. That sounds okay to me, but it's I mean databases hate duplication. They that's like kind of their third normal form job.

[00:13:01]Do databases hate duplication or is it people that don't like it?

[00:13:06]>> Yeah, that's fair. That's fair. I mean, they were built to avoid duplication, right? So, in that sense, >> so but another, you know, another possibility would just be use something like Valkyrie. You could I could have said Reddus, but I'm a fan of Valky over Reddus. I find this is like a little bit nicer project. Um I Are you familiar with Valky?

[00:13:26]>> I am not, and I'm taking notes mentally right now.

[00:13:29]>> Yeah. Here we go.

[00:13:31]So, Valky, if I find its repository, which is hiding down here, it it describes itself. Where does it describe itself?

[00:13:41]So, a fork of the open-source Reddus project right before the transition to their new source available but not open source models. So, if you have say the Reddus Python library, you just tell it that it talks to this thing, this endpoint, this IP address and port and it thinks it's Reddus, right? but it's like more open source friendly. I'm going to star it for the world. So, and it's got 26,000 stars, right? So, it's it's pretty popular. Not that it's totally really relevant exactly which version this is, but just having a cache that has that information of what is, you know, shopping cart is three period.

[00:14:22]>> Yeah.

[00:14:23]>> And put something just put two pieces in place. One, when you do a query, first you check the cash. If it's not there, you get the the playback, you compute it, and you put it in the cache. And if you make any change to that thing, then you invalidate the cache. Right. Those two things would give you basically seamless ephemeral answers to the questions that if you ask them very often, you get super quick responses, right?

[00:14:46]>> Absolutely.

[00:14:47]>> Mhm.

[00:14:48]>> Yeah. And then the other one is if you you know, I always find like these extra caching servers are honestly not necessary. Like you know, we already talked about fast. So I'm a big fan of discache which I had visit warmer on to talk about and discache is awesome. So you could just have like a local file store on your docker image or volume or whatever then you get the same thing but you don't have to have the infrastructure right there's there's different options.

[00:15:15]>> So basically I guess to sum this up I'll throw it to you now operational database that is also has the answers or some kind of caching story. It could be inmemory. It could be a ser key. It could be the disc cache.

[00:15:30]Whatever. What do you think about those?

[00:15:32]Are you exploring them?

[00:15:34]>> I am exploring two or some essentially.

[00:15:37]Yeah, I guess you could say all three in a way. Um I'm I'm experimenting with NATS uh to do kind of the I guess Reddus kind of side of it. But all that to say is um all three are viable options depending on your use case. Um the first thing is like just the event store itself like you um getting the current state of any individual item should not take you know essentially should take milliseconds if if that long. Um one of the so there are two people that I kind of who are my north star for all of my event sourcing uh knowledge. Uh they are Martin Dilar and uh Adam Demetri.

[00:16:18]And Martin uh wrote a book called Understanding Event Sourcing, which was huge to helping me um go from somebody who created an event sourcing project to adopting event sourcing as my kind of default strategy. And in in the book, he mentioned that he will tend to use the event store for uh essentially like if you start hitting 2,000 events in a stream, then he'll think about optimizing it. uh you know changing uh going to one of the alternate uh approaches that we just mentioned. So like that's really impressive. I he uses uh Java or one of the Java derivative languages. So chances are it's slower in Python and so you need um you know that we need to adjust that number for Python. But honestly we should have shorter uh event streams anyway.

[00:17:10]>> 2,000 seems like a lot.

[00:17:11]>> It does.

[00:17:14]>> Um >> seems like a lot. And so kind of the first fallback for me would be the what what um Martin and Adam call the read model which is essentially uh one way you can make a read model would be the database backed read model where you it you have code that subscribes to only the events it cares about and so whenever a event comes in it'll incrementally update that database cache or your file cache or reddis cache or whatever. Um and then I guess the third thing that I appreciate is you know using like Reddus or in my case NNATS is like when you have a you know front-end uh a very high frequency high updating um you know web UI or something that you want to really make sure that the user has up-to-date information I would lean towards that having ways to push down to the to the client.

[00:18:05]>> Okay.

[00:18:06]Yeah. Anytime you've got a live stream seems perfect, right?

[00:18:10]>> Yeah. Yeah. Absolutely.

[00:18:13]>> Yeah. I mean, hook up some JavaScript or hook in some textual or whatever it is you're trying to do and just say or just rand, you know, arbitrary web sockets or service and events and just say when this changes, send me the delta and then we'll adjust.

[00:18:29]>> Yeah.

[00:18:29]>> Right. Which is pretty pretty nice.

[00:18:31]>> Yeah. Yeah.

[00:18:32]>> So, there's a book that you recommended um by one of the guys you mentioned, Martin Dillinger.

[00:18:39]>> Yeah.

[00:18:39]>> Yeah. Tell people about that real quick.

[00:18:42]>> Yeah. This is this is incredible to me.

[00:18:45]Uh he realized he needs there was a gap.

[00:18:49]One of the biggest problems with event sourcing is that the material is sometimes there's gaps in where there's good material. And because event sourcing came out of the domain driven design community, there's a lot of jargon that you have to kind of get through. And um >> they do like jargon in the DDD space.

[00:19:08]They really do.

[00:19:09]>> And it once you understand it, it makes perfect sense. But yeah. Yeah. Getting onboarded does does take >> get your bounded context working and then off the races.

[00:19:18]>> Yeah. Exactly. And so, uh, Martin, um, essentially saw this gap in in in knowledge and was like, I need to fill this with an ebook. And I can't remember, I want to say in like two months he wrote this thing. And it's so amazing because it introduces uh the way that the two of them work. like they both independently kind of came to similar conclusions which is to use event sourcing as your base but to also leverage two other or one main other um or whatever a couple other patterns.

[00:19:49]vertical slice architecture, uh, CQRS, which is kind of that idea of like having those read models ready to to, you know, optimized for you to to download and and use. And then using a documentation technique called uh event modeling diagrams. And that that is a huge key too because as someone who has uh been on a couple teams to uh do the event driven transition to try to really help you know do more asynchronously you need to have a good communication pattern to keep everybody up to date on what does what and I find that uh all three especially event modeling diagram they have refined this to make it simpler and simpler and simpler to the point where there's really just a few elements put together and you can understand the whole life cycle of a of an application.

[00:20:38]>> Yeah. Cool. I'll link to the book on over on Amazon. You know, another before before we carry on, I I thought of another sort of optimized scenario.

[00:20:51]What about a document database like Yeah, your top level elements are just like the computed fields like total lifetime value or you know cart value or cart count item count but then have like maybe a cart item events which could be a a nested list of act of like rich you know many documents that are like item added item added timestamp value like category all and actually storing them in the same record.

[00:21:24]>> Yeah, I haven't tried that, but I think it's a really interesting approach, you know.

[00:21:30]Um, I actually use a document database as my fire store as my as my event store. Um, and I I haven't really kind of dug into like kind of the optimizations I could do. Uh, but I find it curious because like the I mean what you're suggesting is slightly different than how I think about it because it sounds like you have like a for lack of a better word a model that you're storing all of its events in as well and some of the neat some of >> it's kind of inside out of what the real design pattern is. Right. Well, it's not so much that as much as um one thing that I have found interesting is, you know, pimps came out of the domain driven design group. Um everything is about an aggregate what they call aggregate. Many people call it a model and so you you know set up boundaries of this is your shopping cart and these are the events that modify the the shopping cart. What has been a new movement in event sourcing is to uh essentially be model less. Is to like focus on the events themselves because they are so flexible and so many times we as developers can kind of create boundaries around what we think are the models >> that >> but the models change.

[00:22:47]>> Yeah.

[00:22:48]>> Yeah. And coupling is like the hardest >> the bounded context as I would say actually changes because the the problem you're solving might change and the models don't match.

[00:22:57]>> Exactly. Yeah. Yeah. So all that to say is I don't not that I I want to especially say like I don't think that that's a bad idea. I think it could be really fascinating. Um especially as like a secondary approach because you know well whatever you know like I one of the things I really find fascinating about this is this is such a flexible pattern that people I mean they've done so many different ways of optimizing for their event store or anything like this.

[00:23:25]So I think that's a very much a valid approach. It's it's the reason it came to mind is you can atomically update documents and therefore you could atomically update both the the computed value and the series of events as a single action >> which is interesting you know.

[00:23:42]>> Yeah absolutely very interesting >> and and they kind of that that one thing becomes the source of truth for what you're tracking I don't there might be something there I don't but it does sound a little bit too focused on the model I do think >> it's worth experimenting with for sure.

[00:23:56]up. So, just to throw out a little street cred there, look at this.

[00:24:00]Purchased April 18th, 2005 for domain driven design by Eric Evans.

[00:24:08]So, this is kind of the the greater space, right? Maybe >> the book is called domain driven design, tackling the complexity in the heart of software. It's pretty interesting. I think it's kind of the follow on of the refactoring movement that Martin Fowler and all those folks were working on like in the late 90s early 2000s.

[00:24:29]>> Yeah.

[00:24:29]>> Kind of in that space, right?

[00:24:30]>> Yeah. I must say I haven't bought that book. The closest I've come is the uh cosmic Python book or the architecture patterns in Python book that Harvey and >> oh I always forget the other guy's name, but that's such an amazing book. So, that's closest I've come to DDD.

[00:24:48]>> Okay. Cool, cool, cool. There's u a video. I think I think you gave me this video, right? Eventing explosions in football.

[00:24:58]>> Yeah.

[00:24:59]>> And this looks like foosball type football, not American football.

[00:25:04]>> Kind I love American football, but I do believe it's slightly misnamed. Like you don't use your feet that much other than the rug part.

[00:25:11]>> Yeah, true. that >> I mean it's like calling like Formula 1 like foot car because you make the car go with your foot, but like it's not really the main thing of the sport anyway.

[00:25:24]>> Yeah.

[00:25:27]>> Yeah.

[00:25:27]>> Yeah. Okay. Tell tell people about it.

[00:25:28]I'll put it in the show notes.

[00:25:30]>> Yeah. This is um you know like I feel like event sourcing it's like one of those things where it's hard to explain until you get it and then it I feel like it's like a good board game. You know, if you have a a board game band in your life, they're like, "Oh, this is such a great board game. It's so simple." And then they start explaining it and like half an hour later you're like, "Are we ever gonna actually play this? I don't know that I want to anymore." But I felt like the this person has done a really good job of kind of really distilling it down and showing like why it matters.

[00:25:58]Um, and it's a 10-minute video, you know, five minutes at 2x. And it's it's really kind of charming. Really good.

[00:26:04]Really well.

[00:26:04]>> Okay, cool. Yeah. Yeah. I'll put it on there. Can you watch videos at 2x? Are you all the time?

[00:26:09]>> Yeah.

[00:26:10]>> Gosh, >> my daughter does that. I'm like like how do you actually take it in? I just I'm a oneex sort of person.

[00:26:16]>> Do you slow it down or rewind to to pull in things? But yeah, see it's like a like a seek and then focus sort of deal.

[00:26:24]>> Exactly. Exactly.

[00:26:25]>> Uh not that one. This one. I also heard there's this ebook I can get. What's up with this?

[00:26:32]>> Yeah. So, scheduling this podcast episode, I I didn't know, you know, what we're going to talk about, and I know like there's going to be things I forget. And I feel like part of the reason I am so excited about this is I know that there's someone like me, the 10 year ago version of me, who has heard about these things and is curious, but just needs more information. And so, uh, I've spent the last couple of weeks creating this ebook. And, uh, I I put the first version up and, uh, I'm going to continue to improve it as time goes on.

[00:27:06]So, uh, if anything in this conversation is interesting to you when you just need a little bit more and you want to understand a little bit more what's going on, absolutely download it. And, uh, you know, it's free, so why not?

[00:27:17]>> Yeah.

[00:27:18]Cool. And then we'll have to make you do a uh, an audio version. Put that on Audible or something like that.

[00:27:25]>> Sounds good. Yeah, sure.

[00:27:27]>> I always want to do audiobooks for stuff that I'm working on, but just the concept of trying to speak code or config file. I just like, okay, I got to stop, you know? It's it's tough. It's a tough balance to do with audiobooks and tech uh like developer stuff, but still, >> for sure.

[00:27:42]>> Yeah. Cool. All right. So, people can check that out. It's for free. We'll put that in the show notes. Now, I think that this both has really positive or really big possibilities for data science, but also potential challenges. Let me throw it out to you and then you you take us through it.

[00:28:01]>> Super benefits. Incredibly obvious. You have an event stream that tells you over time what times the things happened. you have um both the additions and subtractions or the permutations that it goes through until it ends up in its final state. Not just show me all the customers from California who bought this month, but like show me all the Californiaiforns who and abandoned the cart but then came back and then did the you know what I mean? Like you can just answer way more interesting questions.

[00:28:28]You got time series.

[00:28:31]On the other hand, maybe I would just want to load up a Pandanda's data frame with the answers of what's the average cart size during checkout and that that becomes like a big computation out of an an event source based database >> if you don't Okay, let's hear let's hear I'm just say like if you if you don't do one of those >> caching or multi- database things or the the QRS I don't remember what the pattern is. Anyway, that looks like it maybe is a little bit of a challenge.

[00:29:05]You could do it I think you could do it more easily in pandas, but like maybe I just you know some people do data science just through SQL and they just I'm just going to write queries against a warehouse database you know.

[00:29:16]>> Yeah, absolutely. Why not have it both ways? Um for example on my okay so let's start uh the for the hardcore data scientists actually you know I don't even think the event store is the right format for them I would definitely have some kind of script that would run on some kind of uh um loop that you know maybe every day or every couple hours or whatever would transform the raw events into some format that would be great for >> sure and you hear about all these like OLAP cubes and all these other like super um BI type of systems. None of those not not I can't say none of those.

[00:29:54]Many of those are not running out of the operational database. They're like a some kind of like warehouse data lake.

[00:30:00]We've transformed this so you answer questions. So it's not necessarily just event sourcing like we just want to avoid five joins so we can just ask the question directly. Right.

[00:30:08]>> Yeah. Yeah. In fact, my current project uh that I have in pro in production at work uh it is a service that multiple other services use to process items. And uh the project manager of one of these services reached out to me and said that they have a big query table that has all this analytical information and they wanted to add the information we have to theirs. And so, you know, we set up a conversation. I created the code and every day I'm sending information to their BigQuery instance. And three days after we did a go live, you know, I created a meeting to like kind of circle back with them to make sure everything's working the way they wanted. And uh when they opened up the BigQuery database, they were shocked because they were expecting three days worth of data. I had I set every piece of data I had for months, which was how long they've been sending things to my uh service. And so like I was just I just you know this person was elated because they were like they knew their data scientists wanted this information and they now they have all this information going back to day one so to speak. Um, and also just recently my boss asked me like I have a a reports view that's just a web page that has like how stats on how the my service is doing and he's like it'd be nice to have like some like a table of some of these you know last few days or whatever and I was like okay how many how many days would you like and he's like 30 days. So I created the HTML was pretty easy and I just created a script to like pull the events out of the event store and populate this table um you know as exactly as we needed. It went into production live and we were immediately there with 30 days of history. It was it was so exciting and and like this is what I get to experience every week is like, you know, having the ability to like go back into history and answer questions that we've had that we didn't even think we knew.

[00:32:04]We we didn't have any idea that we would want to know, you know, a month ago. And to be able to answer those questions with precision is is intoxicating.

[00:32:12]>> Yeah, I certainly see the value. like you don't necessarily know the questions you're going to ask. And if you don't have enough data or you don't store it in the right way, you literally can't answer them.

[00:32:25]>> Yeah.

[00:32:26]>> Right. But with it sounds like with events sourcing, you can go back and like, well, what if we ask this over time instead of by region? Like, okay, slightly different query, no problem.

[00:32:35]>> Exactly. Yeah. Yeah. It's it's really quite something. I I had no idea like my I I kind of mentioned earlier my biggest thing was I I can't wait to have fast UI and now that I realized that I feel like our applications obviously serve the primary purpose of whatever it is that the business needs but I didn't realize how much there is a secondary need of understanding how it works and enabling the business to make decisions based on how customers are actually using the application.

[00:33:07]We're going to talk about the AI side later, but I do just want to throw out as different constituents who might care to answer these questions. Like I was just thinking, you've got you've got the operational side of say the website or app or you know driving an API for the app or something like that. That's one view. That's kind of the traditional view. But now you have this much more increasingly popular view of like data scientists and BI tools and the CEO wants a dashboard that updates live type, you know, so events are a clear trigger for those kinds of things, right?

[00:33:42]>> Absolutely. But then also you might ask your AI Opus or Codeex or whatever, hey find me some trends or let's look at this and you know it has more to work with as well, right?

[00:33:58]>> Yeah, >> just think of the different constituencies. Yeah, >> totally. Um, in fact, just today I was uh looking into a bug that was happening in production and I asked Claude, "Hey, can you query the the GCP logs? can you query the event store and help me understand what's going on? And it was like sure enough, here you go. And made made fixing the bug much easier.

[00:34:20]>> Yeah. I guess you know why you have more granularity on what if the thing in the database doesn't look like you expected.

[00:34:27]You much have a much more granular way of knowing like it was this step that made it look like that. Because I've had problems before where I completely upgraded, change, swapped out the data access layer for Talk Python training for the courses.

[00:34:44]>> Yeah.

[00:34:44]>> And for the website, it was perfect.

[00:34:46]Everything was great. But under certain circumstances on Android, the app was resulting in something. It was sending in something that would make the data not right. Right. Like there was some field that was null instead of just taking on the default value.

[00:35:02]>> Oh man. which is fine, but then when the person logged in on the website, the website didn't assume that that thing could be null because it was at a minimum had a non-nullable default value. I'm like, why do we need to check that for null? Like, it should how did it get to be null? It makes no sense. It took forever to figure that out.

[00:35:20]>> Oh, wow.

[00:35:20]>> But with event sourcing, you could see, oh, this was the event that made it null. Not just it is null. What in the world is going on? Why could it how could it possibly be null?

[00:35:30]>> Yeah, absolutely.

[00:35:32]So I I think it's got some interesting debugging sides. And one more thing like I know this is not quite the data science side but another constituency could be um PCI, HIPPA, GDPR like all the compliance frameworks >> you got to deal with >> where auditing or a sort of audit trail or something happens. I mean a lot of times logs serve that value but that might be a they updated the record like to what >> yeah totally yeah and that you know I even though I've been in insurance and I've been in healthcare I haven't um had anything where I have to certify these things but like you've the audit log is the way you interact with everything it is the um source of truth and so uh but what's funny is I have worked on teams that created history tables to try to essentially do that work and yeah, >> it was like two or three months after I started working there before I learned that that table existed. And so >> there were two or three months of work I should have been putting in the history table that I didn't. And from what I hear among other developers, a lot of teams work that way. Like only a few people really know and understand how to maintain that history table. And a lot of times like when they try to replay it, it just doesn't work. And it's it's unfortunate. Yeah. It's like, well, there's history in the history table, but when we run it again, it we don't get the same output as the final database. What's going on?

[00:37:04]>> Yeah. True.

[00:37:06]>> Yeah. But with the event sourcing, it reverses it. Basically, the events are the source of truth. And the other one is some kind of dynamically generated sort of deal. Yeah.

[00:37:16]>> Yeah. Yeah. And it's a lot like, you know, a back backup strategy. You know, if you never test your backup strategy, >> you don't have backup.

[00:37:25]>> Exactly. And I feel like the same thing with the history table. And honestly, to be totally honest, event sourcing is similar in that it's easy to accidentally migrate event versions. You know, like for myself, I uh I was working on a new event um to kind of, you know, on my app and I introduced a new attribute or actually I guess it was a full whatever. The point being is at some point I decided I wanted to change the name of the attribute because it would reflect better what it meant in the domain and not realizing that I had already published that event to production. And so at one point I was I don't know I don't even remember what I was doing looking up issues or honestly it might have been a view the view that was rendering that was supposed to be for was throwing errors and I couldn't understand why and I looked at it and sure enough it was because I accidentally created a different version of the same event. Thankfully all I had to do was change the code to say well if this attribute doesn't exist for this attribute and everything was fixed. But um you know you can honestly fall into some of those things with event sourcing too if you're not careful. But uh the nice thing is because the events are still there you have the ability to recover from them.

[00:38:40]>> Yeah. Let's talk about versioning for a little bit.

[00:38:42]>> Sure.

[00:38:44]>> On a sort of operational third normal form type of database. You know you might run a migration.

[00:38:52]It's one of the reasons I really like using MongoDB is because I almost never have to run migrations. But that's a different it's a different debate.

[00:38:59]However, you might run the migration to say like, okay, we're going to add a column or we're going to split this data apart and move this stuff over here and that over there and then create a foreign key relationship or whatever.

[00:39:12]>> Yeah.

[00:39:14]>> But I can see if you've got this kind of history of things like let's say I don't know how do you deal with versioning, right? Like I I've got these old events and the way you're not storing the current state. So with a migration or something like that, you're like, "Well, let's just transform the current state into the new state." With these, you got like old events and new events, and they might be in in a real way incompatible.

[00:39:38]>> Yeah. Yeah. What What do you think about with that?

[00:39:41]>> You have so many strategies. Um you just have to choose which one works for your situation. So the first one is kind of what I mentioned just a minute ago kind of like the MongoDB or I should say document database way of working with things where if you're adding things to an event adding fields then if you know if the code you know most code will be blissfully ignorant that you're adding new attributes to it so it doesn't really matter and then those that do care can kind of have fallbacks and the way that um Adam and Martin suggest you essentially have like upcasters or some kind of code that essentially says like okay the previous version actually that's two different things I'm sorry uh the code that does care about the new attribute if it get encounters an older event that doesn't have that attribute then you can have a default fallback and it's best to have those close to you know the domain logic that you want to update this is a vertical slice architecture approach as opposed to having like a global upcaster again I said upcaster I realized that I want to keep going to upcasting, which is the second option that I've been trying to get to. Um, but so essentially one is, you know, have a default fallback.

[00:40:51]Second one is to have an upcaster that says, okay, you know, especially if they're two different versions, like truly different versions of the event.

[00:40:58]You know, you have add to shopping or uh item added and item added version two.

[00:41:04]You know, you might have completely different fields. It's good to have a piece of code that can, you know, upcast to the second one. Um those are the >> maybe put that in your like your data access layer. You might write a thing that say give me all the items of the cart and it it looks at the type or the version flag and then does a little processing or >> to me >> would you rewrite the database?

[00:41:25]>> Oh well that's I was going to say that's your third option. Um, so let let me take a step back and ask your first your last question, which was like at least to me the the tent what you're doing with your events is re essentially rebuilding state. And so wherever you're in that loop to rebuild the state, you would probably say like, oh, is it this event or is it this event or whatever behave this way >> right there? Like in my story of using one of the data caches, >> yeah, >> you do a request, it's not in the cache. So you got to run your build it up code, then put it in the cache. And that just look made that part right there would be the part that goes okay, we're going to transform it differently now and then we'll still store the answer in the cache and next time they ask it's it's just fast.

[00:42:05]>> Mhm. Yeah. So yeah, you would have the way I would code it is you would have a piece of code that's listening into the events. So you have it knows about the old version of the event and the new version of the event and it knows what format that data needs to be in in that reddis cache or u whatever was that you suggested.

[00:42:24]>> Yeah Valky come on valky let's go let's go >> I'm going to remember this by the end of the conversation.

[00:42:29]Uh so yeah so what it does is it will be able to say like okay this event is an old one but I can upcast it to the new one and convert it into the format or however we need to save it to Valky and then if it's a new one it also knows okay I'm going to convert it this way to to save it to Valky. Um so it's nice that it's kind of localized to uh you know the process that you need to update upcast.

[00:42:53]>> Yeah. Um, and then we touched on like kind of the quote unquote nuclear option which is to apply a transform to your entire event store. And so you would by doing that you would create a new database or new database table which would be your new event store and you for each event you just copy it over do some kind of map or transform to upcast the entire your entire history. Um, >> yeah.

[00:43:14]>> Yeah, >> I see values value in all of them. you know, if if you're doing a lot of direct SQL data sciency stuff, you probably want to transform the database. If it's primarily coming out of an API, like just let that thing handle it as it reads them. You know, computers are fast. Keep it in mind.

[00:43:33]>> Indeed. And I forgot to mention that um oh, his name escapes right now. The guy who popularized event sourcing back in the 2000s wrote a book on this topic specifically. So he kind of listed out all your strategies and why when you would want to choose them and why.

[00:43:49]>> And >> are we talking Martin Diller?

[00:43:51]>> No.

[00:43:52]>> Um Greg Young.

[00:43:54]>> Greg Young. Okay.

[00:43:56]>> I think it's on Lean Pub. Yeah, that sounds about right. In fact, Martin Diler's book is cheaper on Lean Pub as well, and you get additional content there, too.

[00:44:04]>> So, yeah. Very interesting.

[00:44:07]>> Let's let's um make this a little bit concrete for people.

[00:44:11]>> Sure.

[00:44:12]So we talked about how you might in theory architect some software could be Python or something else to follow these patterns but there is a Python library right?

[00:44:23]>> Yeah absolutely >> do do you recommend it?

[00:44:26]>> I do you know I it's funny I don't have any production code with it but I have used it a lot over the years. Um John Bwatter has done an incredible job maintaining this this repo and and you know all his uh all the people who have contributed as well. he has shephered this through and is has been really making it such an incredible application. Um when I wrote my applications, you know, I the first one I did, I was like, I want to do it myself so I can understand it and so I can really understand and respect what he's done. Um but also, you know, there's a part of me that really like a lot of the uh you know, essentially I feel like it depends on who you are. If you're somebody who wants to grab a framework and run with it, this is an exceptional one to do it with. it essentially you just write Python classes and you decorate them or um subclass from some of his classes and all the magic of event sourcing happens for you and it just make leaves you with really readable understandable code and then you'll have other people who you know a lot of people in the event sourcing space says it's actually not that complicated you can write your own code to do it and I did and I I recommend it for the right type of person for me it was hard because there's so many decisions that you need need to make and I am not the best Python programmer. I do not know all the concurrency issues and all these things.

[00:45:51]I'm getting to learn them more. Um but you know I I have software that's running in production that's that's doing well. So all that to say is yeah I highly recommend this package especially if you're new to it. It can really show you you know how you one option of of things can work. And I love that it by default it uses uh well you can you can use um SQLite Postgres or uh one of the a couple of the docu databases that are optimized for event sourcing and so you can kind of see how it you know some of the many ways you can pattern things to make it easy for you.

[00:46:28]>> Interesting. So I probably need to file a a PR or something here. It says the way you get it is pip install event sourcing. you know, feel like it should be UV pip and solvent sourcing.

[00:46:39]>> No, just kidding. But that's cool.

[00:46:41]However, I am, you know, so it's, you know, you can get it off um Pippi, but >> I'm having a hard time resisting pressing this.

[00:46:48]>> Ask Deep Wiki.

[00:46:50]>> If you ever gone, what is Deep Wiki?

[00:46:53]>> Yeah, >> it's a AI powered document thing that he he opted into which I thought was very fascinating. I um I I was following I was at the time he did it I was I think I was writing my own code and so I had Slack open. He has a a good Slack channel and was like showing all the things that he was able to to all the insights that was able to be gleaned from it.

[00:47:15]>> This is epic. I love it. So it h the deep wiki apparently is like and knows the source >> and the docs and then it's just a chat and even on fast I asked I said give me an example using this library says sure here's a complete example a dog school application and all the code in using event source the li the package dang this is nuts >> all right new uh side quest unlocked must figure out how to get my packages into deep wiki this is new >> nice and I also want to add he has other companion packages that for example um connect into Django and I believe Flask and some other ones too. So um one of my side projects I'm I'm leveraging this to with Django and it's it's really cool because you it one of the things that enable enables you to do is configure um your events table to be similar to your or I guess in the same database as your Django table or at least configurable from the way Django would do it. And yeah, so getting all these read models are very easy with all the migrations.

[00:48:18]You just say this is what I want my data to look like with Django. And of course, apply migrations and there it is in production. So that's really nice.

[00:48:25]>> Yeah. Cool. It also has extension projects. What are these? The Django one, the current DB. K R N T. Imagine current DB is probably a >> I believe that is yeah it's a one of the first event sourcing specific databases I think it was called >> it's for intelligent and responsive systems. I don't know, Chris. I just got to rant a little bit. Like, there's all these projects that are cool and they do neat stuff and now I feel when I go to them, it's like this is the AI compute data frame or this is the AI the intelligent AI. It's like it's just a database or just a data frame.

[00:49:05]>> Yeah.

[00:49:05]>> That AI can use. That doesn't make it an AI data frame. You know, it's like but they all want to capture the excitement.

[00:49:10]It drives me crazy.

[00:49:12]>> Yeah. And and what's worse is when they don't even say exactly what they do. It is. It is your answer for AIing the thing that we're not going to tell you.

[00:49:19]I hate it. Yeah.

[00:49:21]>> Yeah. Exact. And it just obscures what the heck it is. But it's the H1 and the H2. You're like, "Oh my gosh."

[00:49:26]>> Yeah.

[00:49:28]Yeah. Okay. But that does look pretty interesting. Like Yeah. Look, it's example is create a client, new event data, new u new UU ID, etc. Order place serialize.

[00:49:41]So right here, this example is basically it's got a primary key, a category or type of event like a like just an event I guess is what you would call it, but then it has a JSON serialized >> um like a JSON blob that is the details of the event. Is this how you typically do it >> or is it more column oriented where like this one has an order ID and a total. So you might have an order ID and a total in the data structure or is it in a blob level?

[00:50:12]>> Yeah, the implementations I've seen generally will have some kind of blob or JSON serialized or um bite code serialized um optimization of it. You know, because each event, you know, when you're saving things to a database, you know, you're going to save an event, it's going to have an event stream, it's going to have a generally speaking, there's probably an event version. Like there's all these specific things, but the actual payload, >> if there's not an event version, you're going to wish there was an event version at some point, probably.

[00:50:42]>> Exactly. Yes. And so the payload is usually some kind of blob or JSON body or something like that.

[00:50:49]>> It sounds very good to be a document database.

[00:50:52]>> Indeed. Yeah.

[00:50:54]>> Yeah. Because you can put indexes on like the sub items and then if they're not in that event, it just doesn't use the index for those particular ones. I mean, it's >> a lot of bings. Yeah.

[00:51:05]>> Yeah. Exactly. It's pretty sweet.

[00:51:06]>> Yeah. And and again, well, I haven't said it quite, but like one of the things I was have been thinking about for a decade is like the more I kind of thought about it, the event sourcing event sourcing and the patterns in unlocks really gives you so much flexibility. You know, you can you can use document data stores and and like really take those the power of that, you know, if you have um and part of this is too is just data, you know, vertical slicing or whatever. It's mult multiple patterns put together. But like you know if you have a view that would be so much better served by having a a graph query a graph uh database then use it. You know it's um I remember at a one time it it took me a while but like uh somebody told me that they were using um I can't remember uh their their open query. Is that what it's called? the um I forgot the old name of what I'm I'm trying to think of, but essentially it's um you know like a database that optimizes for saving text so you can like search.

[00:52:10]>> Yeah.

[00:52:11]>> Um you know they use that as just for one you know to to to to serve the purpose of one item and honestly this isn't unique to event sourcing. You can do this with event driven architecture as well. But what I love about event sourcing is like you have the benefits of event- driven architecture and the benefits of a monolith in one if you choose to go that way. And uh yeah, it's just it's I guess really what it comes down to is what I love about it and was surprised by is how flexible it gives you the ability. Um yeah, in the book I it reminds me um one of my users was complaining about the status screen that I I show for the users and he had all these great ideas and I was like you know what I want to take advantage of that. So I actually cloned my the vertical slice for that view and created a new database column or or collection uh for that to power that view. And we iterated and iterated to make this thing better. And with each iteration, sometimes I needed to change how the the read model reacted to events. And so I could just blow away the read model, regenerate it from events. And we ended up with something really great. So that when it was time to go live, I just changed which, you know, where the URL went, pointed to the new one, and was able to delete the old code and delete the database table. And it was wonderful.

[00:53:34]>> Yeah, that's awesome.

[00:53:35]>> Yeah. It just gives you so much flexibility to do whatever you need.

[00:53:41]>> So a couple more one more I guess one more really relevant thing two more things to give a shout out with this event sourcing. There's event sourcing Django which is Python package for it with Django. Imagine that probably somehow integrates with OM don't know but also event sourcing SQL alchemy which is kind of cool. So if you use SQL alchemy like yeah very nice.

[00:54:02]All right so this stuff is great. Mhm.

[00:54:05]>> But I imagine that it has times you should use it maybe more and times you should go well square peg round hole maybe not this time.

[00:54:15]>> Sure. Yeah.

[00:54:16]>> What do you think?

[00:54:17]>> For me I feel like it's usually the way I think about it first is because most people are very comfortable with not using event sourcing, right?

[00:54:25]And so I usually answer it the opposite way which is when should you?

[00:54:29]>> Sure. Exactly. the the two biggest the best pieces of advice that I heard over the last decade was number one use it. A good a good opportunity to use it is if you have a database column called status because if you have a column lead status then that means that one item can be in multiple different um statuses right different different states and if you're having different states each states behave different in some way or form and so you're definitely not dealing with true CRUD create read update delete patterns and so event sourcing would be a great option for that. Uh the second piece of advice is do you ever are you ever concerned about losing data because by default event sourcing does not and what it enables you to do is choose when to lose data right because you don't have to keep every event around forever.

[00:55:22]You can just say like after 90 days let's just put it to cold storage or just delete it you know it depends.

[00:55:27]Yeah, exactly. Out in the audience, Mike says, "I'm scared of the physical storage requirements of this potentially." And I guess it depends how many data, >> how many events make up a final state >> in your system like a cart checkout. Big deal. No, probably not. Like, >> yeah.

[00:55:44]>> If used as an app log, that might be a problem.

[00:55:47]>> Yeah. Yeah. Most um I'll say models will have maybe a dozen events, maybe two.

[00:55:53]Depends on your obviously depends on your your domain. Um but ideally you will keep your events short and they have practices called u closing the books will where you will use uh events in your domain to kind of keep it short.

[00:56:07]So like you know for example a a a store will want to know their revenue across the entire year but every night they shut down they get their cash registers or if they still have cash registers and they kind of reconcile how much money they made that day.

[00:56:21]>> And so you know kind of keeping your event stream short really helps. So if you're going to go back, you would just say, "Well, we'll just read the the daily summary and then add today's events or something like to get the final output, something like that."

[00:56:35]>> Yeah. Unless you want to go all the way back to day one, in which case you can, you know, read and say like, okay, the, you know, you it all depends on how you want to do it, right? This is again the flexibility side of it because you could just like say start from today and read forward or you can start from today and say okay what was the event stream before this and read that and keep going back to to the originating >> and >> uh I think you put up Mike's comment that said I'm afraid of the physical storage requirements and it's like that is the trade-off there is uh it will take more space but thankfully storage space is the cheapest commodity in all of online or you know in today's world um and >> most expensive is memory >> and then compute >> and then storage and then bandwidth and then storage. I think that's probably the breakdown, right?

[00:57:25]>> Yeah, I think so. And >> that's why things like uh discache are awesome versus like another thing that's just in memory cache but in another process, right? Like >> Exactly.

[00:57:33]>> Yeah. and having the ability to say like you know what we you know like my current application I have not yet deleted any events but truly like the only reason we have events older than even a week are just for analytical purposes and and just me understanding how our system works and so I'm planning to make a way to offload that event those events and a lot of people just put them straight to cold storage you know just so that they always have a backup just in case but you know chances are they rarely ever use Sure.

[00:58:07]>> So, and one other thing I did want to add is uh to go answer your question when not to use event sourcing.

[00:58:12]>> Yes, exactly.

[00:58:12]>> Um would be um essentially like you know, so let's say you don't care about losing data. Um there are just a number of just simple applications that are truly crud, right? Like I I've worked on a number of these where they're just forms over data.

[00:58:26]>> Um >> that's exactly the term I was thinking, forms over data.

[00:58:30]>> Define that for people if they don't know. Yeah, it it's essentially something where like in my case, one one of the first ones I worked on is like you have a web page that almost exactly mirrors a database table that you're saving the data to. You know, maybe it's a contact form. Uh who knows what it could be. You know, the idea is like there is so the web UI or whatever you're building is just an easy way to get data into the database and chances are you don't have status field. You don't have all these different ways of different rules for how things behave.

[00:59:03]And in fact in my event source application I have a model that is not event sourced. Uh we it is truly a CRUD model. And so just by saying you know adopting event sourcing doesn't mean you have to do it for everything. You can use it for even just a small bit of your your project especially if you want to try it out and see. That's a really good point. Like it's not an all or nothing sort of thing >> because you have a properly factored >> data access layer and you're not doing that inside of your Ginga template, are you? No.

[00:59:32]>> Right.

[00:59:34]>> Not even in your view, but like you've got just a an opaque layer of actions.

[00:59:39]Some of those actions can be driven by events. Some of those actions can just be straight CRUD. Create, read, update, delete. For those who don't know, >> one of um the people who inspired me to really dig into event sourcing, he has a line of business that well, he'll um go to a company who is struggling because their database schema is holding them back, you know, for whatever decision they made, they cannot they're having such a hard problem, hard time creating a new feature because of their database schema. So he goes in, teaches them event sourcing and uses the event sourcing um event store to publish both the dream schema that they wish they would have and the old schema and they live side by side and the event you know the the once the features are complete they'll you know put it up and they'll start slowly migrating traffic over to the new event source version and you know eventually they can delete the old database table or scheme or you know database and most of the uh teams he's worked with have kept with the event source version and gone on from there.

[01:00:44]>> How interesting.

[01:00:45]>> Yeah. Oh, and then finally, one other thing I want to mention too is when not to use is it's up to your teammates because you know I I am sold. I think this is such an incredible pattern. It is just unlocking so much joy again and and so much flexibility as I've said before that I cannot imagine having to go back. That said, if I join a new team and my team members are like, I don't know, I'm going to go with them, you know. Um, >> yeah, it one thing is to not use an optimal pattern. What's worse is to try to use an optimal pattern but have nobody else want to do it and then they work around it and you you know it sounds a little similar to like people who don't want to do unit testing.

[01:01:34]>> Yeah.

[01:01:34]>> So some of the people write the unit test and they set up CI/CD that'll fail if the unit tests fail, but then the other people will check in work without running the test at all and then they break it and you're like >> what are you doing? Like well I don't want to run these crappy tests. You're like >> well now the whole CI/CD is not just not helpful. It's inhibiting me working because you won't even, you know, I mean, it's just like uh and it seems like you do need a certain level of buyin for this to make sense.

[01:01:59]>> Absolutely. Yeah. And you know, maybe >> maybe should Yeah. Maybe they should listen to this podcast and they can see >> Yeah. And maybe they you create a you know example of, you know, one feature in an event sourced way so they can see some of the benefits, but you know.

[01:02:13]>> Yeah. Yeah. Yeah. Like your partial example. Indeed.

[01:02:16]>> Yeah.

[01:02:18]>> All right. We're getting short on time here, Chris, but let's let's talk this this AI >> um flow. I have >> first of all, let's circle back to your comment of like um the your company having a mandate to use AI like what what the heck is going on here? How how is this received and how are you receiving it? And also tell us are you actually writing you know make shipping more features and be more productive or not like what give us your assessment as much as you're willing to share like you don't have to like secrets or whatever.

[01:02:51]>> Yeah. Yeah. I will I will um hide certain uh things but to say >> names and places have been changed to protect the emotions and conversations with multiple other people. Um, I would say at times I am so much more productive. At times it has brought down the it does a really great job. And I think a lot of this has to do with the vertical slice architecture because vertical slices only hold code that is responsible for one feature. And so that really fits very nice into a context window.

[01:03:26]>> Yeah. It doesn't have to scan 200,000 lines and 100 files. It looks at five.

[01:03:32]Yeah. And it knows event sourcing. So it knows, okay, I'm subscribing to events.

[01:03:37]These are the events. I know where they are. You know, all these different things. Um, when I work on one of the other services, it takes a lot more context to understand the state of the code and I really have to work harder to do to do what I need to do in those in those uh parts of the the codebase.

[01:03:57]Yeah. So it's been a very interesting experiment. And additionally kind of when I am I had to curb this but when I have been more productive is creating um git work trees. So it's like I'm I have a kind of main repo that I work out of and then I if I have a feature that I you know I'm like looking at the codebase I'm like oh or or the the web app or the logs and I'm like oh it' be good to like optimize this then I create a new work tree and set cloud up in there and get it working on a thing. And so I have found that I can only I need to limit myself to two or three work trees because any more than that I start losing context and now I know what the LLM looks like.

[01:04:33]>> Yeah. Yeah. Yeah. Exactly. If you over overdo it, it's you just send off five agents and don't look. Like that's how you end up with like, oh, we have kind of like bugs in our code, bad architectures. Well, if you never look at it, >> it's like we got this super energetic, super smart intern and we kicked them off and said go add that feature. But, you know, they they need guidance, right?

[01:04:55]>> All the tests are passing because I changed all the tests to pass.

[01:04:58]>> I know the problematic data has been removed from the database. It works now.

[01:05:03]>> Why is it empty?

[01:05:06]>> Yeah, >> back to your backup comments. I I honestly I'm having like an insane amount of productivity with cloud code and with AI and stuff and but it's an engineering skill. It is not just let's fire it up and ask for it. Like one of the things I'm doing lately that I've been really appreciating is going through like a planning session which I know a lot of people do that and like talk about it but and now if you have the GitHub CLI installed just the GH thing you can tell it hey create you know a instead of just running this plan create a GitHub issue of this plan write all the details in GitHub and then your next comment can be let's work on issue 127 and it'll go work on it when it gets done like let's make a retrospective comment on the issue. Let's create a PR that closes that issue. That's you like there's some really interesting team dynamics that you can put in there.

[01:06:02]>> Yeah.

[01:06:03]>> That you know talking to a chat box >> is not covered but if you know what you know what to ask for.

[01:06:09]>> Yeah. I'm I'm really inspired by Martin and Adam because both of them in one way or another have well let me take a step back. They I mentioned the event modeling diagram. It is a visual diagram that really has a reduced uh visual language and what was mind-blowing to us a couple years ago was that AI understands it. And so the fact that you can essentially say like here's the diagram. Can you implement the slice and it can get you from well let me take a step back. Martin and Adam have both had successful uh um uh um research spikes where they took an event modeling diagram actually no they even did what they did even worse was they started with a conversation with the client and recorded it created the diagram >> they generated the diagram >> generated the diagram and then gener got uh generated code from the diagram that didn't solve everything but got it I think 80 or 85% of the way there in hours >> from you know like cutting months of work down to weeks is impressive. And I've had some similar um uh you know I'm I'm work still working on my personal one because like I just you know after work I just tend to shut down my computers and I'm not like dedicated to like really going at it.

[01:07:24]But like I found some really incredible benefits of doing something like that.

[01:07:28]>> Yeah, that's awesome. I created this open source project. I mean it's more source open whatever. It's not really a project, but I called it Python package guides for agents. And all the projects that I work on, I'll go and download the source and the documentation. So, like if I'm working on discache, I'll like literally clone it. Clone the documentation and then make Claude write a super detailed uh like not use their documentation or its old version, but like have it like legit write down examples, study the source code, study the documentation, source code like Trump's documentation because if the docs are out of date and so on, and then I'll drop, you know, if I'm using like two of these like maybe data classes and discache, I'll drop those things into my project and tell Claude about them. And that's >> that's been that's a pretty neat thing to do as well.

[01:08:13]>> Yeah.

[01:08:13]>> But I want to leave this this convers this portion of our conversation with an incredible joke.

[01:08:18]>> Okay.

[01:08:19]>> Okay. Just cuz I feel like this has to be said right now. It's just so the joke says the word co-pilot. It could be claude, could be codeex, could be chat, whatever. Like just AI, right?

[01:08:31]Friends outside of tech. Well, co-pilot is dumb. Friends in text in tech. I just bought iodine tablets and I've made an offer and land upstate. my supplies of antibiotics and potable water are sufficient, but I need to set up for the hydroponics to make it through the first few years. Like I feel like that's where we are, you know?

[01:08:46]>> Yeah. Yeah. Totally. Totally.

[01:08:50]>> And that maybe also sums up your um your meetup as well.

[01:08:56]>> Yeah. Yeah. Yeah. Yeah. Absolutely. It's quite a a spectrum both of friends outside tech and inside tech.

[01:09:03]>> Yeah. Exactly. Like it's more like believers and non-believers. I'm not really sure, but >> All right, final final call to action.

[01:09:09]What What do we got here? People are interested. They want to get started.

[01:09:12]What do you tell them? Get your ebook, your free ebook that you put out.

[01:09:15]>> That's right. Yeah. So, my website is everyday superpowers.dev.

[01:09:20]If you want an ebook that kind of introduces you into event sourcing and kind of gives you kind of this this kind of fundamental background and and a couple other things, go to everyday superpowers.

[01:09:32]Intro and it'll take you right there.

[01:09:35]Um, I'm on Masttodon mostly, but I'm also watch Blue Sky and sometimes X. I'm Chris May on all of those. Uh, with Maston, I think I'm on fostadon.org.

[01:09:47]What else? Um, I mentioned everybody's superpowers. So, that's I also have a Discord from there, too. So, if you go to my website, you can see how you can join that.

[01:09:57]>> Yeah.

[01:09:58]>> Sweet. Maybe check out the event sourcing library >> for Python people. Yeah.

[01:10:02]>> Yeah. And if you Oh. Oh, the Martin and Adam have a podcast called the Event Modeling and Event Sourcing Podcast, which is verbosely named, but it also is really great. You know, this is how I learn just from these great people. You know, >> there you go.

[01:10:17]>> Just kind of every almost every week talking about patterns they do and stuff like that, they also talk about a bunch of other stuff that isn't relevant. But I have learned so much from listening to them. And they also have >> Discord. So, go ahead.

[01:10:30]>> Cool. I'm honestly impressed. like an entire podcast on a single design pattern. Let's go. That adds commitment to it.

[01:10:37]>> Yeah. Yeah. And it's incredible. Yeah. I mean, as someone who who has played the board game and really enjoys it, it's amazing.

[01:10:45]>> Yeah. Awesome.

[01:10:47]Well, Chris, I really appreciate you coming on here and sharing all your experience and excitement and all the things. Great to talk to you.

[01:10:54]>> Likewise. Thanks for having me.