I built the .NET practice platform I always wanted. Where I've been

[00:00:00]I think this is the longest break I took from YouTube in the past 4 years and there's a very good reason for that. I was busy building a new learning platform that I just launched and it's called Katabench. I'm going to drop a link to it in the comment below and it's essentially the learning platform for.NET developers that I always wish existed. Unfortunately, it didn't so I decided to build it myself. Here's what the app looks like and don't worry, I'm not going to make the video about selling you on this platform. I actually want about the tech stack that I used to build this whole thing. But first, let me just briefly show you what you can do at Katabench and what makes it different from all the other learning platforms.

[00:00:40]So, what I have here is a database type of challenge. In this specific example, we're using EF Core to interact with an actual Postgres database. So, your code that you write here is actually sent to our servers where it's compiled and executed and then tested against a real throwaway database and what you get back is actual execution numbers for allocated memory, for query speed. We also give you the queries that you actually execute on the database. So, you can literally see what is happening with the code that you write. Now, running the code just executes some basic tests. The actually interesting part is submitting it where we validate the code against a set of hidden tests and these are a lot more challenging.

[00:01:22]So, you actually have to write the optimal solution to get the desired results. Now, I'm going to cheat and just reveal the solution here which turns out to be fairly simple. But when you run this, you should see all of the tests passing. You get some experience points, some gems that you can use to purchase stuff and a lot more interesting things. Forgot to mention that you can also view the database schema for each challenge and other than the database tests, we've also got tests for algorithms, SQL where you actually write raw SQL with Dapper, database tests using EF Core, architecture test design principle, refactoring, secure coding and test writing. If you don't like to go blind, there are learning paths that actually structure this content into a step-by-step learning experience where you can solve a set of challenges that each address a specific problem. And I'm again going back to the database test because there are some really interesting things here. For example, there are some challenges that actually test that the query you wrote uses the optimal database index. You can get this info from the actual query plans that the database produces when you run your query and then use that to determine if your solution is optimal.

[00:02:25]We've also got an activity graph where you can track how active you are. You can win badges for solving learning paths. You can also unlock achievements for solving different problems. There's a leaderboard. You can change the editor theme. Let me show you the neon theme for example. I'm going to equip it. And if I go back to the playground, you can see that it now looks a little different. If I reset this to the default theme, I can also show you the light mode, which may cause some of you to go blind, but I know there are developers who prefer it. I'm personally more of a dark mode fan. Now, enough about Catabench for now that is. I'm going to talk about it more in the future, but let me actually walk you through the architecture of the system and the tech stack that I chose to implement this and why I chose some specific components. So, on a high level, the Catabench system consists of three components. Let me draw them in some of the boxes. So, the first one here is the marketing website. And for this specifically, I opted to use Astro because it's very good for static content and SEO. And I've actually migrated my own website from Next.js to Astro. So, if you go to my blog or something like that, it's not going to be served by Astro. Now, when it comes to the actual application that I just showed you at the start of this video, this uses React with Vite and TypeScript, and something I'm familiar with, and you can build pretty much anything using this stack. Now, for the back end, and this is actually the most interesting part, I'm using.NET, of course. Now, let's actually decompose the back end first because I think it's the most interesting part. And I decided to split the back end into multiple components. We're basically running two core servers, but let me break it down.

[00:04:01]So, first of all, all of my servers are part of the same VPN, a virtual private network. This allows me to completely lock down my servers from any public access, and I only expose ports for HTTP and HTTPS traffic, so that my API can talk to the outside world. To build the VPN, I'm using a custom Tailscale server, or rather the open source variant called Headscale, and I'm going to talk more about this in a future video, but for now, I'm just going to note down Headscale for VPN. I'm hosting this on Hetzner servers because they are quite affordable, and they would have been more affordable because they hiked their prices last month by 150%, so I missed out on getting some cheaper servers, but nonetheless, the server costs are somewhat acceptable until this app can get some revenue, and we're going to see how we can scale it from there. Now, when it comes to accessing the internet, none of my back end services are actually public. They are all nicely tucked away behind a reverse proxy, and in this case, I'm using Traefik. And the reason for this is because I'm actually hosting all of my applications with Dockploy. Now, Dockploy is a platform as a service, which I believe I mentioned in the past, but what it allows you to do is to deploy it on a VPS, like what I'm using with Hetzner, and it's going to allow you to easily manage your applications.

[00:05:18]Now, when it comes to my applications, there's actually three of them running in the back end. So, I'm going to draw three boxes here. Now, two of these are actually core to my system, and of course, the first one is the actual API running our.NET code. Let's call this server one. Now, if you're used to running always in the cloud, this might be a bit controversial, but I'm actually running my database server on the same server instance as my back end API.

[00:05:42]Here, I'm also running Redis for caching. I got an OpenTelemetry Collector for exporting OpenTelemetry to my monitoring server. And what's interesting, I've also got a messaging component. And for this, I'm using NATS, which is a very lightweight messaging system that comes with durable storage, and it's been proven to work at very large scale, so it's more than enough for my use case. Now, on server two, I've got another component. Let's call this the worker component. And this is what's responsible for actually taking the code that you write in the application and compiling it, running it against a set of test cases that I've predefined for each problem that you're solving. This is also using.NET. And this component talks with my API using messaging over NATS. It doesn't have access to anything else other than a read-only connection to NATS. And the reason for this is security. I'm trying to minimize the impact of some sort of jailbreak inside of my sandbox, and this is another component. So, the worker isn't one actually runs the code. This would also be unsafe. I've got here that I'm calling a sandbox, although the term has been used for many things, but essentially what a sandbox is in this environment is it's an isolated and highly secure app instance. It's going to compile and run the actual C# code.

[00:07:02]So, all of this happens inside of a sandbox on a server. The sandbox itself is cheap to run, and it's also throwaway. So, basically every submission that you send to our servers runs in an isolated sandbox, and it can't touch any other sandbox in the system. Now, this also allows me to do some pretty cool things, which I'm going to reveal in the future, but essentially the plan is to build a long-lived sandbox where you can literally build a complex system like a modular monolith or a microservice in some sort of learning path or course fashion where you'd have an ID in the browser being able to compile and run your code, and then see the actual output of this because the application actually lives in a sandbox on our servers. So, just to emphasize this some more, the sandbox itself is isolated from everything else in the system, and there's also a bunch of other security hardening things that went into this. Now, the first server, let me add the server free label here, is my actual platform server. This is where Dockploy is running, and Dockploy allows me to manage my two remote servers. This is a Dockploy feature, and this basically lets me just automate my deployments. And whenever I need to scale up, I just add a new server to the fleet. Let's say I want to add another worker server, I spin it up here, connect it to Dockploy, deploy my code, and then it automatically starts working because it can start picking up messages off the NATS queue. This is how I will scale the sandbox if we ever run into scaling problems. The worker server also has an Open Telemetry Collector, and the reason for this is because I'm running my collectors in agent and gateway mode.

[00:08:34]So, the collectors on my two servers are actually streaming the data back to my core monitoring server. And this is what injects it into the platform that I'm using to monitor everything. And this platform is called Grafana. You may have heard of it. So, I'm running the full Grafana stack on this server. This includes Grafana, which is the most popular one, I think, but it's actually just there to aggregate all the data and give you some nice dashboards so that you can piece everything together. Then I'm using Loki for structured logs, Tempo for distributed traces, and Prometheus for scraping metrics. I've also got a couple of other services in there, but this is kind of the core of the monitoring stack. So, let me just denote this. We've got Headscale for VPN, Hetzner for hosting, Dockploy for platform as a service, Grafana stack for observability. Now, when it comes to user management, I decided to use Clerk.

[00:09:25]It's a fully managed auth service, and it handles everything I need for user authentication, and also allows me to easily integrate it into my front-end application, and then stream the users back to my back-end. I'm using Paddle as a payment provider, and then I've got my two client-side applications deployed on Cloudflare, and I'm fully using their DNS, their proxy, and all the other cool features they can offer. And one other thing to mention is that Cloudflare, for my usage, is completely free. So, if you've got a client-side website that you need to host somewhere, definitely consider Cloudflare. I've been blown away by their developer experience, and I'm pretty happy with this. So, that's the high-level tech stack of the system that I built. Now, over to you. Let me know in the comments if you got any questions. The more questions you ask, the more useful content that I can actually make because I'm going to make a couple more videos on how I've actually built this, and your questions are going to help shape this content into something more useful. And I almost forgot, this is kind of a sneak peek of what's coming to Catabench. I'm going to add system design exercises where you can actually build a proper system, much like I just did on the Excalidraw drawing board. We're going to have a challenge to piece together a set of components to build out some fairly complex system. And this is going to consist of some common system design components, which you then have to validate against the desired design. In the future, I'm thinking of expanding this even further, where you can actually take the system that you design by just dropping the components on the UI, and then actually spinning up the system inside of a sandbox, and for example, running a set of load tests to see if the design that you created actually shapes up to something that can stand proud against real-world usage. If all of this sounds at least somewhat interesting, you can head over to catabench.com and give it a try. If you create a user account, you can access the platform for free and even get a decent usage allowance. And of course, the paid plans come with a lot more usage, early access to new features, and a bunch of interesting stuff I've got cooking. Also have to say, it's nice being back on YouTube, and I promise to make more videos in the coming months.

[00:11:26]Thanks a lot for watching, and until next time, stay awesome.