A Practical Guide to Async IO in Zig

[00:00:00]Now, the very sneaky thing about the io.select is if you go looking around it, and if you look at the standard library code, you realize that it's just a very lightweight abstraction around the io.group and io.queue. All right, so here though, this video is going to be a practical guide to the new async io interface in Zig 0.16.0.

[00:00:23]We'll be going through some of the new primitives that they've added and how they could be used, and we'll also be diving deep into some of the implementation details of those things.

[00:00:33]So, yeah, it's going to be a lot of fun.

[00:00:34]Let's get to it.

[00:00:36]So, to start with this, we'll be looking at the release notes on Zig 0.16.0.

[00:00:41]Uh, when we're looking at the io as an interface section, uh, essentially, what it's saying is, uh, from 0.16.0, every function that has any sort of, uh, control flow breaking characteristics or non-determinism, which can come either from, uh, io or time outs or anything like that, would have to use an io interface in its parameters.

[00:01:08]So, essentially, uh, there are a few of the few of the implementations. The major one is io.threaded, which is based on threads.

[00:01:16]Um, and this is also the default implementation that you [clears throat] would get with, uh, the juicy main implementation. So, if you're using something like const io.any.io, it's using the io.threaded implementation by default.

[00:01:33]Um, there is one another implementation, which is io.evented, which is currently in beta.

[00:01:40]It So, it it's currently using the m n threading, uh, which is pretty similar to go routines in, uh, golang.

[00:01:48]>> [clears throat] >> Or stuck with go routines, it would be using, uh, io_uring in Linux and Grand Central Dispatch.

[00:01:54]So, potentially, it's, um, it's a lot more performant, um, in certain scenarios because it it would use those core routines instead of using the actual threads.

[00:02:04]But since it's not available, we'll be talking about i.threaded which is the stable implementation.

[00:02:10]>> [clears throat] >> So if you go a little bit down here, you'll notice that it gives us a bunch of primitives to build on top of.

[00:02:17]The very first one is future.

[00:02:19]>> [clears throat] >> So if you go here, futures task-level abstraction based on functions.

[00:02:24]Now, there are two things.

[00:02:27]i.async and i.concurrent. Now there's a big gotcha here.

[00:02:32]These two things >> [clears throat] >> they could be a little bit confusing because i.async might not be fully asynchronous.

[00:02:42]So what I'm saying is both of [clears throat] these return futures.

[00:02:47]But the thing is i.async might not be truly asynchronous and sometimes when the thread pool is kind of exhausted with other tasks, it could simply run in line.

[00:03:00]So >> [clears throat and cough] >> in order to drive this thing home, I'll show you a bunch of examples.

[00:03:05]So here, we have a very very minimal example. We have a function which takes n and returns n square and does i.sleep just for the sake of it because I want to prove a point here.

[00:03:18]Um so for example, we use var a which is i.async.

[00:03:25]It just It just needs two things which is the function and the arguments which the function needs.

[00:03:30]Now the thing is this thing might not run completely asynchronously and it could just run synchronously and the performance of this thing could would be pretty similar if we had called io this and six here.

[00:03:47]Yep. So the The is this thing could end up performing the same as try square, which is essentially a synchronous operation.

[00:03:57]Um whereas if you use the io.concurrent, it would [clears throat] be truly concurrent. Now, note one thing.

[00:04:06]We can just call io. async and it would just return us a future.

[00:04:11]But, when you call io.concurrent, you have to try it. I wonder why.

[00:04:17]If you go here, it could return an error, which is concurrency unavailable.

[00:04:24]Now, remember the scenario which I mentioned before of >> [clears throat] >> the thread pool thread pool being fully flooded with other tasks. So, let's say if you have a bunch of futures running and the whole thread pool is exhausted with other tasks.

[00:04:37]Um it may not be possible to run your future completely concurrently.

[00:04:43]So, in that scenario, this thing will just return an error.

[00:04:47]versus um versus Oh, no.

[00:04:51]versus your um io.async will just run it uh synchronously. That's kind of the major difference here.

[00:05:01]Um it's also like a very big confusion point. Uh so, it's very important to grab this part home um before that.

[00:05:08]Um yep.

[00:05:11]>> [clears throat] >> And where's this thing? Yep. Okay. So, now that I have mentioned the major point, the thing is, these two both of these, they kind of just return a future.

[00:05:25]>> [clears throat] >> Um both of these are futures.

[00:05:28]Um but, both futures have the same characteristic. So, a future, you can you could either await it or cancel it.

[00:05:38]Now, the thing is, at a very high level, these two things, await and cancel, are doing pretty much the the thing.

[00:05:46]They essentially wait for uh the task to exit. But, But the only difference is the cancel thing. First of all, let me just show you the implementation first.

[00:05:56]You know, everything should be backed by implementations.

[00:05:59]So, if you go to cancel >> [clears throat] >> So, if you look at it, first of all, it just checks, "Hey, is the actual future complete?" If it's not complete, then it requests cancellation and then waits for it to exit. versus the await.

[00:06:15]Await simply checks, "Hey, is a future complete?" If it's not, then it just simply waits.

[00:06:21]So, when I say requesting cancellation, what exactly does it means? So, the thing is each of our asynchronous tasks, yeah, they have like a lot of cancellation points. So, for example, like there could be a lot of cancellation points. For example, you have function this way, it does a bunch of things.

[00:06:38]Uh so, it's doing something and you call cancel.

[00:06:42]So, >> [clears throat] >> at some one of the cancellation points, it would just notice the cancellation signal and return early.

[00:06:50]And once it returns, these both of these things kind of just do some sort of clean up. Um That's kind of the major thing.

[00:06:59]So, [clears throat] if you look at our example, we sort of create a future and we await it.

[00:07:10]Uh we use try here because uh this thing can fail, but And note that we are also always have to cancel it because, for example, uh this thing could fail. This one is fine, but for example, if you have some other code here, some other logic or something, if this fails, if this fails and if you're not if you're not using defer error defer anything like that to cancel it, you your futures would simply get leaked.

[00:07:44]Um so, you should always defer it and do the cancellation.

[00:07:51]Uh it's pretty much the same uh in this thing. Uh we can also try running these.

[00:07:56]Uh so, it should be pretty similar.

[00:07:58]Could be experiments. Async IO guide.

[00:08:02]Click run.

[00:08:06]So, this is the uh async version. It's a very simple one.

[00:08:11]>> [snorts] >> Uh it just takes a function uh that's a square.

[00:08:15]Nothing too fancy. Now, uh this one is interesting because here we have a snooze function, and this function takes a millisecond value.

[00:08:25]Uh let me get This function takes a millisecond value and sleeps for that that much time. So, and it's using io.concurrent.

[00:08:36]Um and we are also keeping track of the time.

[00:08:40]So, we start our timer, run both of these things concurrently, await them, wait for them to finish, and essentially [clears throat] see how long it took.

[00:08:50]So, if we run this, oh, no. That wasn't good. Uh this Yep.

[00:09:01]Uh so, if you run this, uh it says 250 millisecond task because both of these tasks uh have were given 50 in the parameters.

[00:09:11]250 millisecond task took 50 milliseconds.

[00:09:14]Now, this kind of proves that uh these two things were running concurrently and not, you know, >> [clears throat] >> not synchronously, which could be the case in your in your regular async one versus the concurrent version.

[00:09:29]That's kind of the major uh chunk of what I wanted to discuss for these two.

[00:09:35]>> [snorts] >> Uh Uh, there are a bunch of other things here as well.

[00:09:39]So, if you go to group.

[00:09:42]Now, this is an interesting one. Uh, if you go to group, it says Uh, essentially all the thing is one of the major use cases of the regular futures could be for example, um, >> [snorts] >> uh, let's say you have some sort group of tasks, you know, you want to do a bunch of IO operations or anything like that. You have a group of tasks and um, you kind of want to keep track of them. So, you create a bunch of tasks and you fan them out. You could do it manually with futures. Essentially, you could create a slice of futures, start them and then, uh, wait await each of them. versus this is just a nicer primitive on top of futures.

[00:10:23]Uh, so if we go to group, in this scenario, what you can do is, um, make it a little bit bigger.

[00:10:32]In this scenario, we have a worker.

[00:10:34]Worker sleeps and does some sort of other operation.

[00:10:39]>> [snorts] >> Um, it's pretty similar to them. But, we initialize it with a group.

[00:10:45]Um, this is the result vector, uh, result slice.

[00:10:49]Uh, it's pretty similar to them. Uh, we have the defer cancel, otherwise you would leak futures.

[00:10:55]And, uh, instead of starting [clears throat] each of them and, you know, keeping track of those futures, you could just initialize them inside a group and then await all of them at once.

[00:11:06]>> [clears throat] >> So, what this thing does is blocks until all of them are finished.

[00:11:11]And, uh, the thing is if any of them error out, uh, this error defer would catch them and essentially read them all.

[00:11:20]So, uh, we could try running this as well.

[00:11:23]So, zig run for Yeah, [snorts] so it's the squares, um, >> [clears throat] >> we're doing a bunch of these operations.

[00:11:34]Uh so, we're just doing the square for each of them. Right. So, next one we're looking at is select. Uh now, this one is interesting um because uh let's see what this is first. Um it says execute tasks together providing a mechanism to wait until one of the more tasks complete.

[00:11:53]Now, this thing is basically an very lightweight wrapper, essentially a decent abstraction around groups and queues.

[00:12:04]Um so, let me just show you first of all what this thing does.

[00:12:09]>> [clears throat] >> So, uh as think of a certain scenario where you have a lot of tasks. So, for example, um let's say you have a bunch of URLs, uh endpoints, and you just want to quickly want to figure out which one is the closest to me.

[00:12:28]Essentially, what you what you trying to do is like find the endpoint with the lowest latency. So, what you would do is [clears throat] you want to have uh you know, just fan out requests in parallel and figure out which one is the fastest uh in the response to you.

[00:12:49]So, uh you could use groups for that, but you would have to build things on top of it, but this thing kind of gives you that s- out of the box. So, here we have function um it just sleeps like the rest of them for [clears throat] the certain amount of time.

[00:13:07]Here, uh the syntax of this is pretty similar. Um so, we just initialize it with an IO and uh a slice.

[00:13:15]And >> [clears throat] >> I'll tell you what this cancel is called is later.

[00:13:20]Uh but, just like group, uh you can uh you can kind of fan out with this dot async syntax. [clears throat] Uh this kind of this thing and you give the function and the arguments.

[00:13:35]And then you can just switch essentially.

[00:13:38]What this is is you're just waiting for hey, which which one is the thing which returns first.

[00:13:44]>> [snorts] >> And now notice that >> [clears throat] >> our work function returns uh an error or U64.

[00:13:52]Now I ideally well, if you're doing things synchronously, you would be doing things like try work, you know, since that thing could return error. Uh so you would add a try work, catch errors or something like that.

[00:14:07]Um but [clears throat] in this scenario um this thing just returns us this whole package.

[00:14:15]Essentially, this this either this U64.

[00:14:19]And we would have to try it ourselves.

[00:14:22]So the thing is let's say we did this thing uh and we got this. Now the thing is at this point we still don't know if the function succeeded or returned an error because we just got this whole package.

[00:14:37]So we would have to kind of unwrap it with with a try.

[00:14:42]Um and the thing is this thing can fail because since it's a try uh uh so this is the point where this thing kind of uh comes in our interest. This just kind of it's in defer, so whenever we go out of scope, >> [snorts] >> this thing would will fire. And it's just going to clean up all the things. So uh this thing is useful uh when let's say one of the requests succeeded and we go out of scope, so it's going to cancel all of the other things and cancel all the other things and just uh collect them.

[00:15:17]And it's also useful when uh something fails because when something fail, it would also go out of the scope and it would just reap everything.

[00:15:26]Um >> [clears throat] >> the thing I'm super interested in is the actual implementation. So, if we go in here, you'll notice that it only uses these two abstractions.

[00:15:39]Now, these are the two ones that we talked about before.

[00:15:43]Uh group and queue.

[00:15:45]So, select here we're using the um async version, but there's also a concurrent version. It's pretty similar to like I mentioned before, but if you look at it, if you just [clears throat] look at what this thing does, um so, we get in here. This thing kind of prepares a context wrapper with a function.

[00:16:06]Uh if you look at this function, um >> [snorts] >> it takes a bunch of things. Um Here, LM is just uh wait a second. LM is kind of just it it calls the actual function that we give it.

[00:16:24]Um and >> [clears throat] >> and puts that puts the um output of that thing into a queue.

[00:16:35]So, this this is kind of the wrapper this thing creates.

[00:16:39]Uh it wraps it around once again, and notice that it puts uh it kind of puts it in a group and kind of launches it.

[00:16:46]So, if you look at this uh dynamic play dispatch call, remember this name, i.vtable.group async.

[00:16:55]So, something like this, uh if we go to our group, if we go to async, wait, isn't this the same?

[00:17:06]Haha.

[00:17:07]Yeah, so uh the whole point is uh these two um these two, they kind of it's just a wrapper around uh the select one. Essentially, we just create a context >> [snorts] >> uh return the uh the function to wire a group. Uh the very first thing which finishes, uh it kind of just puts it in there.

[00:17:29]Um >> [clears throat] >> we kind of collect it and discard everything, so the channel kind of just closes and everything else kind of gets collected.

[00:17:37]This is kind of the overall implementation of this thing. We can also try running it.

[00:17:42]Uh file select.

[00:17:44]Okay. So, the this what this thing is doing is >> [snorts] >> uh we're just doing two things. Uh it just says, "Hey, which one finishes first, fast one or slow one?"

[00:17:53]Um So, here if we just make it big, now the slow one should win.

[00:18:00]Yeah, pretty much this.

[00:18:02]Um >> [clears throat] >> So, we can go to the next one, uh which is Here we go here.

[00:18:09]Uh I want to talk about a bunch of other things, for example, like cancel.

[00:18:13]Now, you might be thinking, "Hey, what's the big deal about it about this? Why would you, you know, want to make a whole example about it because uh since we have talked about cancel in each one of these before."

[00:18:24]Now, the thing is um the cancel is a lot is a lot more complex than, you know, one of those basic things.

[00:18:33]Um so, [clears throat] you might think that a cancel would would only be used with uh some sort of IO operations uh and not with pure CPU operations. So, for example, if you if you uh let's say in one of these examples, uh in concurrent example, if you started a snooze function, right? And if you let's say if you weren't using uh IO in here, IO in here for instead of IO, but, you know, you you were running some sort of server, while true, accept connections, something like that, right? And if you even if you were not accepting a connection because that's some sort of IO, but if you're doing some sort of, let's say, some math, right?

[00:19:22]Not just math, long running math.

[00:19:27]In that scenario, um it would be [clears throat] extremely it wouldn't be possible for this thing to um to [snorts] get the cancels cancellation signal because hey, this is a pure CPU operation. You're just burning CPU cycles.

[00:19:44]So, the thing is um they [clears throat] have given us a nice primitive for this as well.

[00:19:49]So, there's this thing called io.checkcancel.

[00:19:52]Uh so, if we go there and search for this.

[00:19:59]>> [snorts] >> So, generally we don't need to support cancellation, but for CPU tasks, um uh we can this would be useful for that.

[00:20:08]So, you might be familiar with yield.

[00:20:11]Essentially, uh in long running CPU tasks, you occasionally yield yield for something so that uh you don't just keep burning the cycles for the for the particular task and others could have a chance. So, essentially you're willingly giving up control.

[00:20:29]It's pretty similar to that.

[00:20:32]Uh but the thing is it just checks cancel. So, for example, in this scenario, we have an infinite loop.

[00:20:40]And every once in a while, so for every 50,000 iterations, um >> [clears throat] >> we just check, "Hey, is this particular task still alive or canceled?"

[00:20:54]And if it's canceled, we just, you know, can cancel everything and return.

[00:21:02]>> [snorts] >> Um and this one is just a regular example. So, we can we can run it.

[00:21:08]Um 506 cancel.

[00:21:12]Now, notice that CPU worker stopped.

[00:21:16]And if we didn't have something like this, um in this scenario, the CPU worker never stops because we are never checking for cancel.

[00:21:28]Uh so, yeah, this was one of the examples I really want to talk about. Um next one that we're looking at is the queue.

[00:21:35]It's essentially just an MPMC channel.

[00:21:38]Um many producer, many consumer.

[00:21:42]Uh designed to perform for inter-task communication. So, uh essentially, I'm going to show you the code example.

[00:21:52]So, you have a bunch of futures.

[00:21:55]Uh we have two producers and two consumers.

[00:22:00]Producers produce things to the queue.

[00:22:03]Consumers consume things from the queue.

[00:22:07]And >> [clears throat] >> it it's just there's nothing too special about it. It's just a regular MPMC channel. So, uh we initialize it with a buffer. So, uh whenever the buffer is full, the producers would just uh will just block until uh until it frees up.

[00:22:29]Uh same goes for the consumer.

[00:22:31]Whenever the buffer is empty, the consumers will just block until we have something in the buffer.

[00:22:39]So, it's pretty straightforward. Um nothing too special about it. Um So, uh in our example, we are just whenever we consume something, we just fetch add. Uh one thing that's uh that's interesting here is the consumer API.

[00:22:56]So, like there are a bunch of things. Um these ones, um they're mostly used for internal purposes, but the ones which I found interesting were get one.

[00:23:07]Get one is the most basic thing.

[00:23:09]Essentially, you just take the front thing from the queue.

[00:23:13]And there's also this get one where you could just give min.

[00:23:17]Essentially, like this thing blocks until I get a minimum of that many elements in my buffer and only then it returns.

[00:23:28]It could also return early because the channel could get closed.

[00:23:33]Um that that's one of the scenarios.

[00:23:36]Um so it's something like that.

[00:23:38]>> [clears throat] >> So, for example, for example, we just produce something and our consumers, whenever they consume something, we just atomically add it.

[00:23:47]So, yeah, if we run it, um consume sum is 1090.

[00:23:55]Um and we have a bunch of these. So, yeah, so nothing too special, just a regular mpmc channel.

[00:24:04]Um so, if we go forward, >> [clears throat] >> the next thing that I wish we should talk about is the time.

[00:24:11]So, the thing is the timers are also >> [snorts] >> a part of the um the new IO interface. So, similarly, um I'll just go very quickly through this one since it's not that complicated.

[00:24:26]We've also like talked about this before in sleep. So, there's also um this timeout thing here.

[00:24:34]This timeout as in you could create um a deadline. Essentially, like sleep till that min that much time and um >> [clears throat] >> that's it. So, in this example, we first uh sleep for 25 milliseconds and then we are doing a timeout for 50 milliseconds.

[00:24:54]So, my goal with this example is we should be able to see that hey, this kind of thing actually uh sleeps the particular task. So, uh we are just uh seeing the time, how long it takes. So, in this scenario, it should take around like 40 ms. So, you can just run it. Uh but queue See, 40.2 ms. If you just run it a bunch of times, it's pretty similar to that.

[00:25:20]The next thing that we'll be talking about is the batch operation.

[00:25:24]Um this is for the other core primitives um that they provided in this release.

[00:25:30]The core differences, um everything else that we've talked about till now, uh which is um futures, groups, et ceteras, they operate over functions.

[00:25:42]But, batch here is a much lower-level primitive. Well, by the own definition, uh you can think of batch as lower-level concurrency mechanism at the operation layer.

[00:25:53]Currently, it only supports these things. Uh eventually, they plan to move almost everything over to it.

[00:26:00]Um so, let me just give you a small example.

[00:26:04]Um >> [clears throat] >> Uh with batching, uh we can kind of add a bunch of low-level uh batch operations. So, for example, um uh we initialize uh slice like before.

[00:26:18]And uh we add two operations. First one is file write streaming uh for So, for that particular file, we'll be writing this thing.

[00:26:27]Um the other one is We're just We're just writing to std out. The other one is batch up second.

[00:26:34]So, what this thing does is um we can enqueue these operations, and this thing would essentially start this operation, but await concurrent. This is going to wait until at least one of these previous operations is complete.

[00:26:52]Once it's done, then we can handle that particular thing. So, once we are here, we are essentially know that all right, some IO operation is complete. We can, you know, uh get some sort of information, deal with it, uh update our internal state or something.

[00:27:09]And then when we call next, uh it just keeps moving. So, essentially it's you can think of it like you have a lot of IO operations, you can sort of enqueue them. Um Uh they the background uh IO or threaded activation handles it.

[00:27:23]Um and it just kind of lets you know that hey, something is done. So, you can um keep track of them.

[00:27:30]Pretty much this. Um so, we can try running this as well.

[00:27:35]Um This is this is a nine.

[00:27:39]batch Um yep.

[00:27:41]>> [clears throat] >> So, as you can see, the batch operations they are pretty small. So, it doesn't really matter. So, if you run like them like a multiple times, like the ordering could change because uh they are pretty similar.

[00:27:54]Um so, yeah, the batch operations got complete because they write to STD out.

[00:27:59]So, one batch operation completes. Um >> [clears throat] >> then we can kind of keep track of it, then the other one completes, we can see both of them.

[00:28:08]So, yeah, like this kind of was the um much uh high-level information about this one since it's um it's work in progress. Um I'll eventually be adding like a lot more things to this, but currently this is all it supports. So, this is also why I kind of added this at the end of the video. All right, so that was it for this video. I hope you enjoyed it. And if you are interested in low-level systems engineering content, consider subscribing.

[00:28:40]And uh if you want to watch another video where I kind of build something with the async IO, uh here's another one uh where I built a file downloader with the new async IO. I hope you enjoy it.

[00:28:54]Yep, see you in the next one.