This video demonstrates how to develop audio playback applications on Linux using PipeWire, covering key concepts including audio buffer management, sine wave generation for testing, event loop integration for real-time audio processing, and circular buffer implementation for streaming audio data. The tutorial shows how to create audio streams, manage buffer sizes to prevent crackling, and integrate audio processing with custom event loops for applications like VoIP softphones.
Deep Dive
Prerequisite Knowledge
- No data available.
Where to go next
- No data available.
Deep Dive
this audio interface is actually kinda niceAdded:
Yo yo yo. What's up? What's up? What's up everybody? Welcome back. Uh today we are on day two of trying to work with Pipewire to do audio output on Linux. Um so the context here is I moved around a month ago. My doorbell calls my cell phone. Uh I went for a jog and ate [ __ ] on the concrete. And you know my my my phone also wanted to know what the concrete tastes like, right? which means that the screen shatters, it doesn't turn on. Uhoh, I can't answer my door anymore. That pisses me off, right? I don't think that my life should be tied to this like losable, breakable little thing that I carry with me all the time.
I just don't like it, right? And so I want my doorbell to call my door or my computer, you know, like a normal human being. A normal human being answers the door by clicking a button on the computer, right?
Um, and so to facilitate this, we are writing our own voice over IP soft phone. Um, because I think they all suck and I also want one for other purposes anyways. And also, it's funny. Okay, it's just funny to do this. Um, and one of the things that you need to do if you are putting a phone inside your computer is you need to be able to send and receive audio, right? You need to be able to hear what the person is saying to you and say things back to them. Um, and so that enter this project, right?
Um, Linux audio is like kind of funky.
We're going to skim the details really quick, but it's basically like there's a driver. The driver um has an exclusive interface, right? So like one person can do sound output to each device. You know, my computer might have many devices. When I like plug in my headphones to the front of the computer, it's going to want to switch from using the speakers to the front port, which means that like those are two maybe different devices, but like in my head, they all act as like one sound output thing, right? So what they'll you what what Linux distros will typically do is they will throw a like sound server on top of the driver interface. Um that that sound server has changed many times in the like 20 years I've used Linux. I think when I first started it was using something called like ESD um and then it switched to Pipewire and now I'm using Pulse Audio. All of these have like backwards compatible interfaces in some way. Um but in my mind I'm like okay if my computer is running pipe wire I should try to talk to pipewire. There are other ways of doing this but let's say let's just see how hard is it talk to pipe wire. The other benefit of us making like a pipewire library is that um for the Wayland compositor that we are also working on um we need to be able to do video streams which like OBS the way OBS is getting this stuff is um also through pipe wire. So writing a little pipewire abstraction is like not the end of the world. Um, so last stream we were looking at this pipewire audio source playback example. Um, we wasted a whole bunch of time on like dynamic library loading because my argument here is that um, I don't want to like link pipewire because other people's Linux machines might not be running pipewire and obviously nobody else is going to run this program, but my own Linux machine might also in the future not run pipewire, right? So like the ability to detect at runtime which sound server is available and like fail gracefully in the case where pipe wire isn't installed but you have als you know like that seems like a reasonable goal. So we wasted a bunch of time on like dynamic library loading. Um and we got to the point where uh we can get into the audio loop.
Right? So here he's calling some callback to fill some audio buffer. Um but we never populated that audio buffer with anything. Um, and so the goal for today is to like finish going through this example and then try to maybe migrate there's so there's actually a version of this thing that runs within like our event loop which is nice. So like try to migrate to that then maybe we try to find some sort of sane playback stream interface. I don't know what that is yet. Um, but we'll see. Um, so the example seems to say uh initialize me some main loop. This is an audio thing and you know it's music.
Maybe we change this name later, but we're doing some sort of audio playback.
Um, we create some plain old data for some reason. I don't really know what this does yet. Uh, but somewhere we say, hey, I would like to have two channel floatingoint output at 44K.
Um, then I don't know what this does, but we're like creating some params structure here. What is does this get used here? Here we say I don't know. I don't know what this does yet, but here we're saying like make me a stream of audio. And here is some flags associated with that thing.
And then run. And then on run, it calls this function. And this function is supposed to fill the thing with audio.
And so step one is fill the thing with audio.
So let's do some sort of sine wave. And what do they do in their example? Um, they say, "Sir, please give me an audio buffer."
They say, "Hey, what the [ __ ] is is there a buffer here?
I don't Okay, we're going to have to take a look at this, but okay. So, let let's start with just implementing this part. We're like, sir, I would like one of your buffers, please."
So this is like a con b is uh okay we need to like get our user data as our whatever the [ __ ] we call it. We have a data. User data is now called data. What a bad name for this. Uh but we need to pointer cast the user data that they give us as a void pointer back to the thing that we have down here that we've made that we've said please when you call this function give it to me like this [ __ ] Here we I guess at this point it's fine. I mean you storing the like dynamic library link in here is like a little silly but it's not the end of the world. And in here we're going to want this pw stream dqbuffer but this is not a thing that we've generated a binding for yet.
Uh but we can just copy paste this name into this file and then we will have it.
Uh so we call this this guy is just data stream like this and then we say uh const B is what they want.
We don't know what this means yet. If B is null return and maybe we should log but [ __ ] them.
And we need to align this thing as well.
Classic.
Classic.
He's like, "This thing doesn't exist."
But I put it in there.
Did I not?
Did we [ __ ] this thing up in some way?
Did I forget to save Huh? We've got some sort of cash problem here cuz I bet you if I rebuild that thing will show up.
Yeah. Okay. We we we added we have a bug in here somewhere. He's not correctly noticing when the input Oh, because this thing needs to be add uh input file input that comes from B path.
function list otherwise he doesn't know uh he doesn't know to track that file and watch for modifications. Oops.
Uh actually maybe add file arg add file arg that's what I was looking for.
Does that work? There we go.
Cool. We're so [ __ ] back.
All right, cool. So, we have a buffer and then can we look at like what the [ __ ] this thing is?
Uh, strruct pw buffer. So he's got a buffer.
He's got some user data.
He's got some size.
Buffer is one of these and he has many datas and many metadatas.
Okay. What is our data size?
Uh, why is this not all completing?
That's a little annoying.
That's a little annoying. But we can specify the type here to help it out a little bit.
It's like something like this, right?
Uh I guess here that means that we can do or else return like this and then we have this thing.
Cool.
And then uh buff Whoa, I'm lost again.
DQ buffer return. Right. So, we look at this buffer.
Can we do a little like dot question mark on this? Yes. Then maybe not. Okay.
Then we want to look at end datas.
That's what we were trying to do.
Uh, oopsies.
Why is this only getting called once?
Oh, because we dqed the buffer. So, he is only getting called once because we actually have to send the thing back out now. So, let's do that as well.
PW stream cube buffer which we need to get the uh okay I don't know we'll figure it out and he wants the data stream and the buffer.
Okay, nice.
Okay, so he is check he's null checking the buffer data.
So if buffer data uh data I think we can check if buffer end datas is equal to zero is less than one maybe we we give up and then we say like now we can look at data zero.
Okay. And how do we know the length of the data?
This thing is a strruct spa data which has what a max size of data like what max size over stride.
Oh sure. So this is the stride of one sample. So this is the size of the the buffer in bytes and then this is the number of frames that we can push in which he may have requested some amount of frames as well. What is that be requested?
Okay, can we look at the documentation for that? Because I want to know what this means for playback streams. this field the suggested amount of data provide. This will be the amount of frames required by the resampler. This is only suggested though. So he might be asking for something that's basically less than the amount of data we can fill in maybe to prevent like over buffering, right? You kind of want like the data to come in now. So I guess that makes sense.
Uh so we can kind of say uh like data available frames is data.
Can we convert this into a slice? I think we can right out like data buffer. We need to find better names for all these things but it's fine. We have data dot.
Wait, am I high?
Oh, we've used data twice. Damn it. BD BD dot max size.
So, this is going to be um [ __ ] So we kind of want to like look at the data from zero to this, right? This we've turned this thing into a slice now out slice bytes and then we want this to be like as a slice of floats, right? So we're going to like do out slice bytes and do like a stood mam slice bytes as slice of f32 because we're asking for floatingoint samples. Seems reasonable.
And then we're going to say uh requested is uh I'm just going to I know this like I think that first time chat being like speak in a different language is just instabban, right? That's instab.
Uh so we'll say frames like fill frames I guess is going to be This should be a F32. I don't know why he doesn't like this over number of samples. Do we is number of is like number of channels hardcoded?
What do they do?
Yeah, they do. Uh over channels.
So, we'll kind of do that as well. We'll say we have two and then somewhere we use this below. So we'll say here num channels and then I guess we just kind of put some sine wave in there, right?
Uh, so I guess we do uh four zero to fill frames, we generate more of a sine wave, right? And put that into the outs slice. So we want to look at like out slice um I times dumb channels. So probably like 4 0 to num channels. We want to put this sine wave in there and we kind of want to do this is equal to sample.
Right? And so here we just need the data to put in here now. Right? And is that it?
Oh, let's see. In Phil 32, they just put they just put data into this pointer, which or a pointer is the data pointer. So, I think that's what we've done. And then they populate this stuff.
So for us this is uh BD oopsies and what are they doing here?
So I guess this datas thing it's got data but then like chunks will point into this data I guess so this is like hey in this data where is this stuff right so we're saying hey I don't know why it's like that I would like to look at the documentation of chunk here valid chunk of memory but like What a confusing API. I don't really understand why, but I mean, we'll trust them, but like it I don't I don't It doesn't This isn't like a I don't know why you would ever want like multiple chunks here or something, but that's okay. Okay. sample is going to be some sign related to our accumulator, right? And so if we want to do like 440 htz and we are sample rate uh so like wave 440 htz wave sampled at four what is it 44 441,000 uh 4 44,000 44.1,000 right so I guess that means uh I don't No, I don't know.
We'll draw to right. We're sampling at each of these points.
So we need to convert the sample index into a point on this timeline and then call call sign on it. Right?
So the natural period of a sine wave in radians is 2 pi.
So we need 2 pi to happen every four 2 pi.
We want this to be equal to 1 over 440 of a second which is the same as how many samples?
Yeah. Yeah. So you have like some samples n samples when you want n samples to be 2 pi.
So you do sample index times we do our like grade 12 chemistry stoeometry. So we have uh idx is in sample times 2 pi over uh what we called n here n samples.
So this converts to radians. But then we we need n samples here which is uh however many samples fits into one 440th of a second which has got to be something along the lines of like 44 1 0 0 over 440, right?
It's some linear division. And we want something that's big, right? We want a number bigger than one. So it's got to be this way.
samples per 440.
Num is sample rate over 440.
Um this might not divide nicely.
So this is a floatingoint number probably, right? And so how do we how do we handle that?
I guess we are okay with just not being 440 htz.
Just output the wrong frequency.
Oh, sorry. You're saying if we do the wrong calculation. Yeah. Yeah. It'll be fine because if you do something too high or too low, it just like won't do it. You know what I mean? It'll be fine.
Um, we'll just say ish, you want floats for sign, just use floats. Uh, that's a good point.
That's a good point.
Um, okay. So then act, this is going to represent the frame index.
Um, so we're going to do like data act plus equals i over sample rate, right? So this is like the index of a sample.
Yeah, that makes sense, I think.
Um, yeah, I think that makes sense.
Uh but then this thing has to be so we have act times 2 pi over samples per 440.
That's what we wrote down.
And then this is wrong. This should just be one over sample rate.
Yeah. And then maybe this kind of works.
What is our stride here?
Two times size like numbum channels times size of float.
This seems like kind of like it might work.
This seems like it could work.
Uh this thing we want to cast. Oh, we have a out slice U8 pointer is a pointer to U8 which is an int cast of BD data.
But like why are we doing this when we could just take this as a float pointer which means here oh because max size is in bytes but that's fine. So then we can do uh we can do out slice pointer out slices out slice pointer uh zero to fill frames times no channels.
Yeah, that seems right.
That seems it right is Oh, outs slice len is not there. [ __ ] So outs slice len was going to be uh data bd max size over size of f32 something like that which is like now just the same thing as that's fine.
which yeah, which is just the same thing as stride. So then we do divide by stride here, which is what they end up doing. Okay, what's matter now? Comp time float and comp time int nonzero remainder one uh Do we just need to convert this to a float?
Yes. Okay.
Thank you, Dr. Zule, for the sinewave picture.
Okay. I don't hear [ __ ] I don't hear [ __ ] Can we print I mean, that does look relatively sine wavy.
Um, I guess it's possible that it's like out of range.
Okay, let's stop printing for a second.
And I want to see if like does my sound bar even know this thing exists.
It does think it's playing audio.
You guys can't see it behind my camera, huh?
But like, it does think something's happening here.
You know what I mean? It's just I'm not hearing it.
So, not sure what's going on there.
Uh, maybe we are doing the math wrong.
Maybe we are doing the math wrong.
Too low frequency. Yeah. Or too high.
Um, do I do like have I [ __ ] this up?
So this guy uh let's say that we're like this buffer or this we are asking for this many frames and this is fill frames.
And then uh can we do something like write down uh we'll say we we'll say like con start was data act and then we'll say advanced this many cycles or samples, I guess we'll say.
And that's a bug, right?
Yeah. This should just be plus one.
Yep.
Yep. the I [ __ ] up. I was accounting for the like sample rate multiple times.
Okay, so that made a noise, but I don't know if you guys can hear it.
It's quite loud. Just full warning.
Let's actually uh before I send it to you guys, let's divide this by uh divide by two.
Okay, which of these many things in this graph is making this noise?
It's playing into my speakers, so I should be able to find it.
Where are my speakers?
Here.
There you go. Take that, [ __ ] But it's working. It's working.
All right. Audio playback working fine.
Yep. That's 440. Sure is.
Morse code pitch. Yeah, dude. That's some Morse code [ __ ] if I've ever heard it.
Um, all right. So, now the question is is how do we like hook this up to like our application?
Right. We've kind of been working off like a pipe wire main loop, but like we like to do singlethreaded [ __ ] because we're too stupid to use threads and we like to use like an event loop and last stream we thought that wasn't possible but googling off stream I found it and then a YouTube comment also found it. There is this thing called like pw. There's some sort of like get file descriptor.
Get fd or something. PW pipewire pd or something.
Uh maybe it's in here.
Spa get fd. Uh I feel like it was like called like pw loop or something.
I feel like there was pwl loop iterate and pwl loop getfd. This is the one. So there's an example here in g main where they're hooking up pipe wire to the a gibb event loop. We're not a gibb event loop, but we are an event loop. Is ft short for it's for file descriptor. File descriptor.
So here instead of running the PW main loop um can you do main threaded audio without thread being unstable? Never tried. I don't know. But if not, we can do the same thing. But like I just want it to be we can like stick a second event loop in a second thread. There's no reason we can't do that. But I just don't like the idea of forcing that them forcing us to use their event loop, you know?
Um so how do we make one of these loops?
So what are we doing right now? First of all, let's do a little get status get add-it commit-m smile.
Uh, we are calling pwm main loop new and which is giving us a main loop, right? Probably we don't want that. We want a pw loop instead. So, I think that we can just call pw loop new instead.
We'll see.
Uh, PW loop new. And this guy's going to be like, "Sir, I would like a PW loop, please."
And we need to restart the LSP because he doesn't know about that thing.
And that's not even good enough. So, we need to full quit. Everyone who says type LSP restart every time I do this, you're wrong.
Um okay.
So now we just have a loop directly I think and then instead of doing pwm main loop run we should say I have my own event loop sir var loop is stud io loop please uh chain which needs to the ability to chain events needs to stash those events somewhere. So we do chain buff is 100 U size. It doesn't really matter. We're not using this yet, but we might later. And we want to register with that event loop.
Hello. Oh, we can write try probably. We want to register with our event loop.
This thing. So we say try loop register handle is a probably how are the get fd or something, right?
Um, PW loop get update this thing.
Um, so DB.PW loop get FD with our loop.
Uh, we want to probably pull. Do we want to pull on read or write? Can we look at the API docs for this thing?
Cool. Sick API docs. Do you guys think we pull on read or write? Probably read, right?
And probably not on write. That would be my guess.
This is my guess for what they want.
Okay. And then can we do a while true cone event is our event loop uh poll forever.
Uh if uh I guess or else continue here we're expecting to get this event and this event only. So we're going to assert event is zero.
And then what do they do in the example uh they attach it to the pipe the uh their gibb event loop in a callback which is this thing dwl loop iterate and then that's it. So, we kind of just do this.
Uh, but they said something about down here about having to enter and leave when you enter the event loop. It looks kind of like this only has to be called one time. So it looks like we can do like PW loop enter here and defer loop leave here at like the very top level. That's how cuz this main loop runs forever, right?
So it sounds like you can kind of do this and then maybe that's it.
It seems kind of like that's it.
Uh, we do need to have PW loop enter and leave.
Just ignore this for now.
Holy [ __ ] it works. Holy [ __ ] [ __ ] That's so much better. I wish this was how their default example worked because that's so much [ __ ] better, man.
That's so much better because now we have like complete control, right? So now if we wanted to do something like also on the same thread every once in a while print the word hello once a second or even just print the state of the accumulator in data we can do that right so we can say give me a vio timer service uh looks like he wants a general purpose allocator uh looks like we don't have one huh Uh, stood he s allocator for now. We'll figure this out later.
We'll figure this out later. He wants some sort of GPA. Is that a bug? Why does he want a GPA using ST priority Q? I see he's got our event loop and he's got a service ID. So here we'll say one. We are going to switch on the event. If we get zero, it's time to do this thing because we said that's zero. And if we see one, it's time to handle our timer.
And we can say um timer let's add a thing that happens um in 1 second and that thing is going to be call back ID2. We usually give these things names but that's okay.
And so like ID2 is the part that does the logging every time our timer expires.
Hello. So we say like hello and then we reue this thing.
So the timer fires and then we re we reue it.
Can I I think I have to remove and readit which is funny. This is a bad API.
Yeah, this seems kind of like a bug.
We should have like a rearm function on this thing, but that's okay.
That's fine for now. This is just an example.
Uh that's okay. This is just an example.
Okay. So now We see on the same thread, hello printed once a second while the audio is going.
So that's pretty sick.
Pretty sick. Maybe add a timer type one choppers repeat. Yeah, there's some there's some bug. I I think rearm is fine cuz like really it should be the responsibility of like if the timer fired somebody like asked like somebody has to handle that. So, I think it's fine for them to have to choose whether or not they rearm the timer, but like just having to like reallocate a handle is silly, but that's fine. Doesn't really matter. Doesn't really matter.
So, that's pretty straightforward. And now that means that we can start doing whatever the [ __ ] we want without worrying about like mutexes or anything, right? So what we want to do is we want to find some API that like hides everything that we have here somewhere else, right? So like I guess the question is like how are we going to use this thing, right? In the context of like an RTP stream, which is what we're going to be doing for our telephone, right? We are going to have some sort of like UDP listener, right? who is going to be getting packets from RTP from from somewhere else, right? From someone else. Every once in a while he's going to get a packet and presumably he wants to like feed those packets into a buffer.
On the other side of things, we have every at some interval who knows what this like pwl loop iterate is going to call this call back and ask for data, right? So somewhere else, oh my god, can we do like somewhere else? We have uh pwitter.
So he is going to be like pulling data from the wherever this puts it, right? I guess.
So this could be some sort of like in terms of like API ideally we have some sort of like pipewire service over here who like basically hides all of this stuff and in maybe we as the RDP service he like asks him for like a stream and so he gets like a handle doggy. Oh, there she goes.
Um, yep. So this guy holds like a stream and when UDP stuff happens and he gives him some data, he like uses this handle to fill a buffer somewhere and then the on the like on process callback on process will like pull data from here, Right.
Okay.
So, what about our case where we're generating like a sine wave sign?
I guess here somewhere we're going to have some sort of like fixed length buffer where we're willing to generate into the future if we want to fulfill the same interface. So, we can say like we'll generate like you know 300 milliseconds of buffer, right?
So like this like on process I guess we'll like consume some amount of this buffer and then he will either he says hey I consumed some data so you can fill more or we like fire this on a timer like Because like in the in the RDP case in the RDB case, the the addition of data to the stream is completely external, right? It has nothing to do with the pipe surf. The pipe surface just pulls it if it's available.
So like for in in the case where we're generating and filling like you kind of want him to say back to him and say like, "Hey bud, there's data available."
But like in reality in the real world doing it like the the trigger is external.
So I think that having a callback API in here is just like over complicating it.
I think that for our purposes we can just do this. We can like refill the buffer on a timer as long as the timer is like small enough that the buffer won't run out.
Right? So, if we buffer like 400 milliseconds of audio and fire the fire the timer every 10 milliseconds, like what are the chances that we run out?
You know what I mean?
Let's try that. And if it feels like [ __ ] it feels like [ __ ] Yeah.
So, maybe it's like PW service.
key needs to hold all of the stuff that lives in main right now. This data structure is like completely application specific. So I think we like mostly ignore that for now.
This guy is going to hold the dynamic loaded libraries for these things.
He's going to hold the event loop.
Is it a pointer or what?
Yeah, it's a player.
He's going to hold this thing.
And I guess the the streams he might have like many streams, right?
But maybe maybe the stream is owned by the person who calls into him to like ask for a new one. So maybe he doesn't have to hold that.
Okay, then we'll just start like writing stuff, I guess, and see what happens.
Just like push some of this in here.
Uh, I guess he also has like P PW lib, which is a stood dime lib and spa lib.
Boom.
Boom.
Boom.
Okay, cool.
We initialize pipe wire for sure.
We create the loop.
We probably have to there's probably some like error defer pwd in it.
There's probably some all like equivalent here.
Uh there's probably some clean up here as well.
Then what is going on with this properties?
This properties thing is like a specific stream, right? So this stuff comes later, right? We're going to have some sort of like well let's let's finish this function first.
Loop is loop just take all these things copy paste them.
Uh put a dot find the colon put equals paste co change. Boom boom boom. All right.
Uh okay, easy. So then we have like make like make stream make audio stream new like new audio stream returns a audio stream.
I don't know. We're we're spitballing.
We're spitballing.
For now, we will assume that we have like a hard-coded sample rate and number of channels and stuff because I don't want to deal with that. But it looks like this guy is going to hold one of these.
Okay.
And he is going to be like, "Hit me with that shit." Surely.
I don't know.
Okay. Uh, we need our event loop or our hype wire loop in here and we need some properties.
Do these properties have to be freed?
How do they do it in their example?
In their example, when can we get rid of those properties?
I don't see any freeing of this thing, but like surely, right?
Like surely maybe they get owned by the stream and so the stream deinitializes them automatically.
Seems like it, right? Can we look at the documentation here for props? Ownership is taken. Yeah. So, we don't have to worry about that unless this fails. So, like uh fix me if pw stream new fails can it? We need to free props.
Do we?
It can fail. So like this does not say what happens if this returns null, right? If this returns null, I would hope that it frees events. But like we should just say say uh if stream returns null, does props get freed?
I hope so. Smile. Okay.
Uh data data data data data that's got a pro I assume that that data is going to be like us.
It's got to be like we said that we were going to push data into some buffer somewhere, right?
So the data here is ideally that like buffer, you know what I mean?
Which means that like we want this at the very least to have a stable address. So we could maybe do like an object pool here of stream buffer. I'm spitballing. We're we're feeling it out, right? And then this guy can have like uh stream buffer index or something. I don't know. We'll see.
You want the answer for the pipe? Yeah, I'll figure it out later. Why not doing the fix me now? Because I'm trying to figure out other stuff first. Like, how do I say this? I I like to work like um you know how like sculptors or like artists work where you do like blocking first, right? You start with like a big rock and you're trying to turn it into a person, right? You don't like do the guy's nostrils and like facial hair right away, right? you kind of like get the rough shape of his head and then get like the rough shape of his body and then like once you have like the rough shape you kind of like go into more detail in each point. I find that for myself working that way in code is easier. Right? Once I find the shape of the of the thing, then I can go into the details. Like now that I know the shape, this part goes here. This part goes here. But like until I have the overall shape, focusing on stuff like this is like kind of a waste of time because I don't I don't know if it's going to live here. You know what I mean? Until I know the shape, there's no point. So that that's kind of that's how I work. So, it's just I put a note here so I don't forget, but like it's important to like take a look at this at some point.
Um, cuz like this is going to require some research, you know what I mean? And I just don't want to do that right now.
Um, so the data for this thing ideally we have like um I don't know yet.
This might be inline.
I don't know like how many samples is reasonable here. Should this be dynamically allocated? I don't know.
So, for now, I pro I mean it it probably should be.
I It probably should be.
Maybe this guy takes the buffer in.
Maybe the buffer is owned by whoever creates this thing. And so then this guy can just hold the buff and then he can have like his like re his uh like read index and write index.
And this is just turning into a circular buffer which we already have a type for.
Nice.
Nice. Nice. Nice. Nice. Nice. Okay.
So, does that mean does he even have to hold this now? Yes, we need cuz we need to hold the read and write pointer somewhere. Can I tell you about the pipe wire API? Sure. If you have the answer, sure, I'll take it. Although I'm going to want to double check myself later.
Uh, okay. So, we're gonna have like stream buffer is allocate like acquire this thing which we're going to need some sort of allocation for.
Uh, okay.
And then this thing gives me. So here we can just do like SB val here and then I think that's fine, right? So then we can just return SB index is something and stream is stream.
That seems fine.
Uh if you create a stream, you can ask for it to allocate extra bytes that you use. So if you need to hold a pointer to some stream then a pointer some user data you can hold the stream pointer and use the extra data me owned by the pipe bar for client use.
Uh oh I thought you were trying to tell me if the stream I thought you were just telling me telling me if props gets freed.
Um oh you saying that allocations can fail and that's why return null.
Ah whatever. Ah I don't care.
We'll figure it out.
Uh okay, we need to initialize this thing.
So items is the buffer.
Head is zero, tail is zero.
Okay, cool cool cool cool cool cool cool cool.
So now our on process we need like on process two and we'll say stream events two calls on process two or something while we sort all of this out.
So on process two literally just has one of these. Let's do let's do a proper like stream buffer is this.
He's just a circular buffer of floats.
Okay. And this guy is going to do something very similar to this but except he's literally just going to pull from the buffer. Right. So instead of doing some generation here he is fill frames is now limited by three things. Right? There is the amount of data in the buffer we have. There is the amount of data available for writing and then there's the like number of requested frames. It's like a triple triple min instead of a double min now.
Um the data that cames into the stream buffer.
Boom. Okay. Uh oh.
Problem problem.
This data needs to be able to call gen like pipewire functions. Is there anything else that he was using data for the accumulator which we don't care about?
Okay, here's what we're going to do.
Here is what we're going to do.
These are globals.
Suck a [ __ ] Suck a [ __ ] These are globals.
And you'll somebody says, "Excuse me, sir, you can't write globals." And I tell you, shove it up your [ __ ] I can do what I want.
You'll revert within 30 minutes. Ah!
Ah!
You can't stop me.
You can't stop me.
Okay, I guess it makes sense for these to these to be nullable and then we can kind of use that as a flag, right? If pwl lib is null, then we open this thing. We just never close it and then we initialize the global this thing.
Don't stop me now. I'm having such a good time having a fall. Yeah. Yeah. We You can't stop. No stopping me.
This feels fine.
This feels fine. If it's not fine, then like you know, get [ __ ] me later. But ah then there now this guy can just use the dynamic thing here.
And then he doesn't need any extra information. No. Great.
Oh, he needs a stream ID.
Okay. Well, then that means uh that this thing here needs to be an audio stream, not a stream buffer, which is fine. This is now going to return a audio stream handle.
uh just so that we don't forget.
And this thing now holds the stream buffer directly.
Yep.
Ass.
Yep.
I guess we can just do this after, right? We make the stream uh fix me error defer close on acquire. I don't know how to do that yet.
Okay.
Oh no, we how [ __ ] backwards.
You need to get the pointer to the thing to add it to the stream. You need to get the pointer to itself in to do this.
That's funny.
Uh, so here there's a fix me. I guess this aner self pool release handle.
Um, props doesn't need to be freed because if the stream is no, it means that it was never created. But here we called properties new, which I assume is some sort of dynamic allocation. I could be wrong there. So, I would imagine that there's like some sort of like symmetric PW properties for you or something, but I don't know.
I don't know. I'm making stuff up.
Okay, that seems like kind of fine. So then on process two, he gets a he gets a PW service audio stream.
And now here we have ass here.
And then here we have ass stream. Now we have a buffer. Beautiful. We figured out how much stuff we want.
Aren't we missing some sort of min here?
Yeah, we should be doing like uh max out frames is here.
Uh max in frames is like circular buffer count. We have uh con requested frames.
I guess this is like over two probably.
um requested frames is be requested and then we're saying like uh fill frames is the minimum of max in frames, max out frames and the minimum of that and requested.
Beautiful.
Cool.
Circular buffer writes SB. Well, it was stream buffer before, but SB for circular buffer is pretty funny. That wasn't intentional, but I like that.
Um, okay.
So, now in we're not using this like data accumulator thing. We're literally just going like for each frame like pop the frame.
Right. I think and this is uh crash if we hit this because there the calculation here should prevent us from ever popping more than is in the buffer.
Okay.
And then we're like ask B.
All right.
Cool.
Okay. So now we want to do something where we create the service.
Uhhuh.
We also need the service function on our service. Huh? That calls like uh Oh, we also probably want to enter the uh the main loop, right? When you do enter main loop, maybe that has to happen late.
Like in their example they were like in gain like you kind I think they you want to do this as close to the main loop entering as possible. So maybe we need this. Unfortunately, this service needs an enter leave, which is like a new concept for us, but whatever loop start.
I don't see a way around it.
Did I Oh, maybe I didn't restart this thing yet. That's funny.
And yeah. All right. We'll just do it like that. It's fine. And then we have service.
And that means that like the init function wants to take in the uh event loop as well as the service ID.
so that he can just register himself the loop directly.
I think that's fine.
and he'll just say like before I'm done, let's just do a quick little loop register here.
Given our PW loop, beautiful and v ID is our service ID. Beautiful.
And now did we write our service function? We did not, but it's just whatever was in here. PW8.
Fix me. LOL check error.
Uh, but there we go. We're looking we're looking a little bit like something. So now when we construct our service, we have this um we say this guy takes in the loop now the event loop and event zero which we were claiming. All of this is going away now.
This comes first.
We bleeping and blooping yet. We're getting close. We definitely made some noises at a point.
This stuff is all dead now, I believe.
What was this builder for? I guess this is dead.
Nice. No, it was down here for the raw audio build down here.
Oh, we didn't. We've missed a bunch of code down here. All of this has to be added to the stream when we create it.
All of this goes here.
Uh, and probably just like in general, there's probably some cleanups to do here.
Uh, just like fix me. Everything below this point has not been checked for cleanup.
We'll deal with that later.
Okay. So, we're like, scrap all of this stuff and we're like, oh, did we do a pwnet yet? We did.
So, here we're like, make me a new audio stream. And here is my sample buff.
And let's just make it like a meg of uh you of floating points, right? So here we can do like maybe a meg is a lot 500k.
500k. So this is over four because uh floating point.
Okay.
Cool. So we have our stream here.
Here we do a service loop start and loop end because unfortunately we think we need that.
We want to do that kind of as close to the loop as possible.
Beautiful. This is now uh this should be pw not called service.
That's a stupid name for this.
And now what we're missing is the part of this whole thing that like um populates the buffer, right?
Um oh, we can't call this either. We should call this a PW service. Maybe pipewire service.
Okay. So, let's say on a timer, let's do every uh Okay. So, if we have, let's do a little math. 52* 104 over 4.
Uh that is 131,000 out of 41,000 or 44.
We have around three seconds of audio here around right. So doing servicing the buffer once per second should be fine as long as servicing once per second doesn't cost so much that we miss more. You know what I mean? So maybe we do every 200 milliseconds we service we we add some stuff to the buffer, right?
And so here we'll do our accumulator that we're using for the sign stuff here. We shouldn't have deleted the code, but I think that it should be in here somewhere. Act where was the part where we were doing Oh, we have it on process. Perfect. The old version of on process exists. So we just here say every 200 milliseconds throw some more stuff in the accumulator.
and rearm.
Okay.
And is that it?
We have to do something with the sample.
So for zero to num channels, we push the sample into our stream from this thing. So we have like PWS uh get stream buffer with the handle that we have for this stream.
And we're going to go like try buff push no clobber of the sample or else return error out of room.
I guess we'll say out of memory to reuse existing functions.
And I think that should work.
I think that should work.
Uh, okay. We're missing some things.
These things need to be optional dreferenced.
This guy, these guys are no longer part of this API. We need our expansion allocator, which is just like a a normal Zigg allocator, but with some like metadata about like whether or not it supports free and the minimum size. So you can give it like the page allocator and say only allocate in 496 byt chunks and stuff like that. Um that's why this exists.
Um we need our pool.
So I guess we are kind of expecting to have somewhat less than four streams definitely less than 1024 streams. And we are expecting to come in with you know an initial allocator as well.
So we still throw those in here.
Okay.
I'm sure there's more compiler errors here. We have uh we'll use the general purpose allocator and a general expansion allocator. You know, you can do whatever the [ __ ] you want, [ __ ] Um, this thing needs to be variable, I guess.
Does it compile? We're here. This guy can fail. So, true.
Uh, this is not self DPW. This is just DPW.
Stream here is unhappy because I think this would be like or else return error make stream or something.
What's this guy mad about?
He needs the expansion allocator in case he needs to free. Okay, this guy's mad because we need to strong him.
Get stream buffer buffer doesn't exist.
Uh we'll which will turn a stream buffer which is like maybe a pointer to a stream buffer. He takes in the handle which is an audio stream handle. And here he's like uh return self pool get handle dot SP easy get stream buffer stream is mad because probably the definition of this thing didn't try uh expected optional type found void.
Oh nice. This will already fail.
Beautiful.
Okay. Does it work?
Zero frames.
Did we forget to cue the buffer at the end?
No.
Uh, I guess it's one of these minimums is bad.
Oh, we probably never populated the initial data.
We probably need to populate this thing off rip. No, because he tries to populate after 200 milliseconds at least service. Uh push some dead.
Okay, we are I do see that push from data showing up every once in a while.
So, it should we are trying. Uh BRX, thank you for the prime or the resub tier one. Much appreciated. Much appreciated. So that's not good.
Okay, let's um print just like the three numbers.
Max out frames, max in frames, and requested frames. Which one of these is [ __ ] max in frames.
So, the stream buffer is no good, but we're putting stuff in there.
Oh, we're only pushing one sample while um buff count is less than buff items length.
While we can, I guess it's more like um while true, we rip [ __ ] in here. We say push no clober. And then here we catch break.
And then we only increment this thing if it worked.
Close.
It is crackling sometimes, which is interesting.
I mean, it could be floating point error, right? Um, could be floating point error. Can we do accumulator as a u size and this is going to be as f32 or like float from int maybe? Yeah, let's do float from int.
And let's try to keep this float small.
So we're going to say act plus equals 1 and uh act is equal to act mod uh one cycle.
You know what I mean?
And then maybe that will fix the crackling. I don't know. There could be multiple reasons for it. So it is still crackling after some a little bit which is interesting.
The fact that this is odd is concerning.
I feel like it shouldn't ever be odd. So maybe the problem here is that the buffer is not a multiple of to I mean I think it would have to be um yeah the the pro this 941 is concerning you know what I mean so let's say um cost remaining is a buff items land minus this buff items count over two. We want to do this whole thing over two to make sure we can push two at a time.
And then this is uh numbum samples for zero to num samples. We push twice and on the second one it we should crash if it doesn't work.
Okay, that's not good.
Pushing this many samples.
What am I missing?
Pushing zero samples.
Oh, I'm an idiot.
Uh, this needs to be here. This needs to be four up here.
This needs to be not in a loop. I'm [ __ ] high in my [ __ ] mind.
This is what I'm looking for. So, we push the first one. That one is allowed to fail. No, it's not. They're all failures now. We push it however many times for numbum channels. So, we pre-calculate the number of samples. We get this buffer up front because we don't need to [ __ ] get it every time.
We calculate numbum samples here. We say for however many samples that we have room for in the buffer, we'll push that thing.
There we go.
Let's go.
Let's go.
So, this is really nice because now we have this like generic API, right? where the pipewire service all we have to do from his from like to use him is we initialize the service and we say we have an audio stream and completely independent of the ser the pipe service I will just populate as much as I can when I have data right so in the context of an RTP stream this would be triggered on UDP packet receive but for our case um how jittery realtime is the event loop that's a great We might have to fix that. It's using e.
Um, but if it gets really bad, then we can just we can move like we can always move the audio thread into its own loop and add like a little bit of mutx locking on the buffering.
It's not the end of the world if we choose to do that.
But like I like this API, right? This API is very nice because especially with audio, you need to take your dead headlines. Yeah. Yeah. Understood.
Understand. See if it clicks. I think the clicking is fixed.
I'm pretty sure the clicking was like a complete result of us filling an uneven number of channels. I'm pretty sure like I think that like whatever buffer size we ended up picking was just somehow resulting in some [ __ ] You know what I mean? Once once I did like pre-calculated the size up front, it seems to be fine.
Yeah. I mean, that's that's nice, right?
So, if we're if we're playing if we're playing a file, I sort of heard clicking at the end. I haven't heard it.
Oh, you know what? Sometimes uh the like the there's something where you guys sometimes hear stuttering and I don't which is like probably related to like this [ __ ] that's going on the like noise reduction [ __ ] that can that can like cause the problems. Uh but like I don't know I haven't heard anything so I it's fine from my end.
Oh yeah, API very easy, right? Because for for a file for a file, we would choose to buffer however much data we would just probably again on a timer just consume data from the file until we run out of room in the circular buffer. For a streaming audio, we just push data as it comes in. And we just make sure that we size the buffer such that we can never get too far ahead that we run out of memory.
for anything generated. We can just fill it on a timer as well.
And I think that's kind of all the cases.
Yeah, I'll take that. That this actually ended up being much nicer than I thought it would, right? Like if we look at Let's just delete all the code that's not being used right now. This is dead.
This is dead.
Based off of how I was feeling yesterday, I thought this was going to be way worse. But in terms of like actual amount of code, like initializing the dynamic library, it's not that bad. I hit him with the it works on my machine. Huh? No, but I mean you're seeing my machine. This isn't it works on my machine because it is working on the only machine.
Um, this is a little bit nasty, this like generation of an audio stream. And eventually we're going to have to support like, you know, stream ops, like stream format in here, right? Because like depending on the RTP stream that comes in, we're going to have to do something there.
Then we just have a buffer. The the processing loop is not that bad, too. I mean, there's like some confusion in here. We still don't really understand everything about this thing, right? Like why are there chunks in a buffer? Like there's a reason for it. I'm sure. I just don't know what it is. But like this is just copy pasted from the audio example. So it's like not that bad once it's hidden away.
And then from like a user perspective is great. Like main it's like give me a service, give me a stream, I'm going to put some [ __ ] in the stream. And I presume that we have the option to have many streams, right? Can we actually do that?
Let's make a second stream.
And let's say that we will populate stream 2 at uh 880. Can we hear 880 htz?
We can, right? I think we can.
Uh so we're going to do the same thing.
uh where maybe we scope this and we put into stream two just we'll copy the same [ __ ] but we'll just say uh samples per 880 and then I think actually technically the buffers are we need a second buffer as well huh Yes. 200 to 2 200,000. Yeah, that makes sense. We can hear to 20,000 14,000 if you're not 400. Yeah, because I guess I guess 880 would just be one octave above 440, which we can definitely hear, right? 440 is like which means 880 is like which we could probably hear fine.
Um I guess we need act two as well.
And then we kind of just scope this and do it twice. [ __ ] it for now. I just want to make sure that this works.
Stream two here.
Uh then we are doing samples per 880 which did we change that? We did. And then this is act two and act two here.
Huh? Why don't I hear that?
Oh, because I didn't do act two here.
Oopsies.
Hold on. I just want to make sure that like I can hear the 880.
Why does that sound just like 440?
Because I have samples per 440 here.
[ __ ] me.
There we go. Nice. Okay, that's audible.
So now, can we do two streams at once where we're playing two sine waves, an octave apart?
Hell [ __ ] yeah, baby.
That's pretty sick. Pretty sick. Yeah.
So, so interface-wise, you can just spawn as many streams. Oh, hold on. I want to check also. Um, sorry. Let's just turn this down a little bit for you guys. Oops. Not mute it. I just want to see. Um, does it show up as two streams here?
Yeah, it does. Interesting. Oh, I actually only turned it down for the stream, not for the recording. So, get [ __ ] Uh, VOD viewers.
Um, was that both? I only hear 80. I see I hear both. I don't know if there's some sort of like [ __ ] noise reduction happening, but I was hearing both. So, it does show up as two streams, which is interesting. Um, so like there's a world that makes it seem like you kind of want to like mucks yourself. You know what I mean?
There's something interesting. There might be like part of the uh part of how do I say this?
part of Pipear's API might involve some like mxing stuff. It would it should because at the very least it's mxing between applications. So I would love for it to be able to m between one between like multiple input streams on my own application. I would love for that to work. Um, so there's probably something there where you're I don't know.
Maybe you like ask it to not show it.
Maybe maybe there's some API to create like a stream and then substreams off of that. We can like look for that a little bit. I mucks myself every day.
Every day is a lot.
That's a lot. But like, good on you, I guess.
But What's wrong with mxing between apps? Go mx yourself. Thank you everybody. Never mx the streams. Thank you. Thank you.
Thank you.
So yeah, I think I'm happy with this for now. There's like details to be sorted, but I think this works. I was expecting it to be worse. So once we've kind of figured everything out, it's not that bad. Not that bad. All right, cool. Um, thanks for watching, guys. We're going to call it there. If you like what you saw, we stream most days. We aim for a 12:30 start. Try to end before 4. It is 3:10 right now. If you're watching live, these times are in Pacific time, which in BC is now always Pacific Daylight Time. We are never going back. Hell yeah. Um, was there anything else I wanted to say?
Patreon, YouTube members, Twitch, pay on any of those. You support the stream, you get access to the happy hour VODs.
Today's happy hour was about cleaning up some of this dynamic library stuff. So, we kind of generated like we wrote some code that will generate these str, you know, be attached to our build system and stuff. We were trying to figure out like, you know, how should that look like? If you want access to that, um, you know, pay in one of those spots, but otherwise, obviously, no pressure. You won't miss anything that important.
Is that it? I think that's it. YouTube, peace out. Two raid.
Related Videos
Agentforce NOW AMA: Build with React and Salesforce Multi-Framework
SalesforceDevs
490 views•2026-05-28
How agent o11y differs from traditional o11y — Phil Hetzel, Braintrust
aiDotEngineer
450 views•2026-05-28
Re: 🗣️📍theprophedu📍2026 GST 103 CLASS (E-EXAM REVISION)
theprophedu
636 views•2026-06-04
WEB TECHNOLOGIES UNIT-2 | Degree 4th sem BCOM Computers web technologies unit-2 full explanation💯✅
LearnwithSahera
1K views•2026-05-29
More tests are always better? How to use AI to identify tests that bring little value
Alliance4Qualification
335 views•2026-05-29
Search Algorithms Explained in 60 Seconds! 🤖💨
samarthtuliofficial
218 views•2026-06-01
People of Game of Thrones using JavaScript DOM
AltCampus
296 views•2026-05-30
Introduction to Problem Solving Part - 1 | Lecture 1 | Intermediate DSA
ascensionix
107 views•2026-05-29











