Linux Weekly 8: Small News, Long Projects

Added: 2026-05-12

[00:00:01]Today I'll be drinking from the thermos.

[00:00:08]It's actually quite late at the time that I'm doing this because this was a busy week and it's going to be an even busier weekend. So, I'm getting a head start on things. Uh, this week is mostly just a couple updates from last week and then also projects. It's mostly projects. It's probably going to be super long. First, there's a new copy failed this week. It's Dirty Frag and there is no patches for this. it was released early because somebody else found it or somebody else noticed. Um, most of these kinds of things are because the language models have gotten good enough to find security flaws, which is great, but like the other one, I'm skeptical. You know, I think that these kinds of vulnerabilities were probably introduced by intelligence services or folks that were dual employment for intelligence services. I feel like I had some questions about that from last week. And basically, imagine this.

[00:01:00]You're, you know, somebody from the NSA approaches you and says, "Hey, I noticed that you work for whoever making this product. I would like for you to introduce this vulnerability as a matter of national security. Here's a million dollars.

[00:01:18]All you have to do is have this weakness. And if you it's, you know, force issue, it's totally fine."

[00:01:23]Sometimes those people don't actually work for the NSA. Most of the time they do, but sometimes they don't. Uh, sometimes you have a foreign government that has an agent that says, "You should apply for a job at such and such company and we are willing to pay you to work there for 5, 10, 15 years and we want you to do this thing and you don't know and there's not really like a counter intelligence operation. There's not really a lot of intelligence footprint there to get on surveillance. you can only do like the most invasive and unamerican surveillance possible to even get an inkling that maybe something weird like that is going on. But that is a thing that happens. And if you don't believe that that's a thing that happens, you should look at the counter intelligence espionage that has been done against foreign companies to benefit US companies.

[00:02:10]Things like the windmill blade design thing. I think that's what that was. But there's probably some engagement challenge there for other things. But I don't care how paranoid you are. You're not you're not paranoid enough. So, there's more stuff to fix a patch by the time you're watching this probably or maybe not. I don't know. The dros were pretty slow about getting out some of the other updates and that sort of stuff. So, the security effects are secondary and knock on. And uh it's not paranoia if they really are out to get you. See, also um Snowden and Echelon before that and that AT&T closet where it's like, no, that's crazy. Who would splice into the fiber and observe everything?

[00:02:57]You know, it's like, oh, those times were real, but this time's just paranoia.

[00:03:04]Anywh who Dell and Lenovo have signed on to the Linux firmware like firmware update service so you can get updates for your Dell and Lenovo stuff. Although it's kind of selective. It's like oh everything down not so much. Uh that's pretty much it. Now project updates. One storage Microsoft storage spaces. It's terrible.

[00:03:32]It's so bad. I'm doing a striped mirror storage spaces recovery. And as I work through this, there's a really interesting failure edge case for storage spaces. The particular thing that I'm recovering is only four drives.

[00:03:47]So, it's really it's pretty trivial. But it was so bad that I decided to recreate a more complex but common failure scenario to see what would happen. And to my surprise, it it's bad. Okay, so let me rewind a little bit. What happened? Stripe mirror, four drives. Two of the drives are on the motherboard SATA. Uh, two of the drives were through a secondary SATA chipset that happened to be part of the motherboard design. So, the motherboard had like two native SATA ports and then two through like a J Micron or whatever.

[00:04:28]Um, and somebody was using this as their personal workstation. They're also using storage spaces. This was striped mirror uh REFS. Um, you know, kind of a setup.

[00:04:40]Pretty simple, right? All right. Here's what happened. The drives happened to be the one mirror was on the J Micron and one mirror was on the motherboard RAID is how they just randomly happened to set it up. The J icon controller crapped out unexpectedly.

[00:05:01]That took both drives in the mirror out.

[00:05:05]Storage spaces won't mount. It shows up now. Using storage spaces analysis tools, it shows that the one of the drives uh one of the drives is like dirty and the other drive is like okay, it will not mount because the situation is readon. Like if you use the tools and you do all the command line, you do all the stuff from PowerShell and it's like force online, it won't recover. So I set up analogs and I have one drive crap out on the controller and it's able to recover and everything's fine. It's just that when you cross the threshold of too many drives have failed, the whole storage thing goes completely readon and you can't access your files anymore at all. It will not it will not let you and you can't change the state of a drive that's online because the whole thing like the metadata is also all readon.

[00:06:00]So I created a catastrophic failure situation where I had eight drives, four drives on back plane one, four drives on back plane two and I didn't pay attention to because again striped mirror the stripe was on two drives here and two drives here. So you pull one back plane, you don't have enough drives to do. You you lose all of your mirrors on one side basically. Storage spaces can't recover from that. Like you can't just say, "Oops, the controller failed.

[00:06:31]Roll back, which seems insane." So I've been working on a utility for Linux um to recover from that actually for a few weeks understanding the internal metadata structures and walking the MFTt and doing that kind of thing. But I'm very surprised that story spaces are so brittle because it seems like the loss of a a back plane or the loss of a controller which will remove all the redundant like we'll take the array offline basically that you cannot recover from that in storage spaces because it does something to the other half of the array and it remembers oops all these drives went away or the J Micron controller failed in such a way that it was still able to write to the drives but it just marked them offline. And because it's readon, you can never just say, "Yeah, I'm going to have a little bit of data corruption.

[00:07:20]Just reset it, get everything back online, which seems completely insane.

[00:07:25]Like, did nobody test this? Is this really the failure mode?" And yeah, I've done the testing at this point in a whole bunch of different configurations with simulated failure where I'm just tweaking the communication channel and it fails. So, if you have experienced these kinds of failures with storage spaces or you do that kind of stuff and you would like early access to the forum thread to fart around with this, let me know. It's probably not going to be next week. It's probably not gonna be the week after. Probably like a month or two from releasing something. But I thought it was interesting that the storage space has failed so catastrophically. I was so brittle. Project one. Project two networking related. I've gone down and I mentioned this last week. I've gone down a rabbit hole. So this is the the BE 1800. I've literally been working on a review for this for months, but I've been kind of flumxed because this isn't bad. It says wired capacity 20G, and that's just summing together all of the ports. It's actually eight 2.5 gig ports, but I can actually do close to 2 and a half gigabit wirelessly with this.

[00:08:19]But you'll notice there's no Wi-Fi alliance sticker on this. There's no Wi-Fi 7. Wi-Fi 7 is a really interesting, fascinating rabbit hole of crazy. This is multilink operation, MLO.

[00:08:30]And the thing that I'm looking for, and so multi-link operation means that it can use simultaneously or with rapid rotation 2.4 GHz, 5 GHz, or 6 GHz. And the deeper I go down this rabbit hole, the more insane and baffling it becomes.

[00:08:46]You see, I started with enterprise Wi-Fi 7. The first devices that I had were that were Wi-Fi 7 were ingenious. And even their multilink operation is just rapid changing of the frequency that the thing uses. It doesn't actually use multiple radios at once, and it can be tricky to do that from a particular wireless client. There's not really a ton of good explainers out there. Um, the Arts people have done a good job.

[00:09:12]Like, we should probably just go give them money for this kind of thing. But I've kind of also been working on my own test suite and my own set of things that I want to do with Wi-Fi 7. One of the things is have about four clients on the Wi-Fi 7 setup and have all of them streaming Netflix and then have a fifth client that wants to do file transfers and then monitor what's going on. So, I've been working on a test harness to work with that and do that and have a lot of fun with that. And it's uh buggy and terrible and non-deterministic in terms of Oh, I got a result. Oh, this time I didn't get a result. That's weird. And the ROG Stricks GSBE 18,000 has been really helping. Is this on the band list? Can you even buy this right now? Um, and I'm also using Ingenious and some prototype gear and uh mostly using the Intel BE200 Wi-Fi 7.

[00:10:07]Um, but there's not a lot out there that actually does multilink operation right now. In fact, there's almost nothing that does multilink operation. So, I don't know. Okay. I also just got this, the Dream Machine Beast from UniFi. They didn't send it.

[00:10:21]I didn't ask for them to send it. And the reason like, so I actually shed out $1,500 for this. Thank you supporters.

[00:10:27]And also, I'm sorry for wasting your money. Okay. Well, you know, on paper, this is amazing. The original Dream Machine was amazing. I got one. It was terrible. I returned it. Um, I'm hoping the Dream Machine Beast is much better than the original Dream Machine. So, my favorite piece of UniFi equipment so far is their 25/00 gig campus aggregation switch. And it is amazing at the price point. It has a lot of warts and I'm the crazy rando that can find the warts. See, also crazy rando working on this. I'm also trying to bring up to speed more people on how to do this kind of testing and like put it together mentally. Sean, like teaching Shawn how to do this so that I can delegate. Um, and that factors in with Dream Machine Beast. So, I will be testing Dream Machine Beast for real, but I'm sharing this now because I know a lot of you have used UniFi equipment, probably more extensively than I can, and I depend on your stories on the forum so that I can dig deeper and figure out what they are and what's going on. And a couple of you had horror stories with the uh U6 um wall thing. Hang on, I had notes about this. Uh, U6 inwall access points.

[00:11:45]Okay, so the U6 inwall access points.

[00:11:48]This was an amazing rabbit hole a couple of you sent me down. You should look at this product. This product has a lot of really interesting features, but this is also an example of like why I have to like Ubiquiti is great. Like, let me be clear. Ubiquity is probably the best and most amazing thing for small businesses and medium businesses up to and even including the people that need 25/00 gigabit and I myself think that a relatively small business like 15 20 people 30 people depending on what you do if you spend six eight hours a day on a computer and like you're really hitting your computer hard you need 25 gigabit if your computer can you know if you're obviously if like you're on a Mac or a Mac studio they're just you know Apple doesn't believe in upgradability so 25 gig bit is going to be a little difficult for you there. Oh, you can just use Thunderbolt. Oh, sweetie. No, I didn't. They didn't design Thunderbolt and even Thunderbolt 5 to work that well. So, like even if you get the Sonnet adapter, that's a whole you're can go down a rabbit hole and it's like if you actually try to do that versus like this should work on paper. A lot of people assume that because it works on paper and they never do it. No, no. I I am where the rubber meets the road. And where the rubber meets the road with this may be interesting. where the rubber met the road with the U6 inwall access point is there's a software update that solves a bug with the hardware and the software update that solves the bug with the hardware means that the switching capacity of those is now on the order of like 300 to 500 megbit and this is because the CPU is doing the work. See, when you have a switch like this, you have a computer in it. You have a general purpose computer. But the general purpose computer generally tends not to be terribly fast. And then you have a specialized chip that is designed for a network fabric, an Ethernet fabric or a level two fabric or an RDMA fabric or something like that. And so this thing is a do everything appliance. And that was what went catastrophically wrong with the original Dream Machine is that it was designed to do everything and it did everything badly. And so this has a ton more CPU horsepower to overcome those kinds of issues. So it can be an NVR for a limited number of cameras, plus firewall, plus intrusion detection, plus switching capacity, plus blah blah blah. The thing is, where we are with internet connections that are faster than a gigabit, you really do need hardware acceleration to help you.

[00:14:08]Otherwise, you get into latency and jitter and those kinds of problems. And so you end up with a crazy situation where things like the stricks like the BE 18,000 can actually give you a better lower latency less jitter network router experience than building like a pfSense box or an OpenWRT box or something like that unless you can actually get the hardware acceleration. And so something like this may offer hardware acceleration, all sorts of fun stuff at the relatively low end. Patrick at Serve the Home is also clued into this, which is probably part of why he spent like a million dollars on, you know, crazy network testing equipment and just some, you know, crazy obscene amount of money because you do need somebody that knows what they're doing and has it all tied together. And what's happening with the Wi-Fi 7 stuff is all the the testing labs and like the testing is being done is radio oriented, not client and user experienceoriented. See also the debacle that is media tech that we talked about last week. So, I'm going to be testing this and I'm going to be testing this under the lens of like you really should look up the the U6 because it's it's it's really like it's it is astonishingly bad. Uh those are from like 2022, so they're they're kind of older. Uh the reality is that you're going to get about 300 megabit, which is about 40 megabytes per second uh between the access points uplink and the rest of the network. That's that's only about 30% of the advertised gigabit speed, which is which is terrible. Ubiquity sort of has acknowledged this and says, "Well, the CPU in it is too slow."

[00:15:33]They'll also give you a discount coupon for new hardware, but there like it's it's weirder than that because like older firmware was fine, but newer firmware that fixes the bug introduces more bugs that make the situation worse, I guess, is what I'm trying to say. That pattern of behavior is not atypical for UniFi hardware, but they are getting better and they are starting to understand things. Like all UniFi really needs to do probably is double or triple their programming team. make sure that their programming team really understands the the layers from top to bottom. And uh also probably make almost all of them uh America or Taiwanese or a combination of Taiwanese and American teams. Like I know there's a temptation there. It's like oh we can outsource this to Eastern Europe. And nothing against Eastern Europe like programming or anything like that. It's just that the teams that are understanding the subtlety and nuance of this need to be really close to the hardware and there's a lot of unemployed Americans right now that are really good with this kind of stuff. This is going to be great. And then you could use AI tools to help you red team and do the testing, but the thinking and the what problem are we trying to solve? That's going to take uh wetwware computation to figure this out. And so that is the lens under which I will be testing the Dream Machine Beast. And that is why I also ask you for your feedback and what's going on and how's it going and blah blah blah. So that's pretty much it for projects this week that I've had a chance to put together because yeah, I mean it's like surprise your inall switch is slower. But you look at it and it's like this is not an amazing piece of hardware in terms of you're putting a switch in the wall plus an access point because the office wasn't wired correctly to begin with. So, you've made a mistake in planning and you're trying to paper over that mistake in planning with a piece of hardware, which was fine from 2022 through 2025, but then in 2026, it's like, oh, there's a there's a catastrophic flaw with these, and we're going to have to depend on the CPU to do the heavy lifting as opposed to the actual product. It's a bit of a problem.

[00:17:41]Anyway, because of all of the lessons learned, I'm expecting this to be awesome. And again, because of that, I don't think Ubiquiti should slow down. They're using Sonic in their 25 gigundred gig aggregation switch in my in my full review, which is great. They need to lean into that. Like go to the crumbs that the billionaires are dropping in open compute and everywhere else and bring them to the small and medium businesses. And yeah, it's going to be a little rough around the edges because you got a lot to figure out, but you can put a crack team together to figure it out and you will be fine and it'll be amazing. And then we'll have a bunch of stuff that doesn't suck at the small and mediumsiz business. And for when it does work, it does work fantastically. I just fixed up Chef Nick with tons of ubiquity equipment. And it was the perfect fit for them because it's easy to manage.

[00:18:28]They aren't going to be too far off the beaten path. They're not going to have a weird setup like a like an oddsiz subnet or anything like that that that would cause them to trip over hardware bugs.

[00:18:37]And generally it'll be fine at least until there's a software update or firmware update that that causes it to get stuck or hung or reset the network configuration as happened with the NVR.

[00:18:46]It's like I would like a static IP address on my NVR. That's fine until there's a firmware update and then all bets are off.

[00:18:53]Thanks for tuning in. This is Linux weekly and Linux projects and you know Linux is basically everywhere. Get your stuff updated and patches and things like that. Thanks for hanging out.

[00:19:05]I'll see you next week. But also, we can just chat in the forum because there's there's fun stuff going on. Woo.