The Non-NVIDIA AI Card Everyone’s Ignoring

[00:00:00]This is a GPU. Well, I'm going to keep calling it a GPU, but I have to come clean right up front. It's not a GPU. It can't even draw a single pixel. There's not one video output anywhere on this thing. It does have these two ports on the back, though. More on that later.

[00:00:14]Anyway, this thing is not from Nvidia.

[00:00:16]There's no CUDA. There's no Olama singleclick installation. So, the question is, what can it actually do?

[00:00:22]And to answer that, I needed to do a head-to-head comparison that I never actually planned on doing. So, here's the thing. For years, if you wanted to run a serious model locally, the answer was always the same. Buy Nvidia. CUDA.

[00:00:34]CUDA owns this space. And if you watch this channel, you know I've built a bunch of those rigs. But the real question I care about, the one that I think you care about, is whether there's anything else that isn't Nvidia that can actually compete for local AI. And I'm going to give you the numbers on that, including the parts that aren't great yet. I don't expect one AI prompt to build an entire project for me. In reality, I'm constantly moving between models depending on the job. GPT for research, Claude for coding, Gemini for massive context, Nano Banana, Midjourney, Flux for images, and then I've got Seed Dance and Cling for video.

[00:01:09]That's why Chat LM by Abacus AI makes sense. It brings one support for the latest GPT, Claw, Gemini, Gro, Deepseek, and more in one place the moment they drop. Pick any model from the interface or let route LLM automatically choose the best model for each prompt. Create professional presentations with graphs and charts and deep research detailed content. Need human sounding copy? Human eyes rewrites text to defeat AI detectors. Need visuals? Pick frontier or open- source models. And when you need more than chat, Abacus AI agent can help build complex apps and websites, connect payments, or run 24/7 agents that keep working through longer tasks.

[00:01:50]The best part is app hosting, back-end database, and author support comes with the subscription. All that starts at just $10 a month, way cheaper than paying for all those subscriptions separately. Check out chatlm.abacus.ai or click the link below. So, what is this thing? This is actually the Tense Torrent wormhole N300. There's two chips on one board, 24 gigs of memory, and it's about 1,400 bucks. Is it worth the moola, the dollars, the gold coins?

[00:02:19]Well, that actually depends on how you use it, and I'll get into that. But first, I wanted to do some LLM tests.

[00:02:26]Now, quick confession. This card came up dead after months of sitting around being powered off without any updates.

[00:02:32]And it actually took a forced firmware reflash. Yeah, the kind that can break your card. Plus, uh, a fight with some version mismatches. And finally, I got it to show up. You know, kind of like when you leave Windows for a few months and then come back and turn it on. Yeah, like that. [music] So, is this thing plugandplay? Well, no, not yet. It's It's getting there. But once it is up, it's actually pretty good. And I got VLM server running Llama 3.18 billion. I know what you're thinking. Hold on. I ran some tests. Now, the very first request came back at 3.6 tokens a second. So, I was ready to cancel this video right then and there and be disappointed. But it turns out that it's just a one-time warm-up thing, not the real speed. So, once I got it warmed up and ran it a few times, 30 to 31 tokens a second, fully local on a card that was just a brick a little while ago. Let that be a lesson to you. Keep trying.

[00:03:26]Well, sometimes. Anyway, 31 tokens per second is faster than I can read. Of course, uh, that argument doesn't hold much water anymore now that we've got agents in the mix. But what does hold water is concurrency. So, I threw eight requests at it at once. And it does 164 tokens a second total. That's five times the throughput. As a little inference server, it punches way above the single stream number, which is pretty cool. And it actually chews through prompts pretty fast. Uh, 3700 tokens per second, up to 4,000 tokens per second. And because you're all developers, let's give it a real job. Write a function to merge overlapping intervals. Looks good to me.

[00:04:05]Now, of course, the quality depends on the model that you're using, but it works. But most of my tests are going to be using Llama Beni. That's this project right here on GitHub, which now I'm a proud contributor to also.

[00:04:18][clears throat] Good job, Alex. Good job.

[00:04:21][music] Now, under the hood, this thing is a completely different animal. A GPU is a giant sea of tiny cores like in Nvidia's CUDA cores. They're all running in lock step. Here on the wormhole, it's a grid of bigger cores. They're not even called CUDA cores. They're 106 cores and they're wired by a network right on the chip. Inside every one of them are five little Risk 5, five risk 5 CPUs. They're CPUs. And yeah, risk 5, the open- source instruction set. But the CPUs are not the ones doing the math. They're like traffic cops. They're streaming data through a dedicated matrix engine with its own chip memory. Data flows through the grid instead of hammering slow memory. Remember that because it's going to matter in a minute. And if you want to program at the metal, that's their open- source SDK called Metallium. Yeah, you can do that though almost nobody needs to because you can just use VLM which wraps everything for you nicely and that's what I used. But everything they have here is documented pretty nicely and open source. And this even explains the uh the tensic scores.

[00:05:23]Here's the TT Metal repo for your perusal. Hey, updated 54 minutes ago.

[00:05:28]They're on top of things. But here's the part that makes it a different bet altogether. Those ports on the back, that's not video out. It's networking.

[00:05:37]That's the same kind of ports uh that are on the back of the DGX Spark. Those are QSFP ports. And switches like this, the ones I use to connect a bunch of DJX Sparks together. See, Nvidia has stripped cardtocard connection out of every consumer GPU. Remember NVLink?

[00:05:55]Well, it died after the 3090. Your 4090, your 5090, they don't have that. Uh they scale over PCIe and as a result, they lose about 30% of performance that way.

[00:06:07]I I haven't actually tested that. I need to get some 3090s in here. I'm sure somebody already tested it. Like this guy, Digital Spaceport. Nvidia saves the real fabric for $30,000 data center chips. Tenstor [music] put it on a $1,400 card. The idea is that you can connect many cards and they'll all work together as one, which is going to give you a way faster connection than going through the PCI bus. The catch? Well, with one card, you don't use it. It's a bet that you'll grow into it eventually once you start using their ecosystem.

[00:06:41]Now, let's be real about what this card is and isn't. Remember my 30 tokens per second. Well, Tentor's own official number for this is about 44. Here's their perf. MD, which does get updated pretty frequently. And there it is. M300 llama 3.18b 44. I even flipped my CPU into full performance mode. Remeasured everything. No difference. So, the only thing that could account for this is Tenstor software, not my machine. I thought they might be like uh on a later version than mine on some kind of special version that's internal or something. And then I noticed that their quantization is a little bit different here. I use a strictly BFP8 quantization. Wow, that's a popular model. And they used a BFP8 MLP, only the 302 decoder layer and then BFP4 MLP elsewhere because they used less bit precision here. That's probably what resulted in it being a little faster.

[00:07:35]However, I'm standing by what I said.

[00:07:36]It's an optimization at the software level at that point. You might also notice other devices listed here like the N150 and the T3K and the TG. Tenstor also has a bunch of other devices that they have. By the way, this video is not sponsored by Tentorant. They didn't even send me this card. I got it from a friend. They have these uh black hole cards. Those are the uh QSFP cables that you would use to connect them. If you've seen my videos connecting the sparks, they kind of look like this chunky boys.

[00:08:04]But here's the wormhole N150. You can see that it's using pretty much the same board, but it only has one chip. And the N300 has both of those chips. As a result, the N150, which is actually not that much less. It's not half the price of the 300, but it only has 12 gigs of memory. So, it can't even run some models of the 300 can. The 150 also has only 228 gigabytes per second memory bandwidth, while the 300 has 576 gigabytes per second, which results in decode speed kind of like this across different models. So 24 gigs is a single card ceiling for the 300. And realistically, you're going to be in the 7 to 9 billion parameter range if you use it with enough context. And it's definitely behind the frontier. The newest models like Gemma 4, Quen 3.5, and 3.6, those newer models are nowhere to be found. They're not ported yet. The newest thing on this list is Quen 3. And I was able to run Quen 38 8B. Here are the numbers. Not the bleeding edge, but somewhat capable models, which brings us to the question that actually matters.

[00:09:06]Is any of this enough to justify buying it over an actual GPU.

[00:09:12][music] Now, before we look into whether it's worth buying one of these over a GPU, there's one thing that actually surprised me. The power. I measured the power while it was running and generating and it still only pulled about 70 watts on average. This was on a car that's rated for 300 watts. I think there's definitely some headroom left here and I pushed it over several users.

[00:09:32]So, it loses on speed, but maybe it wins on efficiency. I mean, this is pretty low power usage and efficiency being tokens per watt. So, I decided to test it against an actual GPU. Same box, same model. I pulled out the 10 storm card and dropped in an Nvidia RTX5080.

[00:09:49]They're in the same price range. See, $999 according to Nvidia. And then based on the OEM, you have MSI at $1,800, ASUS at $1,900. Well, here's one for 1400 from MSI. I ran the same 8-bit quantization FP8 on the GPU to match Ken Torrance BFB8. And the raw speed, um, not close. Nope. 93 tokens a second on the Nvidia versus 10 torren 30. Three times faster. That's decode. What about prefill? That's reading the prompts.

[00:10:22]Yeah, the GPU pulls way ahead. But okay, uh the GPU uses way more power, right?

[00:10:28]So, what about performance per watt?

[00:10:29]This is supposed to be where the tentor fights back.

[00:10:33]Um yeah, no, the GPU wins that too. every single batch size I ran. Sure, it pulls more power, but it cranks out so many more tokens that the tokens per watt is still higher. And I'm being generous to the test store because it's 70 watts is just the chip, not the whole board. If you hook up networking, these uh QSFP ports, they take up a lot. So, we haven't done that. I haven't tested that cuz I only have one board. So, let me just say it straight. For running an 8 billion model today, the Nvidia GPU is faster and it's actually more efficient.

[00:11:08]So, case closed, right? Well, I wasn't done yet because I had one more card on the bench, the AMD R9700.

[00:11:16]Pretty new card, pretty new GPU, 32 gigs of memory, a lot of memory, so it can fit larger models. Dedicated FP8 hardware. On paper, it's more capable than the other two because it can run larger models. And the results came in.

[00:11:29]The decode speed wasn't great. Llama 3.1 8 billion actually was right in the middle between those two. But for Quen 38B, very close to what the 10 Storing card gave us. However, if we take a look at efficiency, this is the worst of all three. Dead last. The GPU pulls nearly 300 W and the output is pretty modest.

[00:11:48]And that is the whole story of this video. The Tenstor, the AMD card, the hardware is not the problem. The AMD chip is seriously capable silicon. So is Tenstor. The RTX 5080 isn't winning because the chip is magical somehow.

[00:12:01]It's winning because it has 15 years of CUDA development behind it. Every one of these challengers is only as good as how far along the software stack is. The real question is not whose chip is faster. The real question is whose software is going to catch up to Nvidia.

[00:12:16]And that's what makes this whole space actually interesting and worth watching.

[00:12:20]So if I were you, if you just want fast oneclick local AI right now today, get a GPU or a Mac. And you can argue with me in the comments about Mac or GPU. They have different pros and cons. I've actually made videos about that. The 10 storing cards are for tinkerers, the folks who want real non- Nvidia box, who want to learn AI silicon hands-on, and who are betting that the software will catch up? Because 2 years from now, this exact comparison could look very different. So, which ecosystem do you think catches up first? Let me know in the comments down below. You might also enjoy this video next. Thanks for watching, and I'll see you next time.

[00:12:54][music] >> [music]