MiniCPM-V 4.6: Most Edge-Friendly Vision Model from OpenBMB - Test Locally

Added: 2026-05-12

[00:00:01]We have a sizzling new model this morning.

[00:00:04]Meet MiniCPM Vision 4.6, a 1.3 billion parameter multimodal from OpenBMB that can see, understand, and reason about images and videos. And here is the wild part. It's designed to run directly on your phone or any edge device. It is also suited very well for the drones. We are talking iOS, Android, and even HarmonyOS here.

[00:00:30]But today, we are going to install it locally, and we are going to test it out on images and videos. We have been covering this OpenBMB's MiniCPM family for a long time, and they have been evolving pretty decently. So, let's see what they have done here in this version. This is Fahad Mirza, and I welcome you to the channel.

[00:00:51]Let me quickly start the installation, and we will talk more about the model.

[00:00:55]I'm using Ubuntu. I have one GPU card, Nvidia RTX A6000 with 48 GB of VRAM. If you're looking to rent a GPU on very good price, you can find the link to Mast Compute in video's description with a discount coupon code of 50% for range of GPUs. Let me take you back, and our virtual environment is almost done.

[00:01:18]Let me install all the prerequisites.

[00:01:22]And while that installs, let's talk more about what exactly is the inner holdings of this model.

[00:01:30]If you look at the benchmarks, they are actually quite impressive.

[00:01:33]Because what makes this model interesting is not just what it can do, but what it can do at its size.

[00:01:40]At 1.3 billion parameters, it consistently has beaten GPT-3 5.8 billion across almost every category, which is already impressive, but it also goes toe-to-toe with models twice its size.

[00:01:54]For For on document understanding, OCR, and grounding tasks, it generally shines, pulling scores that rival 2 billion and 3 billion class models. It also handles video understanding quite impressively.

[00:02:11]Not only that, it is bit of you know, I would say struggling model when it comes to reasoning. I'll be honest there. I think especially on the STEM task like MMMU and MMM Pro, where Gemma 4E to B at 2.3 billion really pulls ahead, which makes sense because that's a bigger model with more capacity, but still I think Gemma is a real real fine model. I don't think so that MiniCPM is better than that.

[00:02:39]Anyway, on the efficiency side, this diagram, I think, is the most interesting one.

[00:02:46]MiniCPM Vision 4.6 hits roughly 1.5 times the token throughput of Qwen 3.5.8 billion, which is remarkable given that it's actually the larger model of the two.

[00:02:58]I think the secret sauce here is the mixed 4 times and 16 times visual token compression which they are doing, which lets us trade off between speed and detail depending on our use case.

[00:03:12]Okay, let's go back to our terminal.

[00:03:13]This is going to launch our Jupyter notebook, and then we will download the model and then test it out.

[00:03:19]Meanwhile, please follow me on X if you're looking for AI updates, and consider becoming a member of the channel as that helps a lot.

[00:03:27]Let's first download the model.

[00:03:31]As you can see that it's a very small model.

[00:03:36]The model is now loaded, as you can see.

[00:03:38]Let me quickly show you the VRAM consumption. So, it is consuming just touch over 1 gig. You can easily run it on your CPU if you have good RAM like 32 GB or 60 even 16 GB.

[00:03:53]And for the inference test, first I'm just going to do an OCR of a handwritten letter. This is a code which I'm using.

[00:04:01]Mostly I have kept the hugging face code, just made some cosmetic changes.

[00:04:05]And this is the text which I'm testing.

[00:04:07]This is just a letter to the editor.

[00:04:09]It's a handwritten one. The English font is bit older, so let's see how model performed. It is still working.

[00:04:18]And the model is still working. It It's been like 30 40 seconds now and still processing the image. Let me show you something interesting.

[00:04:26]If you look at the VRAM consumption, it has jumped to 26 over 26 gig of VRAM just for this smallish model. The reason being it is offloading everything to the GPU and yet it is taking long time. Now, this has been bit of a problem with this MiniCPM family for some time now where uh it produces good results. We will see with this one, but they take bit of a time.

[00:04:53]And then the VRAM consumption jump just jumps up.

[00:04:56]And if you are using like a good CPU or a consumer grade GPU, it is going to take even longer because you would have less VRAM to offload everything.

[00:05:08]The model has come back with a response and as is the case with them, the output is of highest quality. If you compare the image with the output, it has done wonderfully well. It has a even missed a comma and full stop. You can see that for example, after this shrink, this full stop is bit like comma, not much, but model was able to differentiate between a full stop and a comma. Spaces are there too. Like for example, between this column and since.

[00:05:38]And every word it has been able to capture from this handwritten letter.

[00:05:44]Okay, next up I'm going to check out its document understanding with this financial statement where you can see that there are a lot of numbers and stable data.

[00:05:53]So, I'm going to ask it that what is the total appropriation excluding special accounts for this 2010 and 11? So, it should just go and extract exactly that value. Let's run this. Again, I'm sure it is going to take a bit of a time.

[00:06:14]So, primarily I have asked it to extract this last value at the very end because it is 2010-11 and then excluding special accounts.

[00:06:24]Okay, let's wait for this model to come back. Takes around 1 to 2 minutes by the way, even more sometimes depending on the complexity of your prompt and image.

[00:06:35]And after taking its sweet time, you can see that the model has come back and given us very correct answer and also has given us some of the reasoning which is totally correct.

[00:06:46]Okay, now let's do some video inference.

[00:06:48]I'll just go here and paste the code. This is a video from my local system and let me show you the video which I'm going to play an AI generated one.

[00:06:59]So, this is a one where some a lion pride is chasing some animals.

[00:07:06]And there are some zebras, some deer and all that.

[00:07:09]Okay, so I'm going to ask the model to describe this video. Simple enough is fine for video, it's a small model.

[00:07:20]And the model has come back with the response and you can see that it has captured the whole atmosphere and identified all the animals including lions, zebras, antelopes. Environment is there, too.

[00:07:32]It is also talking about that the ground is dusty and also the camera panning. So, such a small model but such a complete answer.

[00:07:40]Takes time, but the answer is correct and that is also quite cool for such a smallish model.

[00:07:48]So, I think as usual MiniCPM family is evolving pretty nicely.

[00:07:52]That is a 1.3 billion parameter model that fits anywhere, but I think it needs to improve on its resource consumption plus latency. Otherwise, I think it is a real real good model. Let me know what you think. Again, please consider becoming a member and follow me on X for if you're looking for AI updates.

[00:08:13]Thank you for all the support.