Turning Hermes Agent Skills to Dashboards (Why This is BETTER)

Ajouté : 2026-06-03

[00:00:00]So folks, welcome back to Box Mining AI.

[00:00:03]I want to present some findings. I think you'll find this kind of interesting, Ron.

[00:00:06]>> Yeah.

[00:00:06]>> Um of dashboards versus Hermes.

[00:00:10]>> Yeah.

[00:00:11]>> Right. So we've been using Hermes for some time and we have certain skills and we have Hermes doing quite a lot of work for us.

[00:00:20]>> But there are a few problems I was encountering with Hermes and I had a thought experiment.

[00:00:25]>> Okay. So a lot of times when we run Hermes, it shows us the steps and you're hoping that it works, but sometimes it breaks and it sometimes lies to you >> that it ran something.

[00:00:36]>> Yeah, I definitely have experienced this as well.

[00:00:38]>> Yeah, and I'm pretty sure you guys have experienced this. And oh, it's like absolutely worked. It tells you, yo, the weather is going to be great today. You look outside and it's like, oh, it's raining. You know, like maybe you didn't fetch the weather API properly. You know, let's do it again. Right. So uh I I'll talk about a specific use case that we have and I'll just show you guys the I compare uh the different ways of running it.

[00:01:04]>> So one of the ways that to run it is definitely through Hermes and you can run your agent and collect some data and this is um I'll tell you what this does in a sec. One of the ways that we would do it is a skill. The second way is a dashboard.

[00:01:16]>> So now what is this thing that we're trying to solve? So right now we have the box money AI website. We publish a lot of videos and I wanted to build more content around it which meant that I need to fetch the videos from YouTube, put them into a database, get some transcripts. So I want to convert everything into a transcript. I want to collect all the comments so I can start processing that and do more AI features.

[00:01:39]So this is very much a routine job.

[00:01:42]>> Now normally Hermes is very very good at routine jobs. Okay. So you're trying to move this here. I'm trying to dodge around you too. All right.

[00:01:49]>> Make it slightly smaller.

[00:01:50]>> Yeah. And usually Hermes is great um at routine tasks. Yeah.

[00:01:55]>> Okay. So we wrote up the skill and this is where um I struggled a little bit because our skill was very massive. It was um get the latest videos, filter out which is shorts, then go on YouTube transcripts.com, download the transcripts, use those transcripts and then make a video summary out of it.

[00:02:12]This is a very very long skill. Okay. So the the skill itself is a very very long skill. Uh this went through a lot of iterations because this is something that we had to do and we had to repeat every single day. So we wanted this to run well. Uh but the problem is on Hermes it was kind of like >> hit or miss.

[00:02:31]>> Uh I would say it's like it started off strong. Yeah.

[00:02:33]>> But then maybe like one or two weeks down the line >> we won't get the new videos in. There's no transcripts. I'm like what is in this video? Have a transcript. Make sure you get the transcript next time. Like it there are certain parts that will fail.

[00:02:45]Yeah.

[00:02:45]>> Um and I'm sure you encountered this.

[00:02:47]That's why I'm sharing this information for you guys because I'm sure on any complex task >> um you'll encounter problems and sorting out errors is now becoming the biggest challenge to AI.

[00:02:59]>> The challenge to AI is not designing the skill. The designing skill is so easy.

[00:03:02]I'm just like go patch stuff and then it was so easy to make the skill in the first place as well because um it guided me through the YouTube API process, how to connect everything. The skill literally t took me like 45 minutes to run, but it took me four hours to run.

[00:03:19]Well, yeah. And then uh what's interesting is this dashboard, which uh is it current it duration right now is took me uh 4 days to develop. I'm still developing it.

[00:03:30]>> Okay.

[00:03:32]>> So, basically in this video, we're going to talk about running the same skill, the same pipeline in Hermes chat versus on a dashboard.

[00:03:40]>> Yes. Yes. And we've actually spoken about this where it's uh actually a clever way to think about it in terms of uh probabilistic versus >> deterministic.

[00:03:51]>> Yeah, you got you got this. I got this.

[00:03:53]I thought this would be a good topic for all of us here because like I think like a lot of you guys will encounter tasks that are multi-step that your agents are doing, but they're not doing it well consistently. So anyways, let's take you through how Hermes does it. So the the task is get these videos, put them onto a database, get the number of views, get the comments, >> convert these into actual transcripts.

[00:04:16]Uh, sorry, I I was I wasn't clicking. I was convert them into actual transcripts. Um, and making that work.

[00:04:22]So, uh, it actually runs. Let's say Banner runs this and he has, uh, he tells us what's what's what the videos are fetched and skipped and what it's doing behind on the back end. What it does is it's actually running a bunch of Python scripts.

[00:04:33]>> Yeah.

[00:04:33]>> Okay. So, the the it's actually quite interesting. It's like the AI gets like Python scripts to do its job for it.

[00:04:40]>> Yeah.

[00:04:40]>> Uh but most of the time um >> I would say um it's interesting because it lied to me about this a few times and I had to crack this. Um why it settled on these Python scripts in the end was because I told it to operate these APIs manually, but sometimes the APIs won't connect and it will just make up numbers. It'll just make up the videos for me. They make up the videos of the number of views. I'm like, "Oh, you know, why did you put on the database that got 20 views and it's 4,000?" Oh, sorry. You know, >> uh, I lied there. You caught me. You caught me.

[00:05:14]>> Which was very frustrating because it was uh, YouTube oath API. So, they have that connection. Y >> straight away.

[00:05:23]>> Even that was like, you know, why why is it like that, you know? So I had the brilliant idea of outputting this skill and saying okay look if Hermes doesn't complete this to 100% all the time is it easier to make it on a dashboard. So a dashboard like this is interesting because it's very um >> you you have a lot of things at your disposal. So say for example if you run a pipeline you run it >> uh what I was trying to design this pipel pipeline around was on this kind of ease of viewing what's being completed >> um and this is better in terms in a way it's like it's still bu built by AI I would say this is not you know completely not this is not not AI I'm not fighting AI I'm just saying you can you can use AI to build an entire dashboard to do something >> um and it's interesting yeah >> right because it tells you okay it's it's fetching and it's supposed to tell you you know the progress uh of what it's doing.

[00:06:20]>> Okay. So what what this is is it's not probabilistic. It's deterministic in the sense that um it shows you exactly what it's doing. It shows you the steps of what it's doing. It's not make it's not calling AI at any step right AI wrote this but it's no longer after AI wrote it. Uh it's now um written in Typescript. So now it's executing TypeScript rather than executing AI. M so it's like less text dependent. Yes.

[00:06:45]Which is >> And we're not considered assuming any tokens. It's just like it's just 100% running as a service and it's just kind of running this and that background. You can still set your chrome jobs to this too. Um and you can see more optics of what happened. So like the number of videos it fetched, number of comments that fetched, uh the quota that it used.

[00:07:03]So it's like I optimize this for visibility, >> right? So this is something that you guys can consider too when you're doing something that's you know will be doing be done every day uh is dashboards.

[00:07:16]Yeah, I would say uh I tried this you know after you gave me this advice but even for like simple task it's very I don't know maybe like it's hard to >> this is where it got hard okay this is where it got hard okay so initially I thought this was going to be a onehour operation to convert a skill to a dashboard okay the optimistic side of me was like oh AI is very significantly advanced now yeah >> that we can convert this to dashboard >> yeah because it has the instructions You just need to follow it.

[00:07:47]>> Yeah. I'm like, yo, you you already have the instructions. I say, okay, write your skill as a project requirement. And I sent that into cloud code or I think I forgot this one. No, this one was cloud.

[00:07:57]I used codeex for something else. But this one was cloud code and it was four uh 4.7 and then 4.8 and when 4.8 came out, I was like, thank God, you know, like let's move this uh let's move this speed up a little bit.

[00:08:10]>> Yeah.

[00:08:11]So I realized that um what was very difficult I think this is some of the challenges that you probably encounter was like what authentication was an issue.

[00:08:20]>> Yes.

[00:08:20]>> You're logging into your dashboard you know securing your dashboard up you know you encounter all these programming problems.

[00:08:26]>> Yeah.

[00:08:26]>> Right.

[00:08:27]>> Yeah. What would be the number one tip then to to you know >> what what was the useful part that made you realize oh this is possible? It it was I think I think that's a really good question and I feel like um I was very determined to get this done because I felt like this something that our team needed.

[00:08:43]>> Yeah. Um, and it's nice to show, you know, the comments fetched and what's running and the progress, but the challenge was actually much bigger than I thought it would be because even at the the the get-go, uh, there were problems with like, you know, we're locked in as listing spy. You know, there's a reason why I wanted to do use like proper authentication and that wouldn't work because I was using the superbased database for two particular projects and it's like I was encountering these niche issues that were like very specific to me.

[00:09:13]And I realized that I was spending a lot of time just figuring out authentication, UI, UX. So you you have to consider all these factors. Like I think a lot of times we see apps and like oh they're just great. I just click a button and you're go, right? But when you're designing the app, you're like, "Oh my god.

[00:09:30]Oh my god." Because like uh even for something like this, like say for example, if someone presses the run pipeline thing twice, you have to block them from running it, right? Yeah. So like Okay. See, see, we just did that twice and like uh the let's see if the database uh does that. You you have to think of like what what happens when an idiot presses the button twice. What happens when uh it's both running at the same time, you run out of RAM? What happens if you're blowing up your quota?

[00:09:54]What happens if you do this? So, there's a lot of like >> I feel like because AI is not included during the actual run because when you run it with um something like Hermes, >> uh Hermes is still included. It's in the loop, right? So Hermes can self-improve and if there's like a slight condition and if if you're running it eight times and you click the button eight times like an idiot, Hermes will know and be like, "Yo, stop clicking it so much."

[00:10:19]You know, it know it it has an idea of what's happening. But when you're making a dashboard and you're making your own app, >> right, that app is just a stupid computer. It's going to just constantly do the function. Yeah.

[00:10:29]>> So, um there were constant times when I realized that I was fixing >> just minor issues like, "Oh, what happens when you click two times? Do you have to block them from clicking it?

[00:10:39]Okay. What happens if you >> refresh >> refresh now, right? Like >> there's these things that are um and seem to be very very very um negligible.

[00:10:49]Yeah.

[00:10:50]>> But then they become big issues and even like say for example I had a bug where um it had a old run that was running and it didn't update that the run was finished.

[00:10:59]>> So how do you clear the old run and just delete that entry that blocks you? So I was like I wouldn't run. And I was like, "Oh, I know I'm not running. I cannot run the pipeline because something I started 3 days ago was blocking it and saying, "Hey, look, it's still running."

[00:11:11]I'm like, "It's clearly not running.

[00:11:12]It's being 3 days already."

[00:11:13]>> Yeah.

[00:11:13]>> So, there are all these um very programming specific considerations that came in >> um after um converting the skill into a dashboard. M >> so I think it's about the I think it's like I think it's still I feel like like longterm I still prefer this where I have a very good idea of what's being fetched and what's happening but that being said maintaining something like this is a big pain >> so the takeaway would be it's like hard to oneshot a dashboard like >> it's impossible it's impossible like like the fact is like you um even log in will be considered a full feature and that might traditionally take a week of programmer's time to get you know your proper login in your off your role level security how you're storing your data um all of these things technically speaking if you're using a human programmer will take ages so you have to um you have to plan for that like >> I initially thought oh AI is so smart um these these c these these app or scope issues are not a problem because if you're making an app it's still you know it's still substantial deal right and the UX flow and whatnot so I started realizing saying um like something like this long term is worth developing because especially when you see this um I actually asked this UI to be cleaned multiple times but like now that we can navigate our videos we can probably see uh the transcript convert those into videos or maybe convert uh videos that are like similar >> in content and then maybe like click one two three and then convert that into an article like I I feel like once I bring the content into this database I feel it's very powerful and having a dash dashboard means I have a very good view of what I'm trying to do.

[00:12:56]>> But that being said, making this dashboard, it it is a 3 4 day endeavor.

[00:13:01]>> Uh I'd say like uh I recognize that now like anything to do with data, having the dashboard is sort >> to keep track of everything.

[00:13:11]>> Yes. So you say for example your world monitor idea, you you cannot trust your AI to be your world monitor because AI's potentially lying to you all the time.

[00:13:20]You don't see all the data. You can't mouse over anything. Yeah. So, I think like a dashboard I would still recommend people build a dashboard for something that they do all the time.

[00:13:29]>> Yeah.

[00:13:30]>> Like that that's repeated that they need to see, but they really need to to have a good I guess it's like a good justification for why they want to spend so long on it >> because you will be spending a lot of time just debugging your stuff. You be like, yo, you know, there's a bunch of function calls. You know, there it's it's it's talking to YouTube all the time, but then YouTube's not, you know, the API key is not working yet. Do you want to make a smoke test? Do you want to do all this? It's like >> it's nice to see it working, but then when it's not working, you're like, "Oh, it's it's horrible." So, this this dashboard is not finished yet, but I'm just showing you guys how it is. And um it's funny that a pipeline that took me 40 minutes to make would take me four days to make a dashboard like that. I think that was my biggest finding here.

[00:14:13]>> I actually want to get like started on this again.

[00:14:16]>> Yeah.

[00:14:16]>> Uh >> we'll take you through it.

[00:14:18]>> Okay.

[00:14:19]>> Yeah. Yeah. I'll take you through it.

[00:14:20]>> Okay. So if you guys are interested more content and just doing dashboards and creating content like this, tell us. I think personalized dashboards are really powerful because they are deterministic, not probabilistic. Like this thing functions like a machine. It is a machine. Yeah. Okay. It functions like a machine. It breaks like a machine. You have to build it like a machine. Um but it's interesting and there were a lot of builds on this pro process. Like there was a lot of mistakes I made along the Okay. So, I hope you share with that guys and maybe Ron can ask some questions and help you guys build your own dashboards to improve your life cuz at the end of the day, you know, this is very reliable and dependable once it gets working.

[00:14:56]>> All right, with that guys, thank you guys so much for watching this video.

[00:14:58]See you guys in the next one. Peace.

[00:15:00]John and I.