Why Enterprise AI Agents are Currently Broken (And How to Fix It)

Added: 2026-05-20

[00:00:00]So if you are running autonomous AI agents, uh you might be not might be for sure be sitting on massive security and financial time bomb without even realizing it. You know lately the uh developer community has been obsessed with MCP as we all know. It's a universal standard that lets you plug any AI model directly into your internal databases, your local files and your APIs. But while everyone is having fun tinkering with these agents on their laptops, scaling them into the real team introduces two massive uh production challenges.

[00:00:36]The first one being uh quite literally devastating security risks and the second one being the um runaway API bills.

[00:00:50]So today we are looking at why the current uh chaotic approach to running agents is fundamentally broken and how an open source platform called orchestra.ai acts as a secure cost-saving control plane to solve these challenges. I'll leave the links in the description below for you to check it out. So first of all, let's look at the security nightmare. So to to let an agent actually do the work, developers often uh run these systems locally on their machines and they give full system permissions. So the danger of this setup becomes painfully clear uh during like a massive security exploit involving cloudbot for example, right? Uh which is a highly popular uh local assistant uh framework that was later rebranded as maltbook. Very very popular in the news.

[00:01:35]It went viral, you know, because it made running these terminal commands via your AI incredibly easy. It was like viral on Twitter or X, whatever you want to call it. It was all over. But because of uh insecure defaults, hundreds of users actually, you know, they left their control gateways exposed to the public internet. Security researchers, they scanned the web and instantly found these exposed nodes. So because so many developers put cloudbot behind common reverse proxies, the agents internal security was completely bypassed, right?

[00:02:09]So the proxies that forwarded the incoming external requests as if they were coming from the developer's own local machine, granting the attackers full command execution over the host systems. And you know that is quite scary. And to prove how easily this can um this sort of access can be weaponized, orchestra's CEO demonstrated a simple test. So he basically sent a mock email containing a hidden prompt injection to an active agent and within just 5 minutes of reading that email, the agent followed the hidden attacker commands and located the developer's private SSH key and quietly transmitted it back to an external server. So this happens because of what is known as the lethal trifecta. So basically it means that uh it becomes highly dangerous when there's an intersection of three capabilities. So the first one being an agent has read access to private or sensitive information, right? Sensitive data. Uh number two is that it processes untrusted external context. So this can be anything like an email, a web page, a customer support ticket, anything. And third, it has external communication channels like it has an ability to trigger uh trigger the web hooks or send emails. Right? So it has this uh external communication channel established. So, if you have all the three of these active uh without an independent uh gatekeeper, a single prompt injection can completely hijack your agents reasoning loop, turning its access into an open door for hackers to steal credentials or even delete your resources. Now, that's that's quite scary. But even if you manage to keep all of your agents running securely, um running unmanaged loops is an incredibly fast way to blow through your operational budget. You can have like a massive bill. So many developers have uh shared painful stories of waking up to massive API bills. I don't think many people are, you know, uh realizing how impactful it's going to be in the next, you know, few few like in a while. Uh in one famous incident, a developer deployed a customer support agent that got stuck in an infinite retry loop uh with a CRM tool, right? Um and because there was no uh hard circuit breaker, the agent spent 6 hours while the developer was asleep uh repeating the exact same broken action um and it racked up around $4,200 in OpenAI bill. $4,200.

[00:04:41]Person was sleeping, right? So, in the enterprise world, this shadow AI sprawl is causing a quiet crisis with some companies racking up over $150,000 in unmonitored token spend in a single billing cycle with absolutely zero business output to show for it. So, when the CFO is like, you know, what's up, people, the engineers don't have an answer. This is a structural flaw in how agents process data. So most LLM APIs are completely stateless to maintain conversational continuity. Agent frameworks have to append every single tool call error message and uh step to the history rescending the entire cumulative log back to the API provider on um every single turn. So this basically means that your token consumption grows quadratically not linearly. So a 20 uh step loop is not twice as expensive as a 10step run because it is exponentially pricer because you are constantly repaying for the text the AI already processed. So it's exponential. It's not like linear.

[00:05:45]Now from a cost and ROI perspective, Orchestra can reduce your AI API bills by up to 96%. 96. It achieves this through two main features. First, it uses a dynamic model uh optimizer. So instead of sending every simple subtask uh through expensive frontier grade reasoning models, orchestra automatically roots basic operations let's say if you're doing text formatting, extraction or you know classification to cheaper highly effective models. So it reserves the expensive models strictly for highle reasoning and planning and uh you instantly cut uh your baseline uh API spend. Second, it runs automatic token compression on your uh tool inputs and outputs. So, it basically, you know, it strips out the redundant uh token waste before it resends the logs back to the LLM. So, there have been uh studies that show up to 60% of data returned by the tools is uh completely redundant uh context waste. So this compression basically acts as a physical shield u against the uh runaway uh quadratic bill that we uh spoke about earlier. So that's interesting. You can basically also set uh hard uh granual uh token limits per session and per team as well or even uh per agent if you're running you know multiple agents so that you can ensure that a runaway loop uh triggers an automatic circuit breaker rather than uh you waking up to like a massive invoice from your cloud provider. Now from a security standpoint, orchestra neutralizes the lethal trifecta we spoke about using deterministic AI uh tool guardrails. So most platforms rely on uh f fuzzy probabilistic LLM prompts to enforce safety, right? Uh which are incredibly fragile and easily bypassed with a basic jailbreak. Orchestra on the other hand enforces deterministic contextaware rule policies. So let me give you an example. You can create a tool result policy that evaluates the data returned by a web scraping tool.

[00:07:53]And if the scraped content comes from an internal trusted domain, the system classifies the context as safe. But if the agent scrapes a public web page or it uh reads an external email, Orchestra will immediately flag the context as sensitive or untrusted. And once that uh context is marked as uh sensitive, orchestra's uh tool call policy uh uh dynamically blocks any uh highprivileged u outbound tools. So this can be like uh for example uh sending an email or executing web hooks. Okay. So that basically means that even if an incoming document contains a brilliantly written prompt injection designed to steal your data, your secrets, the agent's ability to exfiltrate that data is instantly frozen at the infrastructure layer. So even if it still comes, it doesn't matter. Now for DevOps and platform engineers, Orchestra is built entirely on standard cloudnative architecture.

[00:08:55]They're also part of the CNCF foundation, I believe. and it deploys directly to your own self-managed Kubernetes cluster using an OCI compliant uh Helmchart. So instead of uh credentials being scattered across uh developer laptops, orchestra uses uh bring your own secrets design integrating directly with the Hashikop Walt credentials are held securely and injected uh server side at runtime.

[00:09:19]Meaning the LLM itself never has visibility of your raw API keys or the database connection strings. It also handles identity and access control at the parimeter. So by integrating with enterprise uh identity providers like Octa via SML or OIDC, it dynamically maps your corporate user groups to specific uh agent permissions and also tool access boundaries. And for monitoring and compliance, orchestra exposes application metrics directly to Prometheus and tracing to open telemetry. So it generates uh immutable uh signed audit records for every single interaction capturing precise prompts tool arguments and model completions.

[00:09:57]This basically gives the security teams uh complete forensic visibility and uh makes uh satisfying strict uh regulatory standards like the EU AI act or DORA for example remarkably straightforward which is going to be it's it's it's actually quite important you know when you're especially when you're working with regulated industries. Okay. So getting started is quite easy. Uh make sure you have Docker running. And I'm just going to pull the orchestra docker image and the quick start uh run command. Just running uh it on uh so that I can access it on port 3000. So it's going to spin up like uh you know some docker containers. Um and uh this is the quick start guide. So it should take like uh around a minute uh maybe less than that and you will be able to access it on localhost uh 3000. So you can basically by the way all the links I'll leave in the description below and u I'll show you basically how easy it is to build your first uh first agent. You would also need an API key, OpenAI, Gemini, whatever you want to use. Um that should be that should be fine um as well. So it should come up now.

[00:11:13]Orchestra. Welcome to Orchestra. Uh you can access the front end on port uh 3000. Okay. So this is the first uh web page that you will see. You can just use the default credentials. Uh it's going to ask you to change the password. You can do that later. I've already uh used orchestra before, but it's going to show you like a prompt to add your uh LLM provider. So here you can add your API keys. I have added uh OpenAI but you can add uh Cerebras, Gemini, Grock, Mistril, Olama, VLMs, XAI, perplexity AI, whatever you want to use. Okay, so I've added OpenAI and uh in order to build your first agent extremely easy go to MCPs the Microsoft Playright MCP which is for browser automation for chat is each user gets its own isolated browser session. Um so that's in the catalog and uh you can just click on install. You can click on details and it says successfully installed right. So you can run this command like to check the cubernetes logs. This is the cubernetes yamel file for the deployment. This is the configuration like it's visible across the organization. This is the name. This is the description and so on and so forth. Now what I'm going to do is I'm going to go to agents and I'm going to show you how to create a new agent. I'm going to give this agent u a name. I can say orchestra docs reader agent which is basically going to read the docs and I can give it some uh you know instructions. You can actually put like uh prompts like you are uh playright uh you are using playright to answer questions about the about the project orchestra based on these uh documentations.

[00:12:55]Now for capabilities here I need to make sure I'm using uh Microsoft playright adding here. So these are some of the tools that will be given access to this agent. You can create uh you know you can add sub aents if you want. You can do a bunch of stuff and you can basically install more MCP servers and add more capabilities by adding those tools in here. So the possibilities are endless. Uh I'm going to leave everything else as a default and uh I'm going to say create. Okay. So here I can select the docs reader kra doc readers let's say gemini and I can just ask how can I deploy our castra because it has that context for the documentation it should be able to answer my question there we go so it's saying you can deploy platform either via docker or helm on kubernetes for production option one option two helm for production and then you can tell it like you um deploying where or on-remise kubernetes I can say you know onremise cubernetes and it's going to answer um how I can basically do it so that's you know essentially the easiest way for you to create your agent connected via MCP tools and um get up and running with orchestra but let me also show you how you can test out u the security functionality in terms of like adding orchestra as the security layer but you know there are a lot of use cases. Um I would recommend you to go to the example repository for orchestra. By the way, give them a GitHub star and they have this examples repository for you to you know um try out um using orchestra uh in various use cases. So for example, this one like uh uh fetches this GitHub issue which has a hidden hidden uh prompt injection in it. So when you run this uh docker command before and after orchestra, it's going to basically show you you know what happens and uh it's going to show you an email uh when you run it in your local uh system. Um so running these commands like open open claw, open web UI, pyantic AI, master AI, dummy email, MCP server for example, right? So just simple docker commands uh for you to try out. So a lot of examples for you to test. I've already shown you how to add your uh API key but uh apart from that it's pretty easy to get started.

[00:15:23]You can even ask the u chat directly you know um how to get started. Uh but these are just some of the examples that are in that repository.

[00:15:33]Lastly also like uh if you want to contribute because it's an open source project um they have a slack community uh make sure you join that and follow the code of conduct and they have some bounties and stuff as well. Um you can learn more about stuff that I already mentioned in this video briefly you know um like the uh lethal trifecta for example so some more security concepts like that like AI tool guardrails uh dual LM agent and things like that. So if you want to learn more about these and uh rest of the things we basically just covered and uh I just wanted to show you you know what the project is all about how you can get started and uh how you can do a deep dive into it. So homework for you would be this examples repository and running it on your local system. That would be the next step after watching this video and getting your hands dirty. But uh we did a webinar with the founder as well um and he did a hands-on demo uh for a hackathon that we did with Vake Devs. I will leave that in the description below as well. So you can check out that also and uh that has a pretty good solid uh Q&A and also a walkthrough of the platform itself. So I'm throwing a lot of resources your way. It's for your own good, okay, for you to play around with it and uh test it out. So, moving from local unmanaged experimentation to a centralized gateway is not just about locking down your network, okay? It's about proving the business case of AI.

[00:16:55]So by eliminating shadow AI sprawl, preventing uh devastating prompt injection attacks and slashing the token overhead by up to 96%. Orchestra the open- source project turns AI agents from a risky expensive uh experiment into a high governed uh high ROI enterprise asset. So if you want to transition your team's agents from playground setups to a secure production control plane, you can check out orchestra's open source repositories and the helm charts to get started today.

[00:17:25]All the links resources that I mentioned in this video I'll leave in the description below and uh join their community, ask them questions, uh contribute. They have been uh gaining a lot of popularity on GitHub. Um so get involved in the conversation and if you have any uh questions for me, leave those in the comment section below and I'll see you in the next one. Have a great day.

[00:17:48]>> [music]