Ukrainian drones learn from 4 years of war | Sitrep podcast

Added: 2026-05-12

[00:00:00]Has Ukraine made a breakthrough in how it uses drones? And could Britain's armed forces benefit? Ukraine's former intelligence chief says it's now less about numbers of drones and more about their intelligence. Simon's been investigating evidence suggesting machine learning now allows some Ukrainian drones to not only identify targets, but also lock onto their most vulnerable spots, and that is improving lethality. He'll explain how UK forces could use the unprecedented pool of battlefield data behind the advance. And we ask whether we're now on an unstoppable path to machines making life or death decisions in combat.

[00:00:40]>> The level of human involvement needed is now reducing.

[00:00:43]>> Once one side has done it, it becomes acceptable for the others to do it as well.

[00:00:48]>> Defense explained.

[00:00:50]>> The drones are actually piloting themselves to the target area. They're being taught how to, for instance, not hit a T72 from the front. They'll hit it from a weak spot at the back.

[00:00:59]>> This is sitrep with Kate Shabbo and Simon Newton.

[00:01:05]In 2022, the year of Russia's allout invasion. Ukraine produced a total of around 5,000 drones. Now, it's using almost twice that number every day. It's something like an 80,000% increase. Simon, that figure is stunning.

[00:01:22]Yeah, it's it's stunning and and I guess frightening in some respects. It's the it's not just the scale that we're seeing. It's the speed of innovation that we're seeing in Ukraine, which has been a character throughout this this war and this intersection that we're seeing now increasingly of artificial intelligence and warfare and this explosion in the whole drone ecosystem within Ukraine. I mean, I went to Kev in just 2024 and was talking to V, who were one of the biggest drone manufacturers there at the time, and they were talking to me about this new idea of of using machine uh learning to get their drones to identify Russian tanks and Russian vehicles, etc. Well, that now seems to have come to pass. They're able to do that. And, you know, there were seven drone companies in Ukraine back at the start of the full-scale invasion 2022.

[00:02:10]There's now 500 of them. There are something like 6,000 AI professionals working in those industries inside Ukraine and they have a new um defense artificial uh intelligence center in the Ukrainian MOD led by just a 35year-old guy. Um and that department is actually partly funded by the UK. So there is this massive excuse the pun explosion in this whole ecosystem uh within Ukraine.

[00:02:34]>> Yeah, that speed of innovation is absolutely staggering. uh Russia's drone production and use has soared in parallel and inevitably that means both sides are trying to get the battle edge.

[00:02:44]Simon, you've been studying footage which suggests Ukraine has developed that edge though through software innovation alongside hardware. Can you just explain what that footage shows and what it means? So, we've seen, you know, we've seen these drone videos throughout this war going going right back to just simple quadcopters dropping grenades into trenches. And then we went through the sort of fiber optic period with this sort of crystal clear video feeds. And now we're progressing to seeing almost the inner workings of these AI drones because we're seeing some of them, the videos of them with with the various lock boxes being placed onto different targets on the ground and not just one target, numbers of targets simultaneously. And that's indicating essentially that these drones are starting to identify Russian vehicles more clearly. There's even talk of them being able to identify humans which would allow them to identify Russian soldiers. So they they are starting to think more about what uh what they're doing. Then this is the AI coming coming into the mix if you like. So, we've not really seen this yet in practice, but there is speculation that they are able to now look at certain weak spots on Russian vehicles that they're being by by by machine learning being taught how to, for instance, not hit a T72 from the front. They'll hit it from from a weak spot at the back. Um, they'll look for open hatches for vulnerable engine compartments, that sort of thing. So, that machine learning is gradually feeding into these AI models. and and you know we we were you used to seeing the the Javelin missile doing that sort of top attack trajectory to hit the weak spot of tanks. Well, that's essentially what these drones are being trained to do.

[00:04:18]>> And how has Ukraine been able to do this?

[00:04:21]>> Well, it's compressed, you know, what would normally be decades of military innovation into a very short space of time, just a few years through necessity obviously. And it's a wartime ecosystem.

[00:04:32]You had hundreds and hundreds of drone companies. And again, I remember going there just a few years ago and you were going to sort of old buildings with guys working away on drones with soldering irons and things like that. That's now been turned into a proper industry within within Ukraine. Um, and you've got, you know, university labs, engineers all working on this on this type of tech. And, you know, if they if they put something into the field on a on a Monday and it fails, they can have it fixed and, you know, innovated by Friday. That's how quick things are actually happening. That's that's one of the reasons. The other one is the access to commercial technology um and also their own indigenous production of components. So they're not relying on China as they were previously. Um they are managing to make a lot of these cameras and the and and the uh other bits that they need themselves. So that is allowing this innovation to happen really really rapidly.

[00:05:24]>> And I guess Simon, it goes without saying that all the footage that they've gathered over the past four years has been essential.

[00:05:29]>> Yes. I mean that's that's the footage they have which amounts to I think I read about 2 million hours worth I think it's 228 years worth of footage they have gathered from these drones. Um and it's being effectively just aggregated into a system. It's actually a nonprofit I think that is actually doing it.

[00:05:50]They're pulling it in from 15,000 different drone teams and that footage is going into a massive cloud and that is effectively feeding the AI's machine learning to try and teach these drones what to look for. So, it's it's a huge bank of information, a wealth of information that they can feed these these drones with.

[00:06:10]>> Well, let's h bring in Dr. Matthew Powell. He teaches strategic and air power studies at the RAF College Crownwell. Matthew, hello. Welcome back to Sitrep. um what Simon is talking about there, is it is it a small step or a giant leap for battlefield drone capability?

[00:06:25]>> Thank you for having me back on. Um I I think that's that's the key question to to be asking and to be thinking about. I think what we're seeing in Ukraine is yes, as Simon has said, we're seeing this huge innovation in terms of the ability of Ukraine to be able to increase the lethality on the battlefield, but it is still at a relatively small scale. It's still yet to be upscaled to uh a wider capability that will provide a platform for almost complete dominance of the air and possibly to really reduce the effectiveness of uh Russian capabilities on the ground. It's certainly an interesting innovation and it it just speaks I think to the the wider history of technological innovation that we see in the history of warfare whether that be the development of something like the tank in the first world war the use of air power that went through the first world war into the second world war through to ballistic missiles in the cold war so it it's that ever um leaprogging of technology and I I've no doubt that the Russians are aware of what is happening and are already trying to create counters to what is happening Again, it's what we see in the history of warfare. We saw 15th of September 1916, the first use of the tank on the SOM. By mid 1917, the Germans have already got an anti-tank gun. So, I'm sure the Russians will be trying to discover ways to try and make this uh less effective and to try and reduce the ability for machine learning. But we're also seeing significant challenges that are emerging here. And one of the biggest ones is in the bandwidth that's available. And I think that's one of the things that's really constraining the Ukrainian use of AI at the moment. So for example, a single HD drone video feed at around 25 frames per second is consuming around 10 megabytes per second.

[00:08:12]>> Mhm.

[00:08:13]>> That's a significant amount of bandwidth when it's relatively limited.

[00:08:19]And because the Ukrainians are using Starlink so much, it's created a single point of failure.

[00:08:24]>> Okay. So if that link goes down, if that link is attacked, all of a sudden all of this disappears and >> extremely vulnerable. Then >> it can be extremely vulnerable and it then means that you are then reliant on huge amounts of power generation to provide the server capacity for these uh AI enabled drones to become effective.

[00:08:45]And so what the Russians have started to do now is to attack strategic targets around power generation in order to try and reduce this capability. So we're not actually countering the technology directly but reducing your ability to utilize it.

[00:09:00]>> And Matthew, what about that question people always have with any use of AI?

[00:09:04]Uh is it can you rely on its judgment?

[00:09:07]But but I guess you won't really know until you try either.

[00:09:10]>> You won't really know until you try.

[00:09:12]What we see particularly from uh the Israeli use of AI enabled drones is that there are much greater cases of collateral damage and that the AI learning that's been enabled and that it is happening. It's very difficult for the machines to distinguish between a legitimate target and a non-legitimate target. And so we're starting to see real ethical questions being raised around the use of AI drones. How far is it able to understand international humanitarian law? How far is it able to apply international humanitarian law?

[00:09:46]And what we're seeing at the moment is that whilst the AI is very good at identifying targets and attacking them, it often does it with greater force than a human would. And so it's that wider collateral damage. And it's those wider ethical and moral considerations that we really need to consider. And perhaps that's not as great a consideration with the way Ukraine are using them in an active battlefield in an active battle space. But certainly if we wanted to use this technology in more complex conflicts like counterinsurgencies like we saw in the early 2000s, this is a real consideration will have to be thought out very very carefully by those involved in the planning, the utilization and finally the application of this new technology.

[00:10:32]>> Yeah. And Simon, is there anything that gives us a confident indication of how widely these more intelligent drones are being used and to what effect in Ukraine? As Matthew said, I think they shouldn't overstate their use, but there's certainly I mean the sheer number of Russian deaths that we're seeing at the moment and and many of them at the hands of drones. You I think it was 35 and a half thousand Russians claimed by the Ukrainians in April killed or wounded. Um you know, more than they can replace. And that does speak increasingly to the use of of these lethal drones. You know, are the pilots getting significantly better in that short space of time? You know, it's probably partly down to just sheer mass, but there's also possibly AI in the mix in these on these missions for this sort of last mile kill chain aspect that we're that we're talking about. And one of the I guess kind of interesting thing I saw is, you know, the reorganization of the of the Ukrainian army, which is taking place. They're talking about, you know, they're going to have a different core structure and and more pay for the soldiers. And one of those things also is they're going to rotate troops more frequently out from the front line. So that actually also speaks to the effectiveness of the drones because the saturation there is so great. The relentless pressure of the static trench warfare that we that we did see has now you know largely gone effectively and so the drones are taking over the surveillance the strikes uh the sort of attritional roles if you like that that actually used to need troops to be present on the ground. So you know that possibly is another indicator of of how prevalent it is. Matthew, you mentioned earlier how Russia is trying to keep up by interdicting um Starlink, trying to stop Ukraine from using these drones.

[00:12:03]It's also innovating. Do we know how far behind or close Moscow is to what Ukraine seems to be doing with intelligent drones?

[00:12:10]>> It's very difficult to have any idea what how um far the Russians are in terms of developing AI enabled drones because we just don't get the information from reliable sources as to how far they are. My best guess would be that they are some way to developing this. They would this is something that they certainly would have been looking at because it's a it's a cheaper it's more effective. It reduces your casualty rate. What we're not seeing, I think, is is the Russians utilize this. Now, this may be because they haven't been able to adapt that technology and being able to develop it as much as the Ukraine has.

[00:12:44]Perhaps Ukraine have had the the necessity to do it because of their smaller population, their smaller armed forces. So they've been forced to innovate to reduce their casualty numbers. But I think with the way that the Russians are operating in the air and on the ground, it's highly unlikely that they are that advanced in what they are doing and are perhaps resorting to more traditional Russian methods of utilizing mass and huge numbers in order to become more effective rather than relying on technology to reduce the burden on manpower.

[00:13:15]Simon, you talked a little earlier about the 200 years worth or more actually of battlefield footage that has been turned into the AI models for these intelligent drones. Arguably, this is the real game changer, the special source, and they're offering that up to us.

[00:13:29]>> Yeah. So, the the the the nonprofit I talked to earlier on say they are in discussions with uh Ukraine's foreign allies. They won't reveal who they actually are, but you would expect the UK would probably be in the mix there because, you know, we have a close relationship on drones with with Ukraine. We we co-produced the Octopus Interceptor for instance. So yeah, you would expect they are I mean it's it's something like as I said it's it's it's footage from 15,000 drone crews across Ukraine that's being put into this into this feed. It's something like 6 terabytes a day going back to Matthew talking about the data. That's how much they're kind of pulling in. So you know this 2 million hours is is food for AI effectively. It feeds this machine learning that they can use. And and just to just to return quickly to the question you asked before, Matthew, I think the the the Russians have used AI on the Lancet for instance. I think at the beginning of the war, they used it.

[00:14:17]I if you remember the Lancet was the was the killer drone that hit the uh the Challenger 2 tank in in Kursk when that incursion happened. So they they have made use of it. They struggle because obviously the components they need are sanctioned. So they're getting those through, you know, the Nvidia chips and the uh systems. they are getting hold of western components to put in these things but through sort of third countries that's how they're getting so that that is one limitation on their ability to to scale this tech whereas the Ukrainians are going very much to try and build their own indigenous source of all of all of these components >> and Matthew that that wealth of footage that we're talking about can it be translated or adapted for us because each conflict is different as you said and we're talking about rapidly changing technology as well could it be obsolete before we're even in a position to use I I think as a at the strategic level it probably won't be obsolete because it will give you an idea as to the capabilities, the equipment, the weapon systems that the potential opponents are utilizing.

[00:15:19]But at the tactical level, it may well become very obsolete very very quickly as we see adaptations perhaps the use of um new methods to try and camouflage.

[00:15:29]That's one thing that we're seeing with um the AI enabled drones at the moment is that it's able to detect camouflaged uh enemy positions, camouflaged enemy equipment. So perhaps we will see a situation similar to what happened in Kosovo where we will have the production of dummy weapon systems, dummy positions that will fool the AI without it being able to understand what it is doing. we might see those innovations start to occur. And so we might have resort to um more traditional methods in modern conflict to try and confuse AI to try and make it make mistakes to try and make it expose itself. Perhaps there will be a way in the future of engaging these drones more effectively than what we see currently, which is either through trying to shoot it down with uh small arms fire or with fast jet air assets.

[00:16:18]You may be able to engage it. So you may utilize camouflage and uh espionage and deception in order to try and allow the AI drones to reveal themselves so that you can engage with them. So I think at the strategic level the understanding of where weaknesses are on enemy uh equipment. I think that will be that will be remain constant. That will be something that will be really helpful and interesting to develop and to understand. But at the tactical level, how you actually find and engage these targets, I think that's where the real challenge and where the real innovation and the leaprogging is going to happen more and more.

[00:16:53]>> And Simon, this isn't an end point, is it? Can we see what is coming next?

[00:16:58]>> Uh, yeah, it is very, very far from an end point. Absolutely. I think both the Ukrainians and the Russians are pushing ahead with AI. The Ukrainians in particular see it as a as a real counter to the to the sheer mass that Matthew was talking about that Russia that Russia can bring to bear. I've been reading about there's there's companies in Ukraine experimenting with facial recognition AI. So you could even use you could use them to take out individual people if you wanted to. And there's also the idea of taking the the chatbot idea that we use for our banking for instance where we talked about you could have that connected to a drone where you would basically watch the drone flying or and you tell it what to do. You tell it I want to go and have a look through that window. Go and have a look in that building. can go and you know it's all of this stuff is is is in the pipeline and and I've been as you said I've been talking to um to David Hamling who's a a renowned drone expert and he's written extensively about Ukraine and I was asking him about where this is all going >> the level of human involvement needed is now reducing um so uh sign in Ukraine has a system called which means beehive and the idea for this is that it basically gives one operator the ability to control a bunch of drones. So the way that works is that the the drones are all launched and they loiter in an area and the operator then only controls them one at a time. So he finds a set of targets, he locks a drone onto the first one, presses a button and that then goes off and he can then take over the second drone, lock that one onto a target and so on. So they are the the drones are actually piloting themselves to the target area when you just click on a box and tell them where to go and they're then waiting there so they can be controlled one by one. That's the level of autonomy we're at now. That hasn't been rolled out very large scale yet, but that is certainly what we're looking at going forward. There will be even more intelligence on the drones and even less human involvement. So the drones will do a lot more of the autonomous intelligence gathering. So rather than having human beings looking over screens looking to see where the enemy are, the AI will be doing all that and saying, "I think I found what looks like a Russian tank here. Please confirm and show them pictures." So they will be able to do all that. And the humans will basically just be the commander in charge saying what resources need to go where and actually approving strikes. So we're we're definitely getting into the stage where that the humans are the mission commanders rather than the actual drone pilots. So in terms of the the time scale, they've gone from where they were even just last summer to this now. Where where is the next moments? Do you think the two things are aligning at the moment? I'm guessing that the potential is coming together with the reality for the first time. I think it's been doing that steadily for a while. The Ukrainians are certainly they have recently started mass production of something that they won't describe, but which sounds like it may very well be an AI targeting system. The Russians call them Martian drones because they supposedly they're using technology that was developed by NASA for Mars, uh, which is a nice myth. Uh, and but as far as the Russians are concerned, it's all American technology is is what they're up against. But these are highly capable, jam-proof, largely autonomous drones um that have a very high hit rate. And those apparently have just been trialled in the last few months at small scale and now now being rolled out at large scale. Uh and it's interesting from the numbers that Ukraine's unmanned system forces are putting out, they are killing a lot more Russians now than they did a couple of months ago. uh and that may carry on going steeply upwards as they get larger numbers of more capable drones. So I think certainly over the next six months um yep look to see things becoming ever more effective >> and the delta system that the Ukrainians have as well.

[00:20:46]>> This is the Skynet to the uh the terminators in the air. Uh Delta is absolutely vital because it's uh a situational awareness system. It's a battlefield planning system. Uh it draws together all the data from lots of different sources um including drones, satellite and lots of other sensors. So it shows everyone where everything is on the map and that allows Ukrainian commanders to target things at a distance. Without that, everything else is a bit useless because you you have no chance of actually finding the enemy and targeting them. With Delta, everything changes. So that is an absolutely vital element to making this all work. and how tightly the drones can be integrated into Delta will be critical in determining how successful they are going forward. But the Ukrainians seem to have taken that and run with it. So, uh I think they're fairly confident that will work. Whether the rest of the world will catch up and whether we can build something that effective is something that very much remains to be seen.

[00:21:49]>> Just one quick question that occurs to me. How uh deconliction on the battlefield when you've got AI? How does it know it's a Russian T72 rather than a Ukrainian? How does that how how does AI know that?

[00:22:00]>> This is where Delta comes into play because Delta tells you where all your friendly forces are and where all your known enemy forces are. And if it spots a tank, it can figure out a one is behind the Russian lines over there rather than over here. Yes, it's a challenge. It gets even more challenging with foot soldiers uh and trying to figure out who is who because it it's not easy to tell them apart. But that kind of thing. Good machine vision systems are better than humans uh which we're seeing in in a lot of areas. So things like um just looking at um uh radiographs for scanning for cancer. The good machine vision systems are now more capable than human radiographers at picking out tumors and identifying them. Uh and the same applies on the battlefield. A a well-trained system should be able to outperform people. The problem is there are still sometimes some situations where the machine gets it horribly wrong but that will reduce whereas humans will always make stupid mistakes. When you interview the likes of Alc for instance they always you ask them about the drone situation and they always say we we can't go and buy what's there now because it'll be outdated in wherever there's this kind of waiting idea you're constantly fed.

[00:23:14]Will will that work? How what what is the way around? I would say that is I would say that is a massive half proof.

[00:23:21]Um because it's interesting if you look at the most popular reconnaissance drone that's being bought by both the Ukrainians and the Russians at the moment, it's the DJI Mavic 3, which is a drone that which is a few years old. Uh so and that is by no means out of date at all. What that both sides are doing is massively updating the firmware when they get it. Um and so it can cope with current conditions. And I think that's going to be the message for future drones. the hardware doesn't necessarily matter that much. So long as you're able to update the software, particularly with regard to things like AI and if you make it modular so that you can plug in new hardware as you need, there is no reason why we shouldn't be acquiring lots. More to the point, we should certainly be acquiring the capability to produce these things at scale. Um that's what Ukraine has been developing over the last few years, this uh sovereign drone production system. So if you're actually in a war situation, you don't want China to be able to turn the tapple.

[00:24:20]>> So the situation we are own is Ukraine is at the forefront of this and pushing forward. The the the Russians are playing to a degree catchup. But we were always in this cycle, aren't we?

[00:24:29]>> The Russians are very much playing catch up catchup, but the Russians are in the game. Uh and they are definitely developing systems at pace. They they are hampered by having a useless military procurement system uh and a a lack of imagination, problems with bureaucracy and corruption, but they are advancing quite fast. And I don't know if you've seen the Rubicon drone unit.

[00:24:54]They are quite terrifyingly efficient in their ability to adopt and efficiently use new technology. And the Russians certainly have some I think you mentioned one of the Russian AI enabled drones. There are a few others out there. uh they are not that great yet but they are certainly learning and they will certainly be taking apart everything they get from Ukraine and learning fast from that. So I doubt if they are more than a year or two behind the Ukrainians.

[00:25:21]>> David Hamling there, military technology writer and author of the book Swarm Troopers. Uh Matthew, would you add anything or take a different view from David at all on what comes next in terms of more autonomous battlefield drones?

[00:25:35]No, I I think what was said is largely accurate in terms of the development and the the position of both the Russians and the Ukrainians. I I think it's a fair point to make that the delays that be uh coming from the UK government around we need to wait until the technology is matured and we need to have things that will be available uh that will be capable of operating in uh future conflict. I I think it's something that we see throughout defense procurement in the UK and throughout the West in terms of there's always a delay almost to wait for the next technology to emerge that will then make you more capable but that's going to be 5 10 years down the line and then you end up just waiting and waiting and you never actually make that decision. So I think that that's one of the key areas for the for the UK to be looking at is to build a mass of industry within the UK that's able to produce perhaps at a relatively small scale at the moment but where you can build in the strategic reserve capacity that if you need to increase I think is uh is one of the keys because it's going to be something that major nations are going to use in war and that if you start behind the curve in a major conflict on with this uh new technology technology in terms of AI and utilizing it for intelligence and for target acquisition, it's going to be very difficult to catch up. And I think that's something that was said there about what how the struggles that Russia are facing. Now, the UK system is nowhere near as corrupt and inefficient as what we see in Russia, but there are still challenges to be overcome. there are still problems that need to be um thought through um prior to the next potential major conflict.

[00:27:20]>> Yeah, I was just going to interject because I I didn't mean to offend Al Kahn's my question about procurement. I hope he'll still talk to me, but um it's I was just going to say I I've so I've been to two drone factories opening in the space of the last 12 months or so.

[00:27:34]Um you know, and one was the UKCec systems site up in East Anglia. So and and that so so there is an there is an effort to bring that tech to the UK and to to work with Ukrainians on this stuff. So that's you know in the government's the defense if you like but but you're absolutely right I think you know it is this question is when you get on when do you get on board this this moving train and and and actually go go for it if you like. Um that's the issue that the the procurement process struggles with most of the time and and historically.

[00:28:05]>> Yes. I I'm sure Alans will still talk to you. Don't don't worry about that. I'm guessing that the Ukrainian forces and the government are not giving you all the details though, Simon, or in fact confirming anything. Is it certain that they do still have a human in the kill chain for drone attacks? Did you get any sense of that?

[00:28:21]>> Yeah, I think they don't necessarily talk about that, but I think there's definitely is a a human in the kill chain in Ukraine. I mean, this this has been done. I think I was reading in 2016 in Libya they've been government used a Turkish drone a fully autonomous Turkish drone to launch a mission where it decided what it was going to hit and killed people on the ground and that generated a lot of ethical debates at the time. So this this tech it's out there and there is evidence that Ukraine can do all this stuff. They can build AI drones that take off on their own that loiter on their own that find targets and strike on their own. What they haven't done is sort of put all those sequential steps together into one long kill chain yet. You know, whether that's just purely a technical bandwidth reason, whatever, or just or an ethical decision, I I don't know, but I'm sure it it can be done.

[00:29:09]>> So, Matthew, it sounds like we're either already at the point though where drones could be sent to kill completely autonomously, or we soon will be, if not actually there now.

[00:29:18]I think the the technology in theory exists and that it it's possible to use a fully autonomous drone to scan the battle space sc and to find and engage targets without any human in the kill chain. But I think as Simon has said it's perhaps a technical issue. It's perhaps a human trust issue as well that it's very difficult and we see this in or we have seen this in Ukraine I should say that where AI has become more prominent drone operators are probably the least willing to use it and put up the most resistance because there there is a a kudos there is a a prestige that's attached to being a drone operator and so they're they're unhappy and they're uncomfortable with this technology musling in in on their territory. And so we we do see some internal resistance from that. But I think that the sort of the ethical debate is the key one that as was said in the interview and as we've talked about on this podcast that we have a position where AI can get things right a lot of the time, >> but it can also make significant mistakes.

[00:30:28]And that significant mistake might be the attacking of a target that looked like a legitimate target and turned out to be a house, a school, a hospital.

[00:30:40]It could be that the AI gets confused.

[00:30:42]It creates a hallucination and it starts attacking your own forces. And at what stage at what what have you put in place then to prevent either prevent that from happening or to stop the AI in its tracks when it is doing it? And so the theory is is there it works. The technology has developed far enough.

[00:31:04]But I think there will always be that concern in the back of commander minds as to AI isn't perfect. And we talk about humans making mistakes.

[00:31:13]AI can make mistakes and they can have much more catastrophic consequences because you can at least communicate with a human. was if you send an AI enabled drone based on machine learning if it creates a hallucination it can attack almost anything and you how do you prevent it how do you stop it I think that's one of the things that will prevent the use or the the total use of absolutely autonomous drones and autonomous weapon systems as we've seen the Ukrainians claim that they've used AI robots to take positions on the ground it's all about the information that is being fed and what if that information changes during the course of an operation. How do you then get update the AI or machine learning? How quickly can you get it to adapt to the new situations it finds? What if your opponent brings in new equipment that AI hasn't seen before? It might completely ignore it and that might cause a strategic or operational catastrophe.

[00:32:10]>> And Matthew, all of this poses a big dilemma for the UK and like-minded nations. We have a very clearly stated principle that meaningful human control must be part of any weapon system. But if our enemies don't have the same concerns and are prepared to attack us with entirely autonomous weapons, is that a principle we'll have will we'll be able to afford to keep?

[00:32:31]>> It's an excellent question and one that I I certainly don't have an answer for at the moment. I think with the way that the UK, with NATO, with the wider western militaries, they place an emphasis on conducting war according to the principles of IHL as much as they possibly can. And so when you're fighting a conflict, you're not just fighting militarily. You are fighting politically, but you're also fighting on the media landscape. And if you can demonstrate that you are fighting this conflict within the realms of IHL and your opponent are, you may be able to garner greater support and greater diplomatic support and you may be able to win the media war that comes along with 20th uh 21st century warfare.

[00:33:14]But how far will these principles hold?

[00:33:16]It's a difficult thing to say. We if we look back through history and we use that as examples, first world war again both the western allies, France and the UK and Germany were sitting on huge stock piles of poison gas and we're almost waiting for the other side to use it so that it would legitimize its use.

[00:33:35]Once one side has done it, it becomes acceptable for the others to do it as well. So, will we be in that position in the future that we have just we're just waiting for our opponent to use completely autonomous systems and then we say, "Okay, the gloves are off here. We can now use this as well." I'm not sure the UK will use will go down that route. I think that they will still want to try and keep a human somewhere in the loop. And I think that most militaries will. I think it's it's vital that they do that just to just as a sanity check if nothing else. Um but if it does happen then who knows we we could easily see wars being fought almost by completely autonomous weapon systems which are simply directed by commanders and warfare changes almost overnight.

[00:34:28]Well, let's um let's get a final thought from each of you after all of that because the chief of the air staff has told the I newspaper he's aiming for collaborative combat aircraft or robot fighter jets to be coming into service with the RAF by 2030. Matthew, how much will the way those and other uncrrewed systems are used have changed by then?

[00:34:49]Again, I think that's an impossible question to answer because I don't think we could have seen how AI and how the use of drones in Ukraine has changed the the the character of war that we see in Eastern Europe. I'm one one of these uh as a historian especially, I'm very skeptical about the capabilities of new technologies. I think that there is always an extrapolation to the nth degree with a lot of ifs, ands, and buts. And I I think that what we will see is that perhaps these will we will see drone swarms being launched from things like Tempest, but they will still have a degree of human control over what's happening. So perhaps we will just see pilot become more of a a battle space manager rather than somebody who fires and engages with targets directly and utilizes drone swarms. Will we see the widespread use of AI robots as we've seen uh on a small scale in Ukraine?

[00:35:43]Again, it's entirely possible. The technology, the development, how you utilize them. I think these are all things that need to conceptually be thought through by commanders and senior leaders within the Ministry of Defense to get a full understanding. And I I often get the impression that the Ministry of Defense likes this new technology but doesn't fully understand how to use it and how to develop it and is often unwilling to engage in what is required to find the widespread opinions that are required to fully uh grasp what this can do. So I think there will be a part um but I I I think sort of this idea that war will be fought by completely autonomous systems. I think there will be still humans. I think you still need humans to engage. I think you still need humans to hold the ground.

[00:36:31]>> Well, 2030 is 5 years earlier than it was predicted to be when this was initially talked about. So that gives you I mean that's purely down to Ukraine undoubtedly and the and the speed of innovation that we're seeing that they're having to condense that time period. There's plenty I I don't I think Matthew's right. We mustn't overstate the AI revolution on the battlefield, but there's plenty of evidence that it's it's creeping into the the kill chain.

[00:36:53]It's making a serious contribution particularly in the way in Ukraine it can overcome the jamming issue and this last mile issue which makes these drones far more effective and increases the sort of kill rate anecdotally at least from 20% to 80% on many of these drones.

[00:37:08]So I mean there's lots of steps to come.

[00:37:10]I think one of the things they've got to do is make AI less brittle in real world battlefield combat conditions. That's one of the issues they've had. But the development is is hugely rapid. I was reading that the lock on distance for some of these last mile AI modules of for instance I think it was something like 400 meters when they first launched these things and now it's gone out to 1 and a half miles uh where the AI is taking over. So you can see it creeping backwards in the kill chain if you like.

[00:37:35]So there is this intersection, this convergence I mentioned earlier of AI and warfare that's going on. And I think it's um to what Matthew just said really it's it's a it's a technological juggernaut if you like that that countries like the UK have definitely got to got to keep up with at the very least.

[00:37:50]>> Yeah. And as Matthew said, you still need a human to hold the ground.

[00:37:53]>> Absolutely. Yep. Yep. So we still need an army and we still need a military. Um albeit with all of this technological kit at their disposal.

[00:38:01]>> Okay, we'll leave it there for now.

[00:38:03]Simon Newton, Dr. Matthew Pal, great to speak to you. Thank you so much.

[00:38:06]>> Thank you.

[00:38:07]>> Thank you.

[00:38:08]>> Now, before we go, I just want to share a taster of an extra edition of the BFBs SITRE podcast, which is online now. In it, I talked to Rose Gottomiller, former deputy secretary general of NATO and US under secretary of state for arms control and international security.

[00:38:25]Across three decades, she's had a ringside seat at key moments in the changing relationship between Russia, the US, and allies like the UK. She's drawn on those experiences for her new book, Security Through Cooperation. In it, she sets out to debunk President Putin's narrative that the US is determined to destroy Russia, and shares memories from her own time in Moscow.

[00:38:50]This is a bit of a funny story because I became the director of the Carnegie Moscow Center in late 2005. In January of 2006, I moved to Moscow and the very first day, uh, it was a Sunday, so the evening TV show called Vestie was coming on and I flipped it on and there was the Carnegie Moscow Center shown in the screen and it said above Carnegie Moscow Center Nest of Spies. And so I thought, "Oh, great. Welcome to Moscow." And indeed, it was very tough. We had uh in my first months in office, we had uh several bomb threats with, you know, the bomb squads coming in with their dogs to examine whether we had a bomb in the offices. We had death threats against some of my researchers, which really concerned me a lot. It's interesting.

[00:39:35]The Russians are rather fatalistic, so the researchers didn't seem to be so worried, but I was greatly worried. And luckily, our ambassador at the time, William Burns, a very experienced diplomat, I think, was able to get that quieted down. luckily.

[00:39:49]>> So when do you see as the marker in time when effectively the NATORussia cooperation came to an end? I really take it down to the first invasion of Ukraine in 2014 because up until that point and I remember uh myself I was under secretary of state for arms control and international security and we had a number of continuing projects going on with NATO including trying to get uh Russia to help in the removal of chemical weapons from Syria which was a project that NATO countries participated in. And Russia at the time was thinking about it was back and forth with a lot of discussion but in the end of the day they did not participate. That was 2013.

[00:40:33]And so it was only in 2014 I would say when the invasion of Crimea occurred and the destabilization of the Donbass that the relationship came crashing down in in a way that was quite remarkable. NATO declared no more business as usual with the Russian Federation and shut down all of the projects that we had going on. I think you know quite rightly there was a reason good reason to do so. Uh but it's interesting that some cooperation did remain when I arrived at NATO as the uh deputy secretary general in 2016. there was still a Russian mission to NATO and a Russian ambassador to NATO. And so uh there was uh NATORussia cooperation continuing in the NATO Russia Council even uh until uh practically until I left NATO in 2019. And you can hear the whole story by searching BFBs sit wherever you get your podcasts. And that is all for now. Professor Michael Clark and I will be back with another sit next Thursday. From me, Kabau, thank you for joining us. Bye-bye.