My Tesla Heard the Siren Before I did

[00:00:00]How did it know that fire truck was coming? Cuz I I couldn't see it.

[00:00:12]>> Cuz normally I'd be pulling out >> Yeah.

[00:00:14]>> more to see.

[00:00:16]Cuz you can't really see cuz there's a lot of grass.

[00:00:19]>> So, my original understanding of how Tesla FSD processed information was always based solely on the cameras and what data they were taking in and using and processing that data to make a decision.

[00:00:36]And I really didn't think it was doing anything beyond that. I always just assumed that it was just the cameras. I never really even looked into anything further to see, you know, what all is taking place. I figured, you know, the cameras are bringing in this information, they're processing it, they're making a decision, and everybody always talks about the cameras.

[00:00:57]But as it turns out, your Tesla's actually doing something much more interesting than just pulling in camera data, processing that and making a decision.

[00:01:10]Your Tesla is actually listening.

[00:01:13]That's right. It's listening.

[00:01:16]Oh, actually we are pulling over for the ambulance, which is very good, very smart. Now, I have about 25,000 mi worth of FSD experience. And throughout that time, I have encountered many emergency vehicles. My car has pulled over for many uh police cars, ambulance, fire trucks, all during that time.

[00:01:42]And it's really interesting to me that sometimes I was noticing the car doing something strange and then pulling over before I ever really saw the emergency vehicle. And there were a couple times it happened before I even heard it.

[00:02:02]And so, that really kind of threw me for a loop, and so, I had to do some digging really to find out exactly what's going on with the car. Why is it figuring this out much more earlier than I am, and is it just the cameras? Am I not seeing the vehicles coming, and that's the problem? Because sometimes I'm not hearing them. And so, I did some digging, and I had to go all the way back to version 13.2 to really find any data on what was taking place with the car, and what I found out was not just me imagining things. It really is true.

[00:02:48]Uh the car is really listening, but it's listening for something very specific.

[00:02:54]So, to start with, let's take a look at the official Tesla release notes from version 13.2.

[00:03:00]And what that says is audio inputs for better handling of emergency vehicles.

[00:03:08]This is Tesla's official statement that audio became an input for FSD emergency vehicle handling. Now, Tesla uses the term audio inputs.

[00:03:20]Tesla does not say the microphone or in-cabin microphone.

[00:03:25]It's not really clear on what exactly it's using. It just says audio inputs.

[00:03:32]That's the official wording. So, it's reasonable to infer that maybe the cabin microphone is providing some assistance to this, but Tesla has not publicly described the hardware implementation. So, we're just going to have to assume it's using the cabin microphone, and maybe it is possible there is more to this than just the in-cabin microphone.

[00:03:55]There might be audio inputs elsewhere on your car, but that's just me speculating. The purpose of the feature, as stated, is that it is for better handling of emergency vehicles. We don't know if there's other things that the audio inputs are listening for.

[00:04:12]It just says in version 13.2, better handling of emergency vehicles. Now, we know this just isn't speculation because Tesla's VP of AI, Ashok Elluswamy, hopefully I didn't botch that name.

[00:04:26]Um he expanded on all of this by saying that yes, they were combining vision and audio for better handling of emergency vehicles.

[00:04:39]And this just reinforces what was released in version 13.2 and confirms that we're not actually going insane if your car pulls over and then all of a sudden you hear a siren after that and then you see the vehicle. So, it might be happening much faster for some of you, maybe it's caught you off guard, but you're not going crazy. It is actually listening using vision and audio for better emergency vehicle handling.

[00:05:10]Now, anything beyond that is pure speculation, but it's safe to say yes, it is listening for sirens, listening for those emergency vehicles.

[00:05:20]Now, to expand upon that, Tesla's privacy documentation states that Tesla vehicles are equipped with sound detection technology and explains that audio processing occurs locally in the vehicle with optional data sharing depending on user settings.

[00:05:39]This was added to the end-to-end FSD stack. Audio inputs were announced alongside other major perception improvements such as larger neural networks, longer context length, improved navigation, improved camera inclusion handling, and audio inputs.

[00:05:57]So, this shows that Tesla considered audio as another perception input, rather than just a standalone feature.

[00:06:05]So, this got me thinking, I wonder if there are certain things the car is listening for that we don't really know that helps the car process data aside from just the cameras.

[00:06:20]Um if you could just let your mind wander for a second, you could think, "Okay, maybe it's listening for a blown tire."

[00:06:26]Which would also, you know, be part of how the car would handle if you're in FSD, a tire blows, and the car has to make those adjustments based on a blown tire.

[00:06:40]Pretty interesting. I'm not saying that it's doing that, but that's just part of where my mind starts to wonder when I start thinking, "Okay, they're using audio. What is it listening for?" Maybe ice on the road, ice crunching underneath the tires is also processing that visual, that the road looks slick, so it's also processing the sound from the tires.

[00:07:05]It's just a thought.

[00:07:07]Uh maybe there's some things that you could think of that the car might be listening for aside from sirens, or blown tire, or maybe ice, or some other things that would cause the car to process and then react.

[00:07:23]I don't know, maybe they're building that data as well. That's very interesting if they're building all this data with audio. We always think they're building all the data with the cameras, but version 13.2, all the way back then, it was processing sound data. So, if it's processing audio data all the way back then, that's very interesting. And what are they doing with that data? Another one that came to mind was when you're on the highway and you have a downpour of rain and you see the little warning sign from FSD, it could also be processing the sound of rain hitting the car.

[00:08:03]That's another thing that I thought about. Uh maybe even hail, you know, tapping the car.

[00:08:10]There are so many different things that they could be adding to all of this with audio. It's very interesting to see how that would uh come into play. You've got all the cameras, you're processing all that data.

[00:08:25]You're now also processing audio. And I don't know how they're doing it. Maybe there's other mics on the outside of the car. I don't know. It doesn't say any of that in the data that we got from version 13.2.

[00:08:39]But it's very interesting to see where they're taking that with audio inputs.

[00:08:44]So, what this basically means is let's just say you're cruising along in FSD, having a nice ride, everything's going good, and all of a sudden your car puts the turn signal on, pulls to the side of the road, and stops, and you have no idea what's going on.

[00:09:02]There's a good chance that in just a few seconds you're going to hear a siren and you're going to have an emergency vehicle come flying by. Because FSD isn't just watching your surroundings, it's also listening.

[00:09:20]And maybe more than just emergency vehicles. We don't know.

[00:09:25]If you enjoyed this video, give me a thumbs up and subscribe, and have a good day.