Using AI to detect whales in San Francisco Bay | KTVU

Added: 2026-05-24

[00:00:00]Well, scientists have been working to try to find ways to combat a recent spike in whale deaths in the San Francisco Bay. A new AI-powered detection system, which is called Whale Spotter, could be one way to help.

[00:00:13]Tonight, we want to go in-depth on this new system with Dr. Dan Zitterbart, co-founder and chief scientist of Whale Spotter. Nice to see you, Dan. So, help explain how this technology works.

[00:00:26]So, Whale Spotter is basically a thermal imaging sensor that measures heat while it's looking at the ocean. And when a whale comes to the surface and breathes, that breath or the whale's spout is a tiny bit warmer than the surrounding water on top of which you see that. And we developed the technology as well as a an AI with that or machine learning model, as you want to call it, that is capable to discern those tiny little thermal signatures from the rest of the noise, other ships, waves, birds, and tell, "Oh, there's a whale out there."

[00:00:58]And we can do that day and night, as well as many miles out. So, the system here works about 2 to 3 miles away from the shore, giving large coverage in the critical area in the bay. Okay, so so are there cameras involved here? And if so, how many and where are they mounted?

[00:01:16]So, it's thermal cameras. They measure heat, but it's cameras and sensors. And right now they are mounted, two of them are mounted on Angel Island overlooking the bay, and two more are mounted on a ferry transversing the area. This is a start.

[00:01:31]We are trying to understand how well that will work in the area and how well that can be communicated to the mariners. Okay, and >> To avoid and Go ahead. No, go go ahead.

[00:01:39]Go ahead.

[00:01:41]To avoid and basically alert everybody in the bay, that's done through the Coast Guard, through a VTS system, that there are whales out there, and so people can be on the watch, slow down, or change their course to avoid the large whale aggregations.

[00:01:55]>> Okay, and these large ships would get a warning in time to be able to make maneuvers to avoid the whales?

[00:02:03]Yes, they can do that. We we have done that all across the all across the world and we see that daily that large ships maneuver to avoid whales and not hit them. How long have you been working on this?

[00:02:15]I started this 2009, quite a while ago.

[00:02:17]So, the research phase of this has been going over 15 years and we made put all our research together and made a product for that about 2024. So, now it is commercialized and it's reliably working and not a research project anymore. Okay, what about false positives? Have you encountered that with the technology?

[00:02:37]We used to, we don't anymore because we verify everything, right? Captains hate false alerts. If a captain gets one false alert a day or an hour, they will stop trusting that system, so they don't like it at all. At Whale Spout, every first detection in a new whale encounter is verified by a human in real time. So, the AI thinks, I think these are whales, then the AI sends it in real time to our team where where experienced marine mammal observers say, yes, the AI was right or the AI made a mistake and only the right ones get transmitted to the captains. And this takes between 30 seconds at the start and 6 seconds at the second blow of the whale. So, mariners get it in near real time, fast enough to take evasive action. I know that this is something that you're looking to deploy in other parts of the world, but specifically here in in the Bay Area, why are more whales coming into San Francisco Bay?

[00:03:31]Because there's less food in the Arctic and so on the travel between Baja and the Arctic, they take a they take a rest, they take a stop in the bay to feed on the crustaceans in the bottom of the ocean. In the silt.

[00:03:46]And so, obviously, given the fact that they're making their way here, It not as if mariners are used to seeing so many whales in the bay.

[00:03:56]Right. Especially they always have been where many whales out of the bay, right?

[00:04:00]This is one of the most biodiverse areas for whales, but in the bay that's not very common, especially not with gray whales. There always have been a few, but not to this extent and especially not for that duration. And our aim is to be able to know when are they in, when are they out, and where are they in real time to actually allow boating to go by its way while protecting the whales.

[00:04:20]Well, it seems like this could this could really make a big difference.

[00:04:23]Appreciate you coming on the show and explaining the the new technology with all of our viewers. Appreciate your time tonight.

[00:04:30]Thanks for having me.

[00:04:30]>> Mhm.