The benefits of deep learning based animal tracking

[00:00:00]so good morning good afternoon good evening and everyone joining in from different time zones for me it's still in the morning for Roman my co-presenter it's already in the afternoon so welcome to this and all this webinar today um where we are going to talk about the benefits of deep learning based tracking and the power of itovision XT so introducing ourselves my name is Stefan panhandinga and I've been working at nordis for over a year now so um a bit of a new phase and I work in the marketing communication departments and I have a background in behavioral Neuroscience from the University of kroning I've been in the in the animal Labs there for for a few years and picked up quite some experience there in this field and then my my co-presenter um from across the ocean the Roman Hollands he um he works uh he's a deer colleague of mine works in sales departments as an international account manager and a consultant and he has backgrounds as well in a neuroscience cognitive and clinical Neuroscience so um and we're going to show you today um well some of uh some of the nicer things and tricks that ether Vision can do in the behavioral Neuroscience field so before we get started you get the most out of today's webinar this webinar is being recorded so don't worry if you miss anything um everything will be available afterwards and you can watch it back on demand if you have any technical issues reload the browser and you can always communicate with us through the um through the chat box which we will monitor at the end we will launch a poll we would love to hear from you so any questions thoughts and comments please let us know in the Q a box so either Vision XT perhaps a lot of you already know about this maybe some of you don't it started out as a as a video tracking software but in in the meantime it has grown in my opinion to do much more to to really a behavioral recognition platform aimed at automatic detection and Analysis of of behavior for any animal in any environments and it has evolved with Maldives also over the last 30 years and it's been used in in a lot of sites worldwide and has been academically validated in thousands of Publications as well um and it is really a complete approach to measuring your behavior in full detail so you can see if you screenshot here as examples it's not just tracking tracking an animal scoring the behavior it's also visualizing and analyzing data using this directly in your in your Publications and in your in your research and to summarize some of these major features we aim of course at experiment automation integration of external systems and data movement tracking is one of the main things of course activity Mobility but also Behavior recognition um instead of just merely tracking a position really recognizing specific types of behaviors and as I said also data analysis and visualization so it's much more of a platform of a complete package so the division um offers nowadays um one system many applications as you can see here a lot of a lot of these tests and tasks can be measured in either vision and also in multiple species fish rodents insects and um and other mammals today we are going to talk about rodents um and I would like to refer you as well to our behavioral Neuroscience ebook which is also about rodents and the technology we're going to talk about is specifically aimed at this use so that's why we're going to keep it condensed on this topic if you want to read more about basic behavioral Neuroscience tests how do I measure anxiety what is an elevated trust maze what are readouts I get out of this feel free to check out this ebook it's a free download from our website I'll click on the link for you quickly to see where it um where it directs you to so it takes you to this location on the website um with the download button so feel free to to check that out and also you use it to acquire literature references which are also Incorporated going back to the presentation let me get the yeah the full screen here we're going to talk about artificial intelligence deep learning technique so a lot of expensive words for a system that we incorporated in either vision for multiple body Point detection and tracking for now in rodents as I as I said and um what's this technology is it's it's also referred to as AI tracking and it's essentially a neural network that has been trained to identify roads and this movie which I'm playing here on the side it's narrated so you can also find that on our website via VIA this link on The Ether Vision XT detection page and it's sort of a comparison to What's um the technology has always been like the gold and standard which is Contour based tracking versus this new technology that we are now that we have now introduced in the division X team so I'm going to pause this real quick as you can see here this black mouse is identified um normally by the software by its Contour you can see a red outline here around the Contour of the mouse and based on this Contour three body points are are assigned which we can then track so in those points is Center points and the tailgates what's different in the Deep learning um in a deep learning technology in efficient XD is that these the the this shape and these body points um of the of the mouse are actually assigned due to a trained input this is a nice little graphic um that that sort of explains that all these images um have been trained into this neural network um from a large uh repository of of movies uh with different backgrounds different types of of rodents different colors sizes Etc um so as I said essentially a neural network if you can if you want to see this video full detail with the full narration I'm gonna again click on this link um afterwards of course you will get this access to this as well so you can look back at it and here we explain a little bit more about how it works and how it's different from the traditional Contour based tracking and also this video which you can for the watch then you go back to the presentation um and the technology can handle up to four Arenas at the same time but I'm going to let the men talk about that in a little bit what are the benefits of this neural network tracker well during this presentation we hope to convince you of that of course um but it's to automate object interaction tests for example normal object recognition test or Barnes maze um the holes in a Barnes maze or in a whole Port tests are are essentially also objects um there are no more no-tail switches so no more no it's not necessary anymore to have manual Corrections afterwards if there is a misinterpretation it's highly reliable in the detection um and also in changing contrasts you can differentiate between hovering head tips and peaking um we you will see that in a moment for arena simultaneously and most importantly there's no training needed of this area we have already trained this as I said with all the inputs um with with all of our own inputs so um you can use either Vision directly with this functionality to get direct results and visualization at this point I'm going to give the presentation to Roman and he will try and convince you of of these benefits so Roman I'm going to let you share the screen so I'm going to stop sharing thank you so much Stefan yeah that's the uh benefit of uh sitting across the ocean from you is that it's quite easy to uh to open up my screen as well so just to check with a few okay um everything's for me hello can you hear me I cannot see your screen yet pushing a bit one second yeah yeah I think my uh my zoom quit unexpectedly that's the nice thing all right so I'm sharing my screen now it's all everything visible to you yes see two and a half years after and zoom still quit some time so before uh I go into the full details I want to give you a small technical note on the examples that you will see in this webinar and that is regarding the visualization of the detection that will be shown to you so there will be a few instances in which you will be shown an animal rodent uh and which a few parts are visible when it comes to the detection so the examples that we use come from the detection settings and it's a bit different that you're used to from the beginning so when we show a rodent you will see that it can have uh perhaps a red outside line um that you see right here and that is the Contour tracker border so we're going to show two examples here we have the Contour tracker that identifies the animal and we have the neural network that identifies the body points what you see the red line at the outside is the Contour of tracker border then what we see and um in the yellow area this is identified as the animal area so again this is the Contour based tracker that is identifying where the animal is and the yellow part is identified as the animal area as we see that this is the animal so the software says this is the animal however what is important in this the nose and tail Point are defined by the neural network tracker and can therefore be outside of the Contour tracker border so as you see in this example there is also that these both are at the same time the neural network tracker identifies the body points of the animal the Contour of the animal is defined by the standard tracker so that means that body points that you will see are disregarded from any of the other identifications of the body of the animal that you see it's a good tactical note that you understand if you see some examples where does it come from and and how does that work because normally as it would be used if you're used to other versions of e to Vision XT this would be impossible right so the body Contour would then always be very specific for the uh the body point of the animal then another technical note on that is that the orange that you see is the noise and the noise means that it's matching with the detection criteria but it's not connecting to the detected animal area so that means that for instance here we have an animal crossing over a hole the hole is the same color as the animal where the Contour tracker would then stop because the color is the same as the background and here you have a pot of noise still at the beginning but as you see again the body points are disregarded um with the identification when it comes to the Contour so the center point is then the Contour based but to summarize this the neural network tracker disregards the body Contour yellow area and Orange area it is included in these examples to ease the in comparison of improvements so that means that if you see something like this like a body that is only half and the nose point is still on the right place the neural network tracker disregards the body Contour right that's good to understand and these following examples that you see because you will see the yellow but it's disregarded for the body points once that is clear um we move on to the next phase and that is I would like to show you how easy it is to set up the detection if you use E2 Vision XT you're already quite used to um to the detection setup that it's easy to do automated setup and you find your animal this is the same for the neural network identification including one step so I will show you video how this works you click on automated setup you select your animal type which is a rodent in this case you click next then you identify your animal so that is by dragging a window around the subject you find the animal it's a result okay if yes you click OK you click on method at the right side you click on deep learning settings Define and you drag a box that is about one and a half times the size of the animal and if you didn't done that you click OK and as you can see the body points are directly identified Now using the neural network tracker so again the yellow part is just a body Contour which is completely disregarded by the neural network tracker so it's a two-step setup completely automated in that sense the only thing you have to do is drag a box that is about one and a half times the size of the animal so that the neural network can identify the body points inside of that area continuously right so then one of the most important tasks that we know of and which the nose point is of extreme importance is the novel object recognition test and if you want to automate it such test you need accurate No spawn detection it's really key switches are not acceptable because if you have switches you need to switch them back manually if it makes a mistake of course you want to differentiate between investigation zones so that's the Zone around the object and you want to differentiate between real object interactions really no spoken for instance this can also say something about the intensity of investigating the object um you want to disregard when an animal perhaps is rearing against or definitely is sitting on an object and you want flexibility of shape size and color and as we see here at the right side for instance I have three examples of some snapshots I made from from some example movies here we see that we have a white object with a white red where we see that the identification of the nose point is absolutely perfect here we have a dark object with a dark animal and the identification is again really perfect and here we have a standard maybe low contrast maybe low pixel identification where the again the nose point is identified at the right place so here what I have in this uh is a video of four Arenas at the same time that is tracked with the neural network tracker so again tracking for an Arenas at the same time is no problem at all so it will start playing and what you will see in this example is that we have very accurate No spawn detection near to the objects and then it's as it's really essential for the readout of this test so also if you follow any animal that is working throughout here I have made a detection in that when the animal is with the nose inside of the object investigation zone or in the object Zone that you will see it outlined with red so that you can see how the interaction is and now I I am delaying the video play they can really see that every single time when the animal is walking or when the animal is near the identification of the nose point is absolutely perfect on the nose um what you are maybe used to from other uh trackers at some time is when the animal rears against and walks back that maybe no steel point or switched this is not at all an issue when it comes to the neural network tracker you can rely the neural network tracker on being extremely accurate when it comes to identification extremely stable and not making any switches so yeah I can play again when I want so no manual Corrections need with a deep learning based tracking so you can yeah of course you want to do a manual check but that is is possible with integrated visualization in E2 version XD so as here you see the zones lined out um and you can acquire four Arenas live simultaneously with the neural network body Point detection and the setup is extremely easy and the and the visualization is quite detailed as well so next to the novel object recognition relying on nose and the tailbase in the center point you also would like to filter out for instance rearing or sitting on object artifacts to come to the complete automation of this test um well how do you do that first of all you indicate the object so we've indicated investigation codes we've indicated object zones then we tell each division please filter out for instance in this example when the center point is not in the object when it's sitting in a specific Zone and now we see for instance if we look at the interaction data of the animal here at the right side we see that in this arena for instance the animal is sitting on the object and while it's sitting on the object we filter out this data so every interaction that is happening here so you see that there's quite some interaction it's grayed out by the software which means that this data is filtered out so to one hand you can rely on the interactions of the nose point with the object on the other hand you can rely on that you can filter out any instances that you do not want to take into consideration like an animal sitting onto an object [Music] um so then we have another very tricky to track test sorry yeah interrupt you with a quick question from the audience from Craig Weiss um Can the four Arenas be from four different cameras very good question actually yeah it's a good question and it can be uh each division XT disregards whether it's a similar video input or whether it's a uh whether it's uh four inputs at the same time uh it doesn't matter it would still see at that point for Arenas and for Arenas to detect so yes you kind of four uh cameras lined up with your PC going live into each vision and each Vision tracking for Arenas at the same time yeah cool thank you Roman for answering quickly no problem so then another tricky track is the whole board test or the bonds maze test as many of you know and what is the tricky to track here that is that you have an animal an animal that is interacting with objects and intra and animal is interacting with holes and these holes are dark these holes are light the animal is walking over poking its nose inside if you want to do the whole Board Test manually or a bombs test it's manually it's extremely labor intensive there's a key difference also between sniffing over a hole or poking in a hole so if an animal is hovering over the whole or going into it that's a key difference and the search strategy is often based on that the animal needs to poke inside of the hole and not just hovering over it and next to that you would look like also like to have flexibility of light and dark holes so that you can play around a little bit with that but what we see here in this video I will play it again from this point is that this is something common that happens and please remember again the yellow that is regarded to the Contour it's disregarded by the neural network trackers purely for visualization purposes only you see that the Contour tracker has issues with finding the animal when the animal is over the hole which makes sense because the contrast is not there there is no identification the animal at those stages because it's practically similar to the hole but what you can see is that while the animal is actually doing a search task it's hovering over the holes and the neural network tracker is still identifying the nose Point very accurately extremely accurately not losing it even if the color is uh completely the same when it comes to the hole so even if it works over here see that it still has an interaction with a hole still as here and you see the nose Point stays on its place this is pretty impressive um I found it pretty impressive at least because I worked with these things as well um so if we move forward and now we can identify the nose point on the exact place where it needs to be and you can actually find this interaction with the holes which arises maybe to another question how do I differentiate between interacting with the whole so poking inside and hovering over the holes um yeah we've looked at this as well and it was actually quite easy to solve this issue because uh because we can now identify the animal no smoking inside of the hole that means also something happens with the body shape of the animal and what I will show you in this video is that it's that we can differentiate now between the standard End Zone parameter of hovering over a whole for instance so being in the zone with a body point and actually doing something else so you will find two um parameters down here the upper parameters the standard End Zone parameter the down parameter makes a whole poke add the differentiation the zero stands for is on the whole the nose po the nose is indeed on the whole but it does not fit my criteria with the animal being inside of the hole the one is yes the animal is with the nose on the hole and it fits with my criteria of the animal being inside of the hole with the nose so you see this changing this is for comparison purposes you see that now when the animal arrives that there's a slight delay in the whole poke before it says that the animal is actually inside here which is interesting to see but most importantly you will find now is that when there are moments that the animal hoverswift had over the hole which will arrive now you will see a zero for the whole box where the end zone is one and again also at the next point I think it does it three times and if I'm correct yeah you see here as well it identifies it as a zero so even here you see I go back with I hover move the mouse over it and also if I go back to these instances it hovers over but it identifies it as a zero now if I go back to moments that it goes inside of the hole it identifies it correctly as being inside of the hole this is all possible because of the neural network tracker because we can identify the nose point so with this perfect nose Point detection you can find interaction with the holes you can differentiate between whole investigations and hovering the head over the holes you can find the exact and accurate search strategy so about that search strategy and at the same time I will play this movie at the left side we have the filter of the real hole Folks at the right side we have the filter we don't have this filter for the real whole box so if there's a whole poke the line will be bright red as you can see now so now all of them are practically the same but the amount of red lines is a little bit more when there is a filter so here as well and afterwards we arrive at the moment that the animal was actually hovering over their hole so at the left side we can see nothing red at the right side we see a red spot and now with the left side again we see something that is gray at the right side a small red dot and again at the right side we see a red dot so if we summarize that we can see that if we filter really by a no spoke we see that the strategy here is one two three four and the third strategy here is one two three four five six seven and this is quite an essential difference difference if you want to automate to a test for instance you want to rely on this third Paradigm so the target visits an errors which also rolls out automatically from E to Vision it's quite different as you see there's quite a uh there's three more uh no spokes detected without without a filter so again um animal contrast extremely low no problem the body point tracker will find the nose in this case so then another point is um transitions between physical zones which is an uh and it's happening where the animal is for instance cut in half so essentially the tracking between physical zones is done very well by you division XT already in the sociability cage the light dark box Place preference all these types of Mazes they have physical zones through which the animal can pass um no essential issue that impact the results but the neural network tracker can be used for fine-tuning this Behavior your animal will not be cut in half anymore meaning that of course the body Point as you can see right here is cut in half and you can find behaviors such as speaking half transitions things that just make it a little more subtle to study this is an example of that so in this case we find that the animal is cut in half where the Contour is completely cut enough but the nose point is still identified exactly at the place where it needs to be this has benefits so the neural network record disregards the body Contour of the animal being cut in half and perfectly detects the nose point and I have an example of this of what this looks like and the upper parameter here is a parameter that I just put into each division which is very simple for you to do as well which identifies peaking to the light zone which means it went into the light zone but it went back and it didn't make a full transition the one at the bottom here is the transition to light so that means that the animal does make a transition a full transition to light so when I play it here it's going to be full screen you will see that the animal is now in a dark zone now replace preference we move a little bit to the light and now we see that there's a scoring right here with peaking to light because the animal moves back the nose point is identified outside but it moves back um now the animal is just again in the dark environment and it starts moving again and now we see at the bottom we have a transition because the animal now fully moves to the next stage actually instead of just poking it a nose outside and coming back again this is a very nice addition because instead of just looking at where does my animal want to be you can also look at very subtle differences because peaking outside is a very subtle behavior that might also indicate something when it comes to anxiety or if you're studying uh the the photoreceptors in this case um so that is something that is a real benefit of the neural network tracker as well that you can disregard these instances where the animal is cut in half but there's some other tricky environments that I would would like to touch upon briefly first one for instance is a common one to other people as well batting material and the same color animals so if you have a white mouse or if you have a white red with the Reds for instance your bedding is mostly quite bright there is a bit darker bedding but often the bedding is quite bright It's Tricky environment to track that animal standard animals are also very tricky to track because the the the the the the line that you have above is constantly going into the field of view rearing onto a light or dark wall it's also quite tricky to track because if the sides look the same then where is my animal and if I want to look at rearing this nose point is quite essential flooring with uneven colors happens quite a lot maybe you have some shadings right Reflections Reflections is definitely something that is difficult to overcome how can you it's it's you have some mazes that avoid Reflections but it's not easy to completely overcome them and maybe also anything else another tricky environment that you can think about please let us know in the chat and we can think about if we have figured out a nice solution for that so let's look at some examples because uh I wanted to accept this challenge so first here at the left side we see a batting material with a white animal and now what you see the Red Line This is the Contour that is visible and what's your common what is very common in a betting environment is that the Contour becomes so disturbed because of the color of the bedding that is underneath an animal that is very difficult to find the nose in the tail base but again we have this neural network tracker this is disregarded the neural network tracker can identify the nose and the tail base very accurately even if the contrast is not so nice even if the environment is quite tricky and you see even while jumping going in and out the animal nose is constantly on the nose and this video I made from the detection setting view in E division so it's nothing that I adjusted afterwards you can you can pull it the detection yourself it's exactly how it comes through the detection settings now at the right side is also quite a complicated situation so first of all between left and right and the left I left the Contour features visible so the yellow and the orange the right side I made them hidden but it's essentially it's the same video but it's just for visualization purposes of what happens to this Contour actually when you are having a dotted animal and you see when you look at this Contour it's completely Disturbed it's cut in half it's flickering it's now with the back it's at the front it's difficult to keep a body shape however filtering that out is a bit more stable to see it's easier to identify what you're actually looking at we see that there's nose Point even if the animal if the data point is moving throughout is very stable it still stays on the nose as you want so that means that even these behaviors even behaviors that involve the nose can still be identified very accurately using the neural network tracker then a tricky to track environment finding a good Contour at the left we have a low contrast wearing so for instance here the animal is dark and it rears against a dark wall and as you see here I'm sorry I clicked again but as you see here the nose Point stays on the nose point so if the animal moves throughout it rears against the wall the nose Point stays on the nose point because the neural network tracker doesn't look at the Contour it looks at what it has learned with the environment um where the nose point is so it really keeps it extremely stable and here at the right side is a nice environment that many people know of open field simple test but if you want to identify body points you often have Reflections and how do you cope with these Reflections yeah you can try some filters that works quite nicely but with the neural network tracker you don't have to worry about it anymore because whatever reflection you have and here at the right bottom side is an extreme example again the yellow is for the visualization purposes of what is identified as the body Contour this becomes nearly a square blob um where you can't find a shape really but the neural network tracker can still identify exactly what it needs to find so it finds the nose it finds the tail exactly on the plates where it needs to be um which is quite cool which is really interesting that it finds it in such a stable way um yeah from that point I will now give it over to Stefan again Stefan will tell you a bit more about the long evidence rat I will stop my sharing so that I get a chance to to yes Stefan yes I will I will share my screen there are there I see all the questions um coming in and we will try to answer some of them um maybe I can get into well yeah yeah this may be easy to answer um that is uh I got a question about can you reanalyze the old novel object recognition data with a deep learning feature yes so we will tell a bit more about either vision and how to do that but yes you can so if you have all videos available no problem you can also track them offline afterwards yes yeah yeah and I can also answer one um from Kirsten and what are the technical requirements for running this module Windows version processor speed graphics card Etc um and we do not have one uh specific requirements we have tested um tested this this deep learning technique in in a few systems that are based off of Windows 7 and Windows 10 and you do need a graphics card to run um the Deep learning module um so it's uh well that's also a very technical story it's it's um dependent on the the Cuda course um but our sales Consultants can guide you a bit more on that but you don't need a super computer or anything like that um the graphics card is essential um and then before I move on there's one question from Adam s how about hooded Rats on a grid floor well I'm not sure about the grid floor but I am going to tell you something about the hooded rats um also known as the long evidence wrap I'm going to show you a nice example so a bit of background first on this on this hooded routes it's a cross between a female albino rep and a male Wildside rat uh higher resistance to respiratory problems so it's actually preferred for specific surgical procedures where you use inhalant anesthetics and they're commonly used in addiction research especially alcohol alcohol addiction research they have a higher resistance to Uncle Genesis but all in all a very preferred rap model for specific Behavioral Studies to link also to um specific brain Networks um the thing with this with this rap model actually when you track it is their distinct fur pattern especially from the top view so they have a sort of a black pattern on top of a white fur um and if I overlay a contour how this would be recognized for example by either Vision um you can already see that this is not the actual Contour of the animal well as Roman also already explains the neural network tracker or the AI tracker the Deep learning Technique we run a few names for it um disregards this Contour so it really based on the word input the text these multiple body points and we don't have this problem of the misinterpretation of the Contour anymore um which makes using the the long evidence wrap or a hooded rat a lot more user friendly in behavioral research and I'm going to show you this in a in an example um an elevated open field um and the goal of this test here is to measure um sort of head dips over the edge of the Arena um as a readout for anxiety like Behavior or or stress um and as you can see here you can see the animal rearing where the nose point is very accurately detected and as I already said this um here I turn on the Contour so you can see how the Contour visually would normally be assigned by a division but because the AR tracker disregards the Contour we can still even if you see a head dip at this moment or leaning over the edge that there's nose Point tracking is is in fact so I'm gonna go over to the next video which is the integrated visualization to see how this data readout would look like where we have a differentiation here in the blue the blue bar on the bottom here where it's just looking over the edge slightly if I pause it here and you can see the green bar on the bottom which is an actual head dip so it has the nose outside of the edge of the arena and it dips down and then there's also a yellow state where it only reaches out which is not with its nose and um at a certain distance from the edge of this Arena it doesn't dip down so really again a very nice example of how does neural network tracker even in a hooded rats can accurately keep this nose Point tracking intact um and this is that these are actually all the examples that we um that's um we're going to show today so um we would of course love to hear from you also based on what we showed you here today um you can always reach out to us these are email addresses a lot of the information is also on our website and you can of course find us on on our social media platforms um and I think we just want to go over to the Q a and try to answer some of the questions um that have come in what I'm going to do first is I'm going to launch a poll so that you can answer that one and then um Roman do you have the Q a in front of you can can already answer some questions yeah so um we the next question I arrive at is from Nadia for instance or Nadia asks whether we are familiar with neurophotometrics fiber photometry systems um and whether it's possible to export timestamps of distinct behaviors such as entry into open arm to separate data file to coordinate calcium signals at that point uh so yeah it's a vision XT even if we don't know the exact system if I don't know the exact system that you're using but it's definitely possible to have timestamps of a specific behavior happening and to also trigger devices for instance through a TTL on the basis of a specific behavior automatically through either Vision so if you want to for instance that when an animal enters a certain point you want a timestamp or a signal to be sent to an external device in most cases this is very easily uh to be done it's very easy to arrange for for a specific system I advise you to also reach out to your uh to your personal um sales colleagues the consultant at all this for for helping you further with setting it all up um so then Adam asks um how about hooded Rats on grid floors the hooded rest we saw what about grid floor excellent example actually uh I tracked an animal on the grid floor and it's no problem at all I don't have the example in this webinar directly but it's it's perfectly possible and it also gives a very very good identification of the body points um we can give you an example afterwards if you if you want if you contact us then make sure to show you an example of that um all right then in the Contour method Suman gunin asks in the Contour method the nose Point often gets switched with the tail Point does the Deep learning feature ensure that this not does not happen yeah so the Contour based method is a very good method of data identifying the animal and it was always based on when you have a contour you find a specific shape and on the basis of that you assign a body Point what I have seen in my experience with I think currently now over 100 different videos that I've put through E2 Vision XT and that I've used in neural network tracker on is that I do not see no stale switches so no stale switches I've not seen happening and that is how the how the network has been trained that apparently this is not a mistake that it makes it's not something apparent that it does um it could be maybe that the nose point and very very difficult situations might be slightly off but this is unique I've seen this only in a few cases that that has happened but no steel switch does not happen um and before I maybe uh move over to uh to to Stefan for some answers it says stina lundback asks what about peaking from or two hidden zones uh yeah so there's one essential uh key point in the neural network tracker the neural network tracker identifies a body point if it has if it has an indication of where that body Point might be at the moment that the animal moves into a hidden zone for instance maybe suppose you have a shelter or you have a zone that is not visible from above um the No No spawn point is visible so that means that also the nose point will then not be assigned because it's not visible however it's still possible to actually have a transition in E2 Vision that if you use the hidden zones that if the nose point is lost inside of that hidden zone right that you have you can Define it in each vision and it moves in it will then identify it well each division will then know that the nose point is inside of that place and then when it moves out again it's a transition from outside to inside to outside without for instance the center point being inside of that object so then you can still identify peaking inside of inside of a hidden Zone yeah um maybe Stephanie you want to answer a next question yeah I see a question for example by Sarah Adams what is the difference in deep learning function between E2 Vision XT 16 and 17 do both have the same level of deep learning capability and there's also another question is deep learning so this related to this one is deep learning um um available and in in in all versions of itovision or is it a separate module um it is part of ether Vision based so the base version of innovation from e2vision xt60 so we are now at Innovation xt17 so the last two versions of the division have this deep learning module and you can still choose in the detection setting whether we use the Contour based or the Deep learning based tracking and the difference between 16 and 17 is um is is some some minor tweaks of course on the background but the main thing is these these four Arena tracking simultaneously that's the main upgrade from 16 to 70 for the rest um I think most of it is is the same if I'm correct from him yeah that's true indeed so that that changed it's it's a bit of I think in the amount of effort is asked from the GPU that has also been improved to 17 so it's much more efficient so also I'm running out on a laptop with a good GPU which is not advised but still it's a it works quite well quite fast so this is definitely true yeah um yeah I see many questions also if you have another questions or if you like a question very much don't forget to upvote it because then it's easier for us to see um a lot of questions are coming in which I which I like that's why we took the time to not make the webinar too long so we really have time to to look at people's uh concerns and what they need of course yeah um let's see can the neural network tracking identify more mice of different colors for example white and black on bedding in multiple Arenas that's a question from Sarah bonini thank you Sarah Roman would you uh dare to answer this one yeah um yeah maybe I want to specify it so can the neural network tracker identify more of different colors mice of different colors in the same So currently the the neurological tracker is specialized for one Arena ONE animal in one Arena that that is where it's been built for currently but might have different colors white and black on batting in multiple Arenas so if you have um at the same time I think you're saying that you have a white animal in some Arenas and a black animal in other Arenas do I get that correctly I think I I don't know if you're able to speak I don't know if that's is that something that we have in this webinar um yes I can but then I have to select the participants if you would like to specify your question uh yes yes can you hear me yes okay so um my experimental setup is to um to move to look at different mice because we have to track the the moving of uh more mice at the same time so we have to um to follow for example six mice in different cages so they stay in the in their cage with the bedding and so only crack the movement of the mice but some of them we have my mice of different colors so some of the them of are brown and some of them are white so um is it possible with this system to follow correctly uh this mice also if some of them are white and some of them are brown yeah thanks for uh for specifying it so yes if you have animals of different color it doesn't matter so the detection settings can be specified for each animal um depending on how much the animal differ in in color it can be either tracked in the same file so in in the same trial that you run if they're very different so if you have a white animal and a dark animal so if the white animal is lighter than the background and the dark animals darker than the background is really advised to track it in two sets so one of them you could do live and one of them you could do offline within the same experimental file using detection settings one specified for white animals one specified for dark animals but yes and the same experiment you can then run same animals at different color animals at the same time and be able to track them afterwards all correctly yes so I have to set to set up different detection settings right yeah so two detection settings yes one specialized for white yes so yeah I cannot use the black and the darker and the brighter animal so you know that we can choose between dark hair or um lighter lighter or both of them so uh it's advised you can try it and it's it's you have a chance that it that it that it does but it's really advised to make it into separate detection settings for the best results in very challenging situation right yeah but it's still if you just recommend after each other because you want you can do live and one you can do from the video it will give you uh perfect results yeah okay thank you very much no problem thank you as well Sarah [Music] um so we still have a lot of questions um there's one with the most upvotes at this moment by Miguel novello can the neural network tracker be used for forced swimming tests I've been struggling on setting up the detection with the animal Contour slash area tracker do you have any experience with this woman yeah so the four spin test is needed a quite specialized test also when it comes to the the the the direction that you're looking at of the animal um it is not that the neural network tractor has been made for this but we have actually very good Solutions uh for automating the four swim tests uh we can automate it in in quite high accuracy as well using the activity detection instead of the uh the Contour detection of the animal so instead of using the control tracker you use the activity detection and this really works very well for the full swim test um Miguel I advise you to contact the knowledge representative in your area and if you don't know exactly what it is you can contact us and we will give you the right information to track uh the four swim test in in a good way yeah okay and then to follow up with another question maybe based on on activity detection um from anuniz Guarino I hope I'm pronouncing all these names correctly I work with epileptic rats do you think it may be possible to distinguish a seizure from for example um grooming with the software mm-hmm yeah so uh that quite specific indeed um the nose and the tail base do uh it really improves the amount of detail that you can find for instance in the seizure when there's a movement of an animal that are quite odd so that the Contour tracker might maybe be confused of the body points uh it is quite in a good way quite possible to find the right settings of the the movement of the nose Tails or the velocity of that what's happening in their vibrations uh to identified a to identify a seizure on the other hand when it comes to the grooming detection um that's different part of the software indeed so that's the automated Behavior recognition that we have in which you can recognize the grooming that's a different path from that so indeed if if the grooming is something that you are quite able to identify for instance then with the neural network tracker you could use for instance the improve detection of nose tail to uh to identify its seizure but it's it's quite um it's quite specific specific to the environment that you have and to the seizures that you have so we do have some experience with that again um if you want us to tell you a bit more about that or some examples reach out to somebody at knowledge um I've had a question before I tried it as well and it the results looked quite nice yeah yeah a similar question came from uh Layla I would like to know if you have something in development for the recognition of seizures and rodents but I guess it would be uh the same yeah yeah so it's uh so on the one hand indeed being able to better detect nose and tail base improves also the way that we can learn our algorithms that we have for instance on recognizing other behaviors um so this is uh this is definitely on our wish list of of uh to develop yeah and manually already if you want to make some detections of movements of no steel to find a seizure we can do that already automatically in a certain point we would really like to have that as well yeah yeah we're going to do one final question and then the rest we will answer afterwards you will all receive this whole list of Q and A's which we will also type in our answers so you can review them um so this final question we will answer live from Washington we perform the open field test under red light conditions a lot of nose or tail or Center tracking points were lost how to correct them efficiently yeah I think the most efficient way of correcting this is using the neural network tracker um doing this manually is quite uh is quite it's very labor intensive if you have many no steel switches so in the red light the animal is red and the background is often also red so it's it's a tough situation but I've seen it being tracked in the neural network tracker before working quite well um again depending the environment but I I would definitely say if you still have the videos um take them through e division 16 or 17 and that is the most efficient way of of tracking them and then I want to quickly point out to a crack wise mentioning indeed that you said I that I said that only one Arena can be tracked at one time if so can four cameras be used for collection and then reanalyze the videos offline no four Arenas can be analyzed at the same time so the E2 version 17 can track four Arenas at the same time using the neural network tracker so that can be four Arenas in the same view or four arenas from four different video inputs now questions I see some about also zebrafish questions and I saw one about dogs we will go into that one uh into those a bit more afterwards you will all get an email as I said and um we will answer everything and if you have any other questions do not hesitate to uh to contact us for now I would like to thank you very much for attending this this webinar and we uh hope to of course see you in the future and um Romaine thank you for co-presenting this with me yeah happy to do so I hope you uh you all enjoyed the examples uh I like working with each Vision quite a lot as you might have seen um so if you have any questions just reach out to us um through email or phone and we'll be happy to come back to you yeah there are a lot of uh situations that require just a little bit of more Custom Touch so uh but we could definitely help with thinking about how to solve those yeah thanks for the feedback indeed uh super thank you nice to talk to you thank you yeah I'm happy to do so