Circuit Breaker Pattern in Microservices | Resilience4j Circuit Breaker Explained

[00:00:00]Welcome to another video of code snippet and in this video we are going to look into another important resilience pattern inside microservices that is circuit breaker. Circuit breaker is very easy and very very important pattern for distributed microservices. So let's get started without wasting any time. This is going to be another fun video. So sit back, relax and enjoy the show.

[00:00:21]All right. So let's quickly jump into the agenda of this particular video. So circuit breaker inside spring boot microservices using resilience 4G. So in this video first we will start with a problem. So what exactly is the problem that this particular pattern is trying to solve. After that we will see what exactly is circuit breaker. We will see why do we need circuit breaker and we will see different states that are present inside circuit breaker pattern.

[00:00:46]After that we will look into the implementation and configuration different configurations that are needed in order to set up your circuit breaker.

[00:00:54]At the end we will see a quick demo. And that is basically the simple agenda. And if you remember before this video we have seen rate limiter inside spring boot using resilience 4j. Even before that we have seen retry pattern and even before that we have seen what exactly is resilience and fault tolerance. So all these things we have already seen and in this video we are going to kick start with circuit breaker pattern inside spring boot microservices using resilience 4j. So so that is basically the simple agenda. So let's get started.

[00:01:26]So let's get started with the problem.

[00:01:28]So if I go to canvas over here then this is basically our setup right. So we have this particular order service we have inventory service. We have ureka server.

[00:01:36]Even before that what do we have? We have API gateway and client will send kind of a request. Now order service need a inventory data from inventory service. So it will just go ahead and request that inventory service will take the data from database and send it back to order service. That is our simple flow. Now we have seen retry pattern. We have seen rate limiter. Everything makes sense. But what if this particular inventory service is down? Now this guy is down. And if this guy is down, order service will keep on sending the request to inventory service. So now order service is sending this request and this request is failing because our friend inventory service is down in the first place. Now let's say user is sending thousand requests from here. This particular order service supports thousand requests and client is sending thousand requests and all the thousand requests are going to your inventory service and all thousand requests are basically failing and now imagine within that we have a retry mechanism over here. Now if one request fail what do we do? First thing we do is our first instinct is try to retry. So what we will do we'll send another request which will again fail. We will send third request which will again fail. So retry is not making any sense over here because inventory service is basically down in the first place. If this guy is down, no matter how many time you are retrying, every call is going to fail.

[00:03:07]So that is basically the problem. Now what will happen if you keep on sending this request? On each request, we will need multiple resources. Each request that we are sending over here needs resources. When I say resources, it will consume your threads thread of your order service. So those are basically the resources. After that there will be network connections. So we are sending this request. We are making this rest calls which will consume network connections. Again basically consuming resources. After that your order service is running somewhere. So that will go ahead and consume the CPU CPU of your order service. Whenever we are deploying order service the CPU will be consumed.

[00:03:50]So again those resources will be consumed. Now imagine you are getting thousand request and you are retrying three times 3,000 request unnecessarily you are sending to inventory service which does not really makes any sense.

[00:04:03]So that is basically the problem. That is basically the problem. If the service itself is down we are sending the unnecessary request and all those requests are going to fail.

[00:04:15]So what do we want? We want a mechanism to know that inventory service is down.

[00:04:22]If order service knows that due inventory service is down then order service can just go ahead and stop sending all these request to inventory service. So all these request will not be sent in the first place saving the resources the energy the time inside this order service. So something like that needs to be done. Now who is going to tell this order service that dude inventory service is down? Who is going to tell this guy? There is no third person over here who is going to tell this order service that dude this inventory service is down. That works needs to be done by order service itself. Now how order service can do? If I want to design this solution, how I can do that? So let me just pull this guy a bit down over here. Now what I can do I can just simply go ahead and add a utility inside order service let's say and what that utility will do that utility will keep on checking the calls to our friend inventory service. So it will check the calls to inventory service and it will check the calls let's say in a 10-second window. So it will just keep on checking the calls and it will check how many calls within this 10 seconds are failing. So if we are sending let's say 10 calls inside 10 seconds then it will check how many calls are failing. Let's say five calls are failing. Now what we can do now in this utility we can just go ahead and tell this guy that dude if 50% calls are failing if let's say 50% calls are failing 50% calls if they are failing what we need to do we need to stop making calls to inventory service and we need to handle it inside order service let's say 50% calls inside 10 seconds are failing then we can assume that inventory service is down or there is a issue with inventory service because major calls are failing and we need to stop sending calls to inventory service and we will stop these calls for next let's say 10 seconds for next 10 seconds stop sending call don't send any calls to your inventory service don't send any calls up to next 10 seconds and after 10 seconds we will again re-evaluate probably we can send let's say five calls again and see if they are working if these calls are now working then we can assume that inventory service is back and up and running then we can go ahead and send all the calls to order service and again keep on monitoring this cycle. So this utility will keep on doing all this. So this is how I would design a solution. I can just go ahead and add a utility inside this particular order service. But again Java people knows about this issue already. They know about what exactly circuit breaker pattern. So they have added this utility already. They have added this utility and that utility we call as circuit breaker. And now in order to solve this problem, this particular utility or we call it as circuit breaker comes into picture. So we have seen the problem right? Now we'll see the definition of circuit breaker. Now circuit breaker stops sending requests to an unhealthy service after repeated failures. Very very easy stuff. Very easy stuff, right?

[00:07:42]Circuit breaker stops sending requests to an unhealthy service after repeated failures. That means in 10 seconds 10 calls we are sending five calls failed, 50% failed then stop sending the request to inventory service considering it is down. Very very simple stuff. And that is basically the definition of circuit breaker. So the circuit breaker definition we have seen. Now again why do we need circuit breaker? So first of all why do we need circuit breaker? As we have seen over here, in order to save resources, right? All these resources we want to save, there is no point in calling an unhealthy service repeatedly. So, in order to save resources, I'll just make use of this over here. Save resources first thing. After that, let's say if you don't do this, then inventory service goes down, it may go ahead and take down order service as well. So this is basically very basic of your resilience. That is something we call as cascading failure. So to avoid cascading failure, prevent cascading failure, we do this. After that by using circuit breaker, we do faster failure detection.

[00:08:53]We detect that this particular service is down and we fall back. We make use of a fallback mechanism instead of sending the request to inventory service. How cool is that? So that is again another reason why we do why we need circuit breaker. After that we want to protect downstream services. We want to protect this service from going down because of this service is down. So that is another reason it is similar to preventing cascading failures. And at the end we want to improve the system stability. We want to improve system resilience fall tolerance. That is the reason we need circuit breaker as well. Simple stuff basically. That is why we need circuit breaker. So now we have seen our simple solution. But let's see how circuit breaker solves the problem and that is when something called as circuit breaker states comes into picture. So we need to understand different states inside circuit breaker. It's very simple. So if I go back over here then this is basically how it looks. Now here if you see here majorly you see three states.

[00:09:59]First is closed, second is open and third is half open. Now it is pretty similar to your electricity analogy.

[00:10:08]That means if you connect two wires that means if you close two wires that means the current is flowing that means your services are communicating. Simple stuff. Closed means your circuit is closed that means it is flowing the current is flowing that means your requests are going through. Open means circuit is open. Now if this particular circuit is open how the requests are going to flow that means we are stopping the operation. It is also marked as red color marked in red color as well. That means open means when your circuit is in open state that means the operations are stopped the request are not going to your other service the communication is stopped. And what is the halfopen state?

[00:10:52]Half open state is basically when your service when your order service is checking that inventory services came back or not. That means it is just trying to send let's say two three request and checking if it is back or not. That is basically the state we call as halfopen state. Now what we will do?

[00:11:13]Let's say everything is working just fine. The circuit is closed. Now we will check if in the 10 seconds 10 calls we send 10 calls and we will check the threshold. Let's say 50% is the threshold. If 50% threshold limit is reached over here then we will move to open state. Then the circuit will be moved from closed state to open state.

[00:11:38]After that after some amount of time it will go ahead inside a halfopen state and it will check if the inventory service is back up by sending few request. If the request again fail it will again go back to open state. But if the request pass request successful with threshold limit then your circuit will go back to your closed state. That means the requests are flowing again. So very very simple stuff. Basically if you understand this diagram you'll be able to understand it very easily. So that is basically different states inside your circuit circuit breaker states. So we have seen that one as well. Simple stuff. Now let's jump into implementation configuration and demo part. Now if you remember we have these three services. First we have Ureka server which is basically running. We have inventory service which is again running. After that what do we have? We have ecom order service which is again running. If you see over here, all the services are basically up and running.

[00:12:38]And if you see over here, we have this inventory service which is sending the request to your inventory ecom inventory service by using a fain client. So our inventory client is basically fain client and we are trying to get the inventory from here. Right? We have seen this multiple times. And in the same example, we have seen rate limiter. We have also seen retriable. So all these things we have seen in the last videos.

[00:13:02]So probably what I will do I'll just go ahead and comment these guys for now so that we can just go ahead and focus on our friend circuit breaker. We have a simple get inventory and fallback method but we are not using that. Let's see over here. Now let's go back over here to ecom inventory service. Now here if you see our ecom inventory service is running. So what we can do probably we can check the scenario where our friend ecom inventory service is completely down. Before that what we need to do is we need to implement circuit breaker and the implementation of circuit breaker is pretty similar to rate limiter. Circuit breaker is coming from the same library.

[00:13:41]Which library? This particular library which is resilience 4j. So if I go back to pom.xml then you need to add this particular resilience 4j dependency.

[00:13:51]Once you add that along with this aop once you add that you'll be able to access rate limiter as well as circuit breaker. Now how you can implement circuit breaker you can just say let me just press enter over here I can just say circuit breaker from resilience 4j I will just give it some name and fallback method here I will just give it a unique name let's say inventory service circuit breaker so that is basically circuit breaker and circuit breaker fallback method right so that is basically the name that we give over here and we'll just replace this name over here as well and that's That is basically simply how we implement circuit breaker. Now what I will do I'll just go ahead and restart this particular order service. Let's see what happens by this much of changes. We have not done any configurations yet. We are just adding a simple annotation and let me see over here. So our application is back up. Now let me try to send this particular request. What we are getting?

[00:14:56]We are getting product out of stock.

[00:14:59]Perhaps what I will do, I will just go ahead and fill up some stock. So I'll just add 50 inside our stock. And now I will just go back and send some order.

[00:15:09]So order will be placed. I will send it again. And if you I go back over here and try to see if any fallback was called. So fall back method I will just go ahead and check.

[00:15:23]So you will see that there is no fall back called. That means everything is just working fine because our friend is fine. Let me just stop order service over here. So I will just go ahead and stop this guy. So if you see inventory service is basically stopped now. Now if I try to send this request you will see that we are going inside this particular circuit breaker fallback method. Now what I will do? I'll just mute it and send. Let's see what happens. Now if I try to search this method then you will see that there is this particular one fall back. Now here we have not defined anything. We have not defined after how many time we need to retry. So now what I will do? I'll probably go ahead and send another request. Let's see what happens. It is sending some internal server error. And now you will see that our fallback method was called again. So that is how your circuit breaker will work. But in order to fine-tune, in order to configure what exactly is needed for your service, we need to add configuration. So that is when this particular configuration comes into picture. Now let's go back to application YAML. Now here if you remember we have done the configuration for this particular rate limiter. Now in similar manner we will need to do configuration for circuit breaker. Now here if you see after rate limiter I have added circuit breaker and we have added inventory circuit breaker or what was the name that we have given over here. So let me just copy that. This name should match over here. So this particular name of our circuit breaker should match with this configuration name over here. Now first we will have sliding window type and the value is count based. Now there are two type of sliding window types. One is time based other is count based. Count based will work on count and time based will work on duration. Basically simple stuff.

[00:17:11]After that once you give count base you need to give sliding window size. That means it is going to check last 10 calls. How many calls in last 10 calls check? So what I will do? I'll probably go ahead and give it a value as six.

[00:17:26]After that minimum number of calls it is it is very important. Now these are the minimum number of calls until which your circuit breaker will not calculate failures. So circuit breaker won't calculate any failures until five calls.

[00:17:40]Now this is a safety check because let's say your failure threshold is 50%. But inside this particular window there are only two calls and one of them is failed. Now that becomes 50%. Now only on failure of one call we need not go ahead and open the circuit. So that is basically the problem. And in order to prevent that minimum number of calls we can give. So until five calls this guy will not calculate anything. What I'll do? I'll just keep it one over here.

[00:18:11]After that failure rate threshold let's keep it. Let's keep it 50% probably. So that is basically the failure threshold.

[00:18:19]After that what do we have? We have wait duration in open state. That means once the circuit is in open state for 10 seconds we do not allow any request to go to inventory service. After that permitted number of calls in halfopen state. Now we have halfopen state.

[00:18:37]Right? Now this is the configuration for this particular halfopen state. Now for this halfopen state permitted number of calls three maximum weight duration is 5 second and automatic transition from open to half open is true. That means after 10 seconds circuit will be transferred to halfopen state and this guy will check if within 5 seconds three requests are passing. Once those are passing what this guy will do it will move the circuit to closed state again.

[00:19:07]So that is basically how it works. Now what we can do let me just go ahead and start our friend restart our friend order service with this much of configuration. Let's see what happens now. Now here if you see our service is up and running and what I will do now I will just go ahead and so our inventory service was again down. So I will just start it and let's see what happens. So what I will do I will just go ahead and add some inventory first. So this inventory is added. So now if I send this then you will see order is placed successfully. Now what I will do I will just go ahead and stop this particular inventory service. Right now let's stop this inventory service and let's see how many calls. Now I can just go ahead and probably send few calls from here. So five six calls I sent. And now you can see over here if I try to look for fall back method called then let's see how many times it was called. It was called many times basically that means our circuit is basically open state in open state. And now what I will do I will just go ahead and start our friend inventory service. Now what I will do I will just go back and try to send it.

[00:20:26]Now you'll see product is out of stock.

[00:20:28]That that is because we need to add this particular inventory. Even if I have added the inventory, it is still saying that product is out of stock. That means it is going inside this particular call back method. It is still going inside this particular power back method even if my inventory service is back. So it will do it for this particular amount of time. How much? It will do it for 10 seconds. And now if I send it or let me try to send it. And now if you see the order is placed successfully. Now the circuit is closed again. So now if you see if I keep on sending request all the request will be successful because our friend has moved the circuit in closed state. So that is basically how your circuit breaker pattern works. It is pretty tricky to demo it because it everything depends on this particular second and stuff. So that is basically how your circuit breaker pattern works.

[00:21:24]We have seen the implementation and configuration and we have seen the demo part as well. Simple stuff basically again as we have seen multiple times.

[00:21:33]Again as we have seen inside retry and as we have seen inside rate limiter as well it works on a proxy based approach.

[00:21:40]So it will have a spring proxy.

[00:21:43]So probably what I'll do I'll stop inventory service now and let me just open this and let me send some request.

[00:21:53]So here if you see here if you see do you see this this is basically the proxy. So order controller order service inventory service and we have this proxy over here circuit tracker format method which is present inside a proxy inventory service cg proxy which is handling all this. You will see these hidden frames which will have a lot of abstract methods which will handle this circuit breaker for you. So all these details that we have discussed. So if I go back over here, all these details inside the utility that we were trying to come up all these utility related details will be available inside this particular 25 hidden frames which are basically the internal methods which are supported by this particular proxy and used by this particular proxy. So that is basically how it works on your proxy based approach inside spring AOP. It is pretty similar to what we have seen. Now another configuration we have over here is basically the type of exception that means on which type of exception which exception type you want to invoke circuit breaker that means record exceptions that means if runtime exception is coming from inventory service then invoke your circuit breaker or if IO exception is coming or or HTTP server exception is coming then invoke this particular circuit breaker but If illegal argument exception is coming, ignore this particular circuit breaker. So if this particular exception is coming, don't stop inside circuit breaker. Don't move to open state. So those exceptions also you can make use of. So this is basically another configuration which you can make use of. So that is basically your circuit breaker. So let me just go back. So we have seen everything over here. So circuit breaker is basically very very simple pattern.

[00:23:44]You can just go ahead and try it on your own and see how it basically works. So that is basically it for this particular video. We have seen everything and that covers circuit breaker. Now in subsequent videos what we need to cover.

[00:23:56]So here if you see we have seen retra pattern, we have seen circuit breaker, we have seen rate limiter as well. Now bulkhead, fallback and time limiter is something that we need to see. While looking into other patterns, we have seen fallback pattern. But time limiter and bulkhead pattern are important patterns which we will look into. That is basically the agenda for future videos in this particular playlist. So that's it for this video. If you like the video, hit the like button. Don't forget to subscribe to code snippet.

[00:24:24]Share this video with your friends so that they also have idea about circuit breaker pattern inside resilience 4j.

[00:24:30]That's it for this video. See you in the next video.