GDPAI Lect4 Inclusive and Diverse AI System Design & Ethical AI Development Practices

Added: 2026-05-18

[00:00:00]And welcome to this next lecture.

[00:00:03]Lecture four. And So in this lecture, in the previous what we saw that what is bias and how to detect bias. And what are the different way we can mitigate bias. Now the question comes that we have built a AI system. Now we need to find out who gets left out when we have already designed an AI system for everyone.

[00:00:30]And let's say we have designed AI system for whole Norway or whole India or for the whole world. Did we miss anyone? Now inclusivity and ethical AI design we should question that who is centered and who has been ignored while designing the system. So now many AI systems are built for the average users. Means most of the time the commercial entity they're building the AI system. For example, uh if someone is paying, let's say commercial entity, but they're building system who is paying.

[00:01:09]Now if their paying audience belong to a certain income strata so they will be typically be building for that income strata.

[00:01:19]And or average users who are going to using not outliers, they're often ignoring it. But in real society the are not average. People differ in language culture, disability, different gender, age.

[00:01:40]They have different kind of life experiences. So how do we design an AI that works fairly across that kind of diversity.

[00:01:53]So that's bring to this lecture that inclusive and diverse AI system design and ethical AI development practice.

[00:02:01]So let's begin with an example.

[00:02:04]Why does it matter?

[00:02:08]We can see that AI system are getting embedded in everyday life. Whether you take it a hiring decision immigration decision. Now the immigration decision uh someone applied for visa and the visa is rejected.

[00:02:25]That's fairly fine. At least it is not a justice for that but person. But what if the visa is granted for some malicious intent person?

[00:02:37]And that person enter the country and it does harm to the society.

[00:02:43]So uh it is a bigger impact. The missing the person is a bigger impact. Now AI systems are getting used health care for credit scoring.

[00:02:55]Now if we do not do inclusivity and adding the ethics then real world harm can be done.

[00:03:03]So our sis challenge is that can be built an AI system which serves everyone and humanity.

[00:03:11]So we saw that face recognition system can can it detect different kind of faces?

[00:03:19]Can it uh detect work for different types of thing?

[00:03:24]So AI systems are no longer experimental tool. Means before 10 years the AI system used to be experimental tool. Mostly it was in a research domain. Some of the AI tools were getting used for real society, but nowadays it is everywhere.

[00:03:42]So in this present time the embedded decision in the AI system can affect livelihood freedom, health and access of very various people. So application of AI is there in education sector and many sector.

[00:03:59]So bottom line is that if inclusions and ethics are missing then harm will be evitable invisible and it will be systematic and scalable.

[00:04:15]So in this lecture what you're going to learn today is that what an inclusive AI design mean what ethical AI development mean, most common barriers to inclusion and practical strategies for building more responsible AI system.

[00:04:34]We talked a lot about that.

[00:04:37]Let's have a quick reality check.

[00:04:40]Some AI system have excluded or some kind of disadvantaged users.

[00:04:48]Others have improved access and inclusion. The difference often begin in design choices. Means we have a built a system which by default excluded some kind of users. We have already a system we have improved access and inclusion.

[00:05:05]But the difference which was been in built from the beginning it is going to be there always because differences we can see.

[00:05:16]So AI system can either widen inequality or reduce the barrier. It depend on how we have imagined while building the system we have built the system, we have tested the system and after that we have governed the system.

[00:05:34]So what is an inclusive AI design?

[00:05:38]Inclusive AI design means that we are building a system that account for different user groups and their experiences.

[00:05:47]It addresses also the historical data biases.

[00:05:52]And it include the underrepresented community in the design process of system itself.

[00:06:00]Now inclusive design is not only about fixing the model like we were talking about during the training process or after the deployment.

[00:06:09]It is about deciding from the start. Means it is easy that once the training process starts data collection we remove it. Uh we in the training time we do the adjustment in adversarial learning etc. Post processing we use some sort of threshold to remove the bias decision.

[00:06:32]But it is better to start from the beginning.

[00:06:36]For whom we are building the system where we should start understanding the system from the beginning.

[00:06:42]So that lead to uh what exclusion can look like. If we think I'm just uh example is that uh a speech system that fail on non-standard accent. For example uh we have built a system for a uh Norway itself. Now Norway has many types of accent. And now there are many um immigration immigrant people are there.

[00:07:15]Their accent will have some inbuilt accent of their own home language or their own mother tongue.

[00:07:22]So that kind of non-standard accents uh system is failing.

[00:07:27]Now we have built a face analysis tool and this face analysis tool is performing not good for certain type of skin tones.

[00:07:37]Uh we have built an interface that ignore people with disability or access need. So these are the things we need to think that system might be working fairly overall but it it is failing for the groups who were not meaningfully considered during the design and testing.

[00:08:00]So common barriers to inclusion is that we have had a biased training data then we have a very homogeneous design team.

[00:08:09]We have built a system which uh one size fits all without user experience assumption.

[00:08:16]We have lack of accessible testing environment.

[00:08:20]So that leads to the question that what is an ethical AI development?

[00:08:26]Now ethical AI development means making a design choices. Means see in the previous bias in the training data, homogeneous design team one size fits all, these are all uh can come into when the team which is building this AI system uh they have to think that are they building an ethical uh process doing the AI development? So ethical AI development team means we have to make design choices which is guided by justice.

[00:09:04]Are we doing justice? We are following the rules and regulation and laws, accountability.

[00:09:10]We cannot simply say that oh I built a system and for example the people who are building the foundational model. I have big objection to them.

[00:09:20]They they're building the foundational model. Now the they say that we are building the model how it is getting used right way or wrong way. Uh it depend on it it belong to them who are finally using it commercially to get it to the end users.

[00:09:37]But come on, you are building this foundational model, you have the responsibility. So you should also be accountable. But currently this uh foundational model companies are so big that uh I'm surprised and very disheartened that no country and no government is bringing them into and hope in the future they are bring they bring them into accountable things.

[00:10:04]It's not that okay, I have built something and someone is using it. It depend on them. Come on, you are making the money when someone is using it. So, you are making the profit. So, the responsibility does goes to you.

[00:10:17]Now, the transparency. The transparency system should be transparent when we are building it everything should be there.

[00:10:24]And responsibility. So, it is we have the UAI Act.

[00:10:30]And UAI Act when we are making an AI based system it ask for compliance. Compliance says what is allowed.

[00:10:39]But, we can make a system which is UAI compliant. But, if we simply make a system to get past the compliance where is the ethics? Ethics guide us what is right.

[00:10:53]So, we should do what is right rather than just sticking to compliance and going forward. Now, uh AI team which are ethical should ask these five questions.

[00:11:07]That fairness.

[00:11:09]Is the system is putting some group into disadvantages?

[00:11:14]Transparency.

[00:11:16]Can important decision be explained? Means when AI system is giving some inferencing can important decision explainability is there that why it is taking?

[00:11:29]Accountability. Who is responsible when harm occurs?

[00:11:33]So, in my view that everyone who's in the AI pipeline and everything used in the AI pipeline and who are making profit out of it should be responsible if harm occurs.

[00:11:48]And one group cannot say that I am not there and specially the foundation model companies.

[00:11:55]They are the biggest gainers of this AI boom.

[00:11:59]Trillions of trillions of dollars they have become and they are not putting themselves accountable. Which should not be there.

[00:12:07]That is my opinion.

[00:12:09]Um I hope in the future they are bringing to accountability.

[00:12:13]Or they themselves become ethical enough to be make themselves accountable.

[00:12:18]Privacy. So, is the data protected? So, in EU if you are building a system, you are governed by GDPR law. So, anyway have to build a uh data protection.

[00:12:30]But, even not if you are not building even for you, if you are being building for other places and those country has not strict GDPR law or privacy law we should build an ethical AI team should build a system which is uh pri- which adhering to the privacy. So, if in in absence of any kind of laws exist in other country why don't we adhere to the GDPR law?

[00:12:56]Now, inclusivity.

[00:12:58]Where the diverse community has been considered or not. Now, with this ethical principle become more useful when translated into the design process.

[00:13:09]So, we should think uh if we want to take into a nutshell uh the final thing where the AI is there is in the center of the thing. So, what should a team actually do?

[00:13:27]Inclusion and ethics must be in operation, not just declared. Means an AI team should not only declare that oh, we are following the ethical standard or something.

[00:13:39]But, that should be in operation, not on papers. Sometime people are putting on papers and they are not following.

[00:13:50]So, strategies.

[00:13:52]First strategy is that inclusive data sourcing.

[00:13:56]We should build a better data.

[00:13:58]Uh source data from various data various groups for which we are targeting so inclusively.

[00:14:05]Check for gap between different representation.

[00:14:09]Document who may be missing. Let's say we tried all our way to collect the data.

[00:14:16]And we still not able to get. Sometime it happen that some representing group is not in the access. We trying to we and we know we need to include it. But, we do not have it.

[00:14:26]But, in that scenario we should document what is missing so that we have a transparent system.

[00:14:33]Then after that we should re- review the labels for any kind of bias getting introduced while labeling the system.

[00:14:41]So, the next strategy is auditing the model for bias. So, audit of the model uh we should do not for just accuracy, but in general.

[00:14:53]Now, we need to also compare the performance across for various groups our model is working.

[00:15:00]Use fairness metric, identify unequal errors patterns and examine unintended consequences. So, that need to be done.

[00:15:10]A highly accurate model can still be unfair if its errors are not evenly distributed. So, if you do not have equal error rate kind of system across the group, then it can be accurate to one class of group, but it can be more erroneous to other class of group.

[00:15:29]For example, a system can be 99% accurate.

[00:15:35]Now, 1% inaccuracy so, 99% accuracy coming from 100% accuracy for one group and 1% inaccuracy coming from 100% inaccuracy to other group. So, it can happen.

[00:15:50]Strategy three. This is the human in loop oversight. This is as per UAI guideline that human in the loop must be there.

[00:15:58]So, you has given the different uh category of AI model 1 2 3 4 5 and human in the loop has been included in the more critical category of AI model.

[00:16:14]It's better to keep human meaningfully involved. So, human review in a high stake decision models or let's say we built a diagnostic system. It's better to give the decision making power to medical professional. That yeah, AI model can say that I probably see some error, but final decision should be in the hand of doctors.

[00:16:39]Escalating for uncertain cases. So, escalation if uncertain cases comes, we should have a uh zero tolerance policy that we should immediately address it.

[00:16:50]Override mechanism. So, human in the loop can override if anything wrong wrong happen. For example, in the UK immigration visa making where AI system they are using.

[00:17:02]They do have human in place to finally sign it whether the decision has been done or not correctly.

[00:17:10]I have come to know that even having human in the loop the decision have been made incorrectly. With the same type of document some people have given the visa and others have not been given the visa.

[00:17:25]So, a human override mechanism if should be there or human in the loop is there uh it could have corrected better way if that human in the loop have done the job correctly.

[00:17:37]Then accountability in the final decision or found for final decision should be there. So, now human in the loop should not mean we should not mean that humans are present only for appearances.

[00:17:52]Just like in the UK visa immigration system the human was there only for appearances.

[00:17:59]While what it means that human should be there for meaningful oversight and responsibility.

[00:18:09]Next strategy is diverse testing and research panel. So, we should test with the diverse user.

[00:18:16]When we have built an AI system, this system should be test with a diverse set of user.

[00:18:23]So, by including the demography different demography, by including the different accessibility needs like people with red green color blindness, people with blindness or partial blindness people with hearing problem and other kind of accessibility issues.

[00:18:43]Now, we should also include the cultural and linguistic diversity into the system. That the system we should not be building let's say we built a AI model which only support New Norsk and it not it does not New Norsk and Bokmål.

[00:18:59]It does not supporting the people with Sami accent. So, that is going to be a trouble.

[00:19:05]We should also include the users who are at the margin, not at the center. Means there are in the middle lot of people are there, but there are also outliers in this end of the tail and the lower end of the tail. So, we should include those users as well.

[00:19:26]So, if a system is tested only for very average cases so, in that kind of system we should not be surprised when it start failing for others. So, if it is working for average people which typically a trouble of a deep learning model then if non average data comes, then it if it fails, then we should not be surprised because we have not built a system to work for everyone.

[00:19:54]Now finally the impact analysis.

[00:19:58]So we should evaluate the real world impact of our AI system by measuring the fairness before the deployment like some pilot study monitoring after the deployment so continuous editing look for harm if it is happening not just in theory we just say that okay it is not we should look for what is a harm happening when it is getting used.

[00:20:25]And update the system if any inequality or inequities appear.

[00:20:30]So model updating should be there when impact comes and we are we are able to mitigate it so update the system when inequity comes. Uh we should take into account.

[00:20:43]So in a nutshell five practical strategies that can work is build a better data audit for bias keep human accountable and test with diverse users monitor real world impact. Now design strategies that work I have mentioned now if you follow this pathway start an ethical AI development uh include global standards like EU USA China or other country define ethical principles then implement guidelines for data use and AI training uh conduct regular audits then adjust to the standard and also the when the standard changes uh ensure the global and local accountability um in decision making and then that's how the ethical AI can be achieved.

[00:21:44]And while design strategy we should start working in the beginning identify what is the objective then in the collection we should adhere we can adhere to the EU AI Act in the data processing we can take the biasness correction into account model development US AI principle can be added in the deployment China's AI ethics can be added China has also given this AI ethics and then by monitoring and feedback loop we can maintain. So now we all industry follow this kind of DevOps pipeline right?

[00:22:23]So which is also ML Ops pipeline machine learning operation. So we should have this kind of principle in each and every stage of this development and operational pipeline like planning then collection and this whole steps we should have the ethical principle bias identification mitigation should be there.

[00:22:50]Now comes the case studies. Okay.

[00:22:53]So we will see some case study where what good and bad design look like.

[00:22:57]Okay.

[00:22:58]So positive design and negative design where this isn't isn't not happening correctly. Positive the design has been done correctly and inclusion lesson.

[00:23:09]So So here three cases where Seeing AI is a system by Microsoft.

[00:23:20]What they have developed that they have developed a Seeing AI which can assist people with the disability and as well as people with the not disability so they have taken into account that they can hear it crosswalk then it can also detect currency and tell that this is a $10 bill $5 bill $20 bill so they have included assistive inclusion accessibility first thinking and that's the quite a positive design.

[00:23:55]Now Amazon hiring algorithm for resume screening that has of biases because it has been built based on historical data.

[00:24:05]So historical bias has been included as got included in the recruitment process.

[00:24:12]Now the AV recognition improvement are there which uh is there in the system where people can speak I was just fine um catch the bus then it it converted into I was just fine I catch the bus.

[00:24:32]So better performance can be included by including the linguistic accent.

[00:24:38]Uh so this way we can have the better system now.

[00:24:44]What do these case studies teach us? For example in Seeing AI what we saw is that it include assistive inclusion and accessibility first thinking while designing the system. While Amazon hiring it has been based on historical bias and the bias has been reproduced in the recruitment.

[00:25:03]While AV recognition improvement uh better performance it is giving better performance by including the different types of linguistic uh variation.

[00:25:15]So now we come to a point where the key takeaway from our lecture four is that inclusive AI design start with representation and participation ethical AI development goes beyond compliance means a system can be compliant but if you have not included ethical aspect so we won't be saying. So typically we should have an ethical AI development which also adhere to the compliance not that it is a compliant but not ethical.

[00:25:50]Okay.

[00:25:51]Real barriers for building the ethical AI system is the data team inter interfaces testing practices and to make a responsible AI development of team it requires a practical strategies not only that we are adhering to the respon we are responsible AI team we should actually include it and do it rather than some just mentioning on papers.

[00:26:18]So you can read more about this topic if you uh want to look into the book and I hope you enjoyed the lecture and see you for the next lecture. Thank you.