Working with AI Research Agents to Engage with Academic Literature

本站添加: 2026-05-07

[00:00:02]Hey everyone, so great to see you here.

[00:00:05]So I just wanted to introduce myself.

[00:00:06]I'm a professor at war business school and also at Harvard Dube affiliated with it and the AI is excited to be collaborating with Ibraat and new scholars and OMT the organizational management division of course management theory division of academy of management one of the largest and most important I would say divisions in academy that is strongly supporting the thinking about the use of AI for research. So there will be a series of those and collaboration between kind of those three entities on how we should think about AI. So we in AI the artificial intelligence innovation network we really try to think about the use of AI in two ways. first of all how it impacts the world and you know the field and we engage with practitioners as well to see what are the important questions to ask and what are the most relevant but also on ourselves and that's today that's the beauty of talking about what does AI mean for ourselves so this is something that Ibra has been leading the thinking about and truly I can't wait to hear what he has to offer us so I will put in the chat the email if anyone wants to join in any way for more of those things that we will do thank you so much Ibbert Thank you, Hila. Sh over to you.

[00:01:28]>> All right. Uh, thank you Hila. Thank you Brad and and good uh morning, afternoon and evening everyone. Uh, my name is Sheila Ling and I just graduated from Imperial College London and will be joining SCMA this fall. Uh I am really excited to share uh what I've been experimenting with uh engaging with AI in research uh in my own thesis and and and the research afterwards. Uh so for those who had signed up very early on uh the session was originally called working with uh AI research agents to engage with academic literature. uh we will get there but as you can see I as I was preparing for this session um I realized that the most important question isn't really which agents to use as you probably see in the description we talked about clock code um but it's really about how to think with them so we have widened the lens a little bit so by the end you will have seen the agents and and the practices that make them actually useful in research practices this um so okay so to start uh for those who have uh joined uh me and Karen Quley last time you've probably seen some of this but uh let me just recap uh really quick here so most of conversations we have been hearing about AI in research starts here right we talk about application uh just let AI do the work work automate everything uh and it can be efficient but epistemically hollow.

[00:03:10]Uh and on the other side we are also seeing the abnagation you know uh some people uh rightfully refuse to engage that they want to wait it out and that's safe but increasingly untenable uh and what we are offering uh is a path between so to start I want to offer three orienting principles uh to guide us through this journey. Uh the first is how we think through right AI as a medium for developing understanding not just a machine that automates it for you. Uh the second thing is how we design this engagement. Uh so we are looking at a skill or knowledge in how we can structure the interactions between human researchers and AI systems. not just which button you press or which tool you use or which model you you you pick and lastly it's uh the inward lens uh because often times we see the output can disappoint uh and I would encourage us to first exam examine our input first uh so briefly on this and we will come back to them um and oh yes and uh based on your uh feedback in the chat room. So feel free to let me know if you would like me to kind of um uh stop in between to take some Q&A. So happy to do it or we can do everything at the end. Uh so please please let Ibra know if you have any preferences. So okay. So for this uh session I I picked uh uh mock research question to to guide us through this journey specifically to to turn the lens uh back on ourselves.

[00:05:01]Right? So when management researchers run audits, evaluations or experiments on large language models, what methods do we use and what kind of claims do we make? And we will leverage AI to interrogate how our field has engaged with AI uh and and kind of point out this reflexive move.

[00:05:23]And so for this we will work through three configurations and I want to point out that uh we're not calling them levels uh as uh the practices and principles and arguably the the return could be similar uh but they are just different ways to think about how we can engage with AI. Um and so the first type or configuration will be the conversational interfaces that we are familiar with. Uh and the second one is deep research which actually is embedded in those interfaces but they have more things happening in the background. And lastly we will briefly touch on uh agentic workflow and foreshadow what might come afterwards in this series of uh talks on new scholarly.

[00:06:14]So okay so configuration one uh how do we have this socratic conversation uh and thinking with AI as a colleague uh and you know I had a few papers that I shared with cloud and started you know thinking okay what's interesting here and this is the initial question that came up you know it took probably I would say five different terms uh to come up with this prompt because I I do want to explore the literature and understand more. So the act of writing this prompt forced me to know what I was actually looking for. And to me the clarity achieved by writing this is a prompt uh is a insight here. And so I took this and you know ran it through KIGPT and cloud and got kind of a interesting outputs very competent I would say but not quite what I was looking for. So I started taking a step back and look at what I was really asking in that prompt.

[00:07:32]Right? So uh I actually took the reports and take it back to cloud and said hey look read all these transcripts u and why is it isn't it getting what I'm actually trying to understand. Um, I'm not trying to argue whether the answer is right or wrong here with Claude. Really, it's just to bring the transcript to a new conversation and figure out what I was missing when I was talking to the other cloud when I was trying to write up this prompt. So this is something that I have found uh quite useful and honestly I I have encouraged many many others to do it and they have found it useful as well is to take the transcript as almost like an artifact that you can analyze. Uh so for I guess for qualitative researchers uh this is kind of like uh memoing uh but at this uh in this context you're doing it together with the AI.

[00:08:41]So what happened next is later I realized well what the prompt asks was what methods researchers use all large language models right all these evaluation experiments. So it became about a synthesis of of the literature about using large language models in research. What I actually wanted is how researchers treat large language models as theoretical objects. You know how how do they evalu how how do they approach evaluative cognition compliance mode fairness reasoning? Uh so it's I was looking for something more like a literature about what large group models are like uh in the eyes of management researchers. Uh so notice this gap you know it came out of this prolonged discussion uh with cloud after I bringing those transcripts and those reports uh from the initial framing.

[00:09:38]So we refined it uh and the question became when management researchers treat large language models as theoretical objects of study making claims about what they are like as evaluators, decision makers, reasoners or cognitive entities. What evidence do we use to support those claims? Right? Um so here I want to point out uh the principles three here the inward lens uh again it's very easy to blame the model you know they're not smart enough they're they're biased but what I realized was you know whether the output mirror the input usually that's the case and that helped us to notice what the frame was and how we can improve it.

[00:10:29]And so this is a screenshot of the conversation I had with clown. Uh it's very long. So I I just share the last bit of it. Uh and this is CHPT. And as um many of you who are familiar with treasurept, it likes tables. So here I'm using uh the extended thinking model of treasurept 5.5.

[00:10:54]uh very nice but uh still a bit shallow uh given this is just a oneshot conversation and this one is by Kimmy 2.6 6 thinking model that just came out last week.

[00:11:11]And so with this, you know, we have something going on. We have three slightly different reads, some emerging patterns, but it's not really done. Uh so what the agents gave us uh through this uh these three conversation was you know there are different research postures there are different behavioral evidence uh and there are some critical counter current surfacing uh so again engaging with cloud and me also reading the research reports generated um so I was able to figure out okay where do I need to push on um so we have the map but we don't really understand what are the mechanisms that are in those management research studies uh and that brings us to the next configuration um and to echo back what I said earlier right the transcript is the work um we need to articulate what we actually and be proactive in steering the conversation. Um, be patient, you know, do diagnostic turns and be curious about how to reframe things. Um, and also notice uh what may be hidden, right? uh especially in conversational interactions uh it's hard to trust the outpost right how do you verify the underlying claims uh we know that a lot of times this AI only read the title and the abstract or what other people's have written about those studies so it requires us to going back to the actual paper um so think of it as the conversations we have had so far only surfacing the names uh uh and to verify we need to still do the reading. Um and later on we can also touch on how we can engage AI agents uh to participate in this reading together. Um and also is that from this conversational inter uh interactions we get to get a sense of the emerging criteria we are looking for right it can be about journals it can about keywords and any other inclusion rules you might have uh for your particular research question um and also especially with the prompt we ran um we are implicitly looking across cross disciplines. So we need to be careful about what different disciplines may use different phrases or theoretical constructs to talk about the same thing.

[00:14:07]While AI can be great at understanding these nuances uh but oftentimes it requires us to spell those out. Uh so we actually uh clarify the boundaries or the scope we're looking at. Um and also uh as you might have noticed we jumped around different sessions with different models, right? Um so often times it's important to ensure this cross- session continuity and again the artifact of of a memo of a reflecting node can be really helpful here.

[00:14:44]Okay, so let's move on to configuration two. Uh unless sorry we can we can we can stop and take some questions.

[00:14:55]>> Yeah we have uh one hand raised. U Malikica do you want to go ahead?

[00:15:00]>> Sure. Thank you for the great presentation so far. I have one uh experience that I want your sort of feedback on. One of the frustrating things for me is that when I'm doing you know this uh you know this literature reviews or any other kind of tasks my experience is that you know 90% of the content is correct but then there are 10% things that are that are made up completely made up it's saying oh this is what's been argued but that does not exist and and and because it was a you know it was a lot of papers that initially you know I read each of those papers individually before using the model. I knew that this was now not said by anybody anywhere and so I could easily you know take that out but for a you know bunch of literature like you know for like this session I would not know the content of each and every single paper in detail and so this 10% hallucination is quite a lot and and so I was very concerned uh that oh my god you know so when it you know presented maybe there hopefully Hopefully there will be some audience who said look this this has never been said by anybody but I have to rely on myself. So what are some of the strategies you know or what's your experience and what strategies do you use for papers that you may not be 100% read yourself or may have forgotten maybe read it in grad school and just don't have it in your memory. So I'll stop there.

[00:16:38]>> Sure. Um may I may I ask a follow-up question so I can answer better. So can you talk about what uh which AI system were you using and how many papers you were looking at?

[00:16:51]>> So so so I use uh two primary uh tools um cloud code and uh chat GPT.

[00:17:01]So so so basically initially I used cloud code you know and because it's it's I find it easier to use and it's very good frankly. So I used it but then you know it gave me um gave me a lot of missing answers like so a lot of papers he did not basically summarize so that oh there that you know the tool gets lazy so I gave it around 50 articles I mean it's not I mean articles you know not book chapters it's just literally PDF files and I would say it did around 30 very well but after that it got lazy.

[00:17:41]So I could see that the tool got very lazy. So it didn't want to do the work anymore essentially like a human agent.

[00:17:48]And so so I then you know I kind of got whatever it had it's all in the same folder. Then I moved to codeex and I said chat GPT because those papers I know were correct but these 20 paper that were missing like I mean they just did a very shallow thing and I wanted more in-depth analysis. So I actually asked Codex, can you go through these 20 papers and tell me what's and and then this time then CEX went and revise the code that that Claude generated. So some of this aggressive coding missed some of the nuances and then it did it. Now around the 20 papers I would say 15 were very good and the five it completely made up. you know those things don't exist in those papers and >> and I was very concerned uh so so that's kind of my experience but it also I also did the other way starting with codeex and move but it's the same thing whenever I use it ask it to use high effort so then that's when it hallucinates if it's sort of medium effort so this is the their best model and it seems that oh I have to come up with an answer so it just made up stuff or Maybe maybe you know it's on a time constraint. So that's kind of been my experience and it's it's quite concerning and that's why I read all the newspaper stories you know from law firms to everybody else has got this problems.

[00:19:18]>> Right. So >> right sure understood. So so I guess to to answer this we might be jumping ahead of ourselves a little bit but I will try to answer. Uh so for those who are not familiar with codecs or clocko you know they are the agentic um AI systems that are largely first developed for for coding purposes right but they have become general purpose and quite user friendly enough that uh you can hand them any knowledge work tasks uh and to to address your specific question what I do in my own workflow not necessarily you uh the same kind of literature review tasks. But what I noticed is first is uh how do we think about we can work with AI better not just communicating our intentions and ideas but also making things easier for them to work with. So that means for PDFs uh I personally would uh transform everything from PDF to markdown format so that they can actually read through right and also do their search keywords or whatever search they do they can they can match right uh so uh I will speak more on this so you don't worry about which tool I I I will talk more about that um and also what I noticed uh in own workflow. Similar to actually the conversational approach we're talking about is you it's like collective wisdom right the the the best comes out of collaborative work even when it comes to AI agents. So what I have in my own workflow say I'm in clock code uh I want to leverage uh the capabilities of codeex it's very thorough uh to the point that it can be mechanical but that means it can it can be very rigorous uh and I also like ki because it brings in different perspectives that uh chcodex or or cloud might miss right so in flood code you can spin up sub agents that you can see which is kind of hidden in this conversational AI interface but in PL code you actually see agents so you can bring in codeex agents which is a plug-in made by open AI so you can just tell club be like hey you know bring codeex agents and I and myself I actually I made a plug-in for ki agents so what I have if I were you trying to recreate what you did. I will be like, let's divide this test, right? Let's transform PDF to markdown and let's figure out what are the claims we're really targeting and let's fan out, right? Let's plan and see how we can f out and you know make sure every stream of task um or tasks that we have at least two or three agents you know of different models tackling the same thing and then obviously that will be too much output for any human to effectively digest in real time. So then you know you you have the main agent you're interacting with whether that's codeex whether that's cloud you try to specify hey here are the things that I'm worried about what did they find and actually especially with those agents you can tell cloud hey like we need you know adversarial reviewers right you're worried about hallucination so you can say let's get a few ki codecs and cloud agents that specifically try to verify theoretical claims around this concept, right? Treat it like a coding problem, right? Like people will say, okay, what about the integration of this these two codes, right? Like would they actually work? You can't just write the code and not run a test. So with this, it's like, yeah, you can write everything that sounds wonderful, but what if we run a test? What do we if we see this claim and we see a citation then we send agents to look at the actual papers to test if those claims hold or not. And often times they might surface things that will surprise you because you think you are you know you are synthesizing everything correctly but maybe you are operating from a different epistemic assumptions uh or different methodological assumptions from those paper. So uh to we can even go beyond by just uh you know backtracking but also extracting the potential tensions in those assumptions uh embedded in the papers. So that's what I will do.

[00:24:20]Okay.

[00:24:22]All right. Uh let's >> Yep.

[00:24:26]Okay. So configuration tool. So uh we we had a little bit preview of this actually. So now we're looking to deep research right as we many of us have published right uh here we're are showcasing cloud charge PT and KI but uh Gemini deep research is quite good and you you can actually export the deep research report directly into notebook LM uh and you can also run this deep research within uh notebook LM so that's that's actually quite useful uh so I try to be as broad as possible in covering different tools but by no means I'm endorsing you know any particular company. Um so okay so what I did here is you know from the previous prompt we the query for literature research um we had something and then we share the reports with club and see the map. So now we have a broad understanding and how do we translate that into a effective uh literature search prompt for deep research and a lot of times people will be asking how do I write this can you give me a template and I would say there's no template actually uh the the best template uh happens in your interaction with any particular AI agent because they know their systems capability best. We don't know what's happening in the background, what's being updated. So just talk to them, you know, share with them the insights you have gathered and they can write the prompt for you. So this is like a nice rendering of what uh cloud wrote here. Um and so we want to be precise about the scope structure and we will we want what we want to be surfaced and trust the agent will be able to split the work because uh to us we are still chatting with the agent in front of us but uh for Gemini cloud and charge they all have their agentic workflows uh and we don't know the exact number of agents they run but just so you know they they are running agents in the background and and later with Kimi we will see an actual example of what happens because they actually expose the back end.

[00:26:59]So with this um we come to the next part. So thinking you know really the query is a methodological decision uh rather than just than a simple question as we we have talked through so many things right uh this is almost like you you have uh a research assistant help you run u a literature review using the old approach go to web of science or or Google scholar and try to uh run multiple queries and synthesize things multiple back and forth but now you can compact everything into into a query uh because the agents will be able to do things and there might be lateral communications between the agents. Um so what you send out can be actually quite powerful. Um so again I ran this and this was what I had earlier and I want to see if actually finished it. Okay, he actually finished. It ran for 53 minutes.

[00:28:05]which is pretty pretty intensive. Uh and one thing I don't know if everyone notices is uh they actually made made it possible to export to markdown. Again, this is what uh AI agents are familiar with. So try to get that as try to use it as much as possible. Word and PDF they there word is like plain text plus some coding languages that are hard for AI to understand and then PDFs are just images. So again that then that impedes uh the effectiveness of the MA models because then they have they have to rely on uh Python script to to scrap the text or they have to read it as uh image using their vision capability which usually lags behind their um understanding of uh pure text. And this is the uh research ran by Opus 4.6 six and you can see here it actually ran only for eight minutes. Uh maybe that question isn't as challenging for for cloud. Uh but as you can see if you read this here you can actually see that it's actually doing four parallel streams of research. Right? So again with this prompt uh because we specifically laid out the four stream so it took the least resistant path and okay so that's one um okay so now we look at Kimmy agent swarm which is pretty cool um so basically if you can think of it as it now the agent has a running computer on the cloud. Uh they can coordinate a bunch of agents and create a bunch of different files.

[00:30:02]Essentially you will get a folder rather than a report you would see from cloud or charge GBT or Gemini right so here this is just an example of what I would ask to get back we can get PDFs you don't you don't need to manually click anymore if it's open access access u and you can get the bibiography reference file that you can import into zotterero or endnote and you get per stream summary so you can check if uh something along the way that interests you. Um, and you can even do face byace reasoning, have the agents write write down his reasoning. Uh, and in the end you have a production doc. So, uh, that makes the research process through these agents transparent and and not exactly reproducible, but at least it's auditable. Um so basically what you would do is you know same research question but then um add something about the workflow knowing that it has these capabilities. Um and so the lesson here you know again coming back to engagement design you can be precise about what kind of artifacts or outputs you expect uh and be charitable about how how the agent orchestrate you don't have to force everything because it could uh dynamically adjust its strategy halfway through. Um and also be clear about the analytical stages just like how we would do it ourselves. uh we can we can channel the same rigor and and sort of uh scholarly wisdom into this uh to to help us better follow along, right? So in the end you don't just get a nice sounding synthesis but also it's something that you can engage from beginning to end to to actually uh capture uh the whole journey of this learning. And so this is what it looked like when I first started.

[00:32:01]And let me show you what it did. Okay, so massive prompt. We have this whole orchestration deliverable stuff. Uh, how did I get here?

[00:32:14]Because Claude helped write the literature search prompts. And then I was like, okay, let's send it to Kimmy.

[00:32:20]And I talked to Cloud about what Kimy's agent swam is capable of uh, and what we want. So he wrote out this specification. And so Kimmy started by saying okay it will do a broad research across the five streams. So it did a bunch of search and then it started created sub aents by itself. We didn't specify the number of sub aents, right?

[00:32:47]And this is actually what's happening with Chupt and and Gemini did research as well, but we just don't get to see it. But with Kimmy here, you actually see that it created 12 12 agents. And let's see if we can replay some of the stuff. Um but basically it because he has a computer for each agent so they can actually browse website. So when they don't get PDFs they just browse it and read off the website. So then after this phase uh it did cross val verification and let's see extraction and and then it actually went to call more agents and then five agents to write and get PDF to create bibliography and okay it's still running. We will see if it can it can finish the job by the time we wrap up this this session. But it's been running for almost an hour. So uh theoretically it can spin up 400 agents and uh each agent can perform like 3,000 steps. So it's really powerful. Um but again depends on what you can give give to it, right? And all we have discussed so far really help us to develop this massive prompt that can produce something that that's actually valuable rather than um just a lump sum of everything that has been written about AI and management. We we have a very targeted query about a very specific area and we we care about the nuances right we figure out we are not trying to learn how large language models large language models are being used as part of the method but rather u how researchers treat them as theoretical object. Uh so that gives us much targeted search uh and hopefully yeah we will see some interesting results. Uh and you from this screenshot you can already see it tries to fan out uh to look at computer science and AI research uh papers.

[00:35:06]Uh so yeah so before going back to the questions in the chat room I just want to highlight you know a lot of us would stop here right we get the reports we have some citations uh and we move on and I want to say often that can just be a dication wearing a lab coat uh because the report is really raw material it um oftentimes it has very broad coverage across journals and years and surface papers I wouldn't have found on my own. Um and in this case it would it had like very clean structuring of method families. Um but taking these reports you can actually make it into something new right start a new session with the same model or car be be the messenger so to speak carry reports across different models and have them surface each other's assumptions and blind spots and that's usually from my experience the best insights come from where when you see different models in in agent forms is having done a long running tasks and they come up with some insights, they usually come to very different conclusions and and that usually helps us to to see where um the discussions are happening. um what might be interesting um what might be ignored uh you know bear in mind they might have different focuses and they use different search engines in the background um so highly recommend running multiple sessions multiple agents and and always synthesize them share them across the agents Okay. So, yeah, this is an example uh prompt I would use um to help facil facilitate uh this kind of cross discussions.

[00:37:21]Um and I'm reading the chat room. Um, Ibra, do you think we want to I can I can stop here and try to address some questions before moving further.

[00:37:36]>> Yeah, a couple of hands raised. Um, if that's okay.

[00:37:40]>> Y, do you want to ask a question?

[00:37:45]>> Let me just you hear me now. Yeah, I was just trying a simple clarification. I'm really impressed and I'm a big I'm a deep uh cloud code user and I'm impressed with the cross uh tool you're doing. Can you clarify for the Kimmy the version you're using and also if there is the call from clo code to Kimmy you hinted that it was kind of custom is it custom where is that custom thing or if it's not custom what is it just clarification so we can reconstruct your your flow on our own later >> you are you asking more about the agents in clock code or >> well it seems that you were calling from clo code the kimi part and maybe that's something we want to deal also for for Ibraat somewhat offline for everyone.

[00:38:29]Maybe you should do a very short memo saying I'm using cloud code this version, I'm using Kim this version, the desktop version and I'm using this custom plug that is available on my repo on GitHub. Maybe you should draft that out so that we don't bother you all the time with question about how do you do it, how do you do it because I'm really that's my I'm listening but but my brain is really oh I really want to know the details because I produce it.

[00:38:52]>> Let me that's a fair question.

[00:38:54]>> Maybe offline. maybe offline and you promise to us you'll send by the email newsletter saying this is a brief with a bullet point instruction like we only need like six steps but we want to know >> um maybe that's a best contract at that stage instead of trying to uh >> yeah fair fair question and and thank you for raising it so we can capture everything that people are interested in. I'm more than happy to uh share my tech stack so to speak. uh I I I tend to make everything open source uh because so for for this session because we're not quite sure where everyone is at at the moment. So we try to talk about so as you can see so far we're talking about the conversational interface and the consumer interface and you know more more user friendly kind of stuff and less about the coding environment. Um, but yeah, more than happy to do it and I I just reply you to the chat about the plugin I I have >> or maybe you prepare later and send to everyone after we finish because maybe some other stuff will surface. If I may, I think your stack in general is very useful. I went to your GitHub. I can see you produce stuff. This is all amazing.

[00:40:00]>> So both we want your stack in general if you want to share. Secondly, for what you present, you say, "Listen, for what I was doing, the minimal stuff is clo code, Kim, desktop, da da da, plus that plugin and so that we know that we want to reproduce, we should try to >> ah okay yeah >> so I would recommend do it offline. I don't want to take more of the bandwidth of the group discussing this, I want to know the exact list of tool, >> right? So I I think so far uh for for those who who who who who are who who may or may not be concerned about you know missing out if they are not using clock code or codeex uh I I don't think you're missing much right I I I think the principles with learn the orchestration that's happening in your head that that carries through you can do as much uh as others using clock code or those are the tools in this more traditional you know chatbot interface they they have integrated so many functions in the back end that you can pretty much do everything um nowadays uh unless you are you're building an app or something then I would say okay then then clock code for sure u but yeah so far I think a lot of stuff can can be there might be involving some u manual copy pasting uh you know you really being the conductor here um but that But that still exists even in agentic environment like clock code. Okay. Uh more questions.

[00:41:31]>> Sangita.

[00:41:32]>> Yeah. Hi. Thank you Brett. Um Shoulie by the way amazing presentation. Thank you for organizing this and sharing this with us. Um so I wanted to ask you I've been doing and running into a lot of sort of latency and token burn issues using cloud and cloud code. So, and it's just like resetting, you know, even though I've I feel like I'm not I haven't gotten to where I need to get to. So, I'm curious, what model in cloud are you using? Cuz I've been switching between them and they all seem to be having similar issues, although I've gotten feedback that adaptive um is better with this, although I haven't seen that. So kind of curious to know your thoughts >> right with cloud I I I use uh I use the extra high for opus 4.7 um >> okay >> yeah just it may not use all the thinking budget but you know I just turn it on um and then uh sometimes I switch to Opus 4.6 six uh it can be better at you know spending time talking through things with you. Um and then you always have the solid model which is uh cheaper and then quite uh capable as well and and for I would say yes the the you know relying on a single model company can can be brittle you know you don't know what was happening with their servers.

[00:43:11]Uh so what I have is you know I I I have my trapd account I have my Kim account and their models can all be swapped hot swapped into clock code right uh and open source communities have made it quite easy you can just like open up web page and then just pick the model provider and the model you want and and still launch things in clock code. Uh so that's something um >> okay >> I would I would keep as a backup option because you know whenever clock goes down you see those fun tweets or memes about you know the the war stops um but actually like then that that's the point about you know don't you can't have a single point of failure right you switch to podax or or charge PT or or ki uh I I do use less Gemini in in in work uh because it's I it can be quite weird uh in in in my domain of research so I don't use it as much um but yeah but definitely check out you know all those open source models take a look at uh open router you know they they have so many models available there and um for most of our tasks I don't think we don't I I don't think we always need the latest opus model right so you can you can swap up to some alternative models.

[00:44:36]So, >> okay, thank you.

[00:44:38]>> Yep.

[00:44:41]Lee, >> hi. Um, I have never used the Kimi um and uh I want to see how how many tokens does it use. So, when you use it and also um how to connect it with cloud or Gemini because you said that you actually can can go into Kimi. So more details will be great and more demonstration will be great because I think it's too general right a lot of people have questions in the chat they don't know how to this yeah >> so so for Kimmy like besides the so they have a separate usage package for the coding environment they call it the Kimmy code right so that that plan is quite generous and it's much cheaper than cloud uh and then for the online version like I have uh running quota or running agents. I don't know the exact number but uh yeah like I have I have once I ran a task that involved almost 100 agents and so I needed to cool off for three hours but but I mean that that that wasn't too bad. Um so yeah. Do you specify how many agents you need or does it automatically allow you based on your prop?

[00:46:03]>> Uh I deliberately try to give it the autonomy and freedom to to judge along the way.

[00:46:11]>> Uh I I have tried specify very specifically how many agents you need to run. But >> um it's like us doing research, right?

[00:46:20]you finish step one, step two and then you might adjust something in step three and and whatever you you had specified might not >> be valid anymore. So okay, >> you don't want to you don't want to foreclose, right? Maybe you need more.

[00:46:33]>> Yeah, I understand. So it's like a parallel, right? But parallel based on what different stream of research you as to search or right just a random parallel >> that that again you know depending on your research question and and you know that went into you know what you discovered right and here obviously we wanted to do a more parallel stuff and then bring them together but I can see instances where you you want things to start not parallel but you know you start with sequential and only parallel afterwards, right? So, because essentially it can run up to 400 agents, right? So, if you think >> 10 agents per round, >> then you have 40 rounds.

[00:47:19]>> Okay. So, you need to subscribe to it, right?

[00:47:22]>> Right. For for this agent thing, you you need to subscribe to it. Um >> and you can you can run any models, not necessarily a specific model. For example, >> this is just Yeah. This is just >> model. Okay.

[00:47:37]>> Yeah. Yeah. If if you want to recreate this, yes, like with this agentic environments like you know in clock code or or codec then you can recreate this.

[00:47:46]But yeah, it will be your custom workflow. But there are enough >> open source projects online that you can just ask uh >> cloud or codeex to write it for you and you can run it.

[00:48:00]>> Okay. Okay, sounds good. Uh, and another question is on on security. I know when you do AI agents.

[00:48:06]>> Sorry, Lee. I'm sorry. Um, I'd like to give a chance to other people ask a question as well, but we can get back to you in the next round of Q&A. Is that okay?

[00:48:16]>> Joy, do you want to ask your question and then Yang and Mumita? Um, if it's okay, we'll raise discuss your questions in the next round of Q&A.

[00:48:24]>> Thank you so much.

[00:48:26]>> Uh, thanks for sharing. I have a quick question. introducer how to connect AI with Zotterero to let AI answer my questions based on the real papers such as I'm using GMI but I don't know how I tried many times.

[00:48:42]Yeah. Um again that that comes to the question we we said earlier right like usually when when you have a deep research or just a quick oneoff conversation they look at paper they read the title and abstract and that's what Zotterero stores right um so there are like MCPS so for those who don't know MCP stands for model context protocol those are like a little communicators between whatever app or database you have uh and the AI agent So there are multiple MCPS for Zotterero.

[00:49:16]So you can just install them uh in in cloud desktop app or or cloud code or codeex.

[00:49:23]>> Um so that's easy to run but then for them to actually read papers. Um there are actually zotterero plugins that bring AI agents into zotterero but everything gets stuck there. So what I do is actually like I said earlier convert everything into uh markdown file right. So I actually store everything outside of zotterero uh and also I have the bibiography file exported uh in whatever format XML or uh CSLJSON right those are all easily readable and searchable by AI agents. So I still use Sautterero a lot. That's for me as as human researchers. But for AI, I have tried to like here's what I would do for a research project. I have already selected the papers. So I I would export that collection out of Zotterero and then I will find the papers for that collection, you know, the PDF files and have the AI agent help me convert everything from PDF to markdown. So I I lock it down as a sandbox fully AI agent to to play with.

[00:50:31]>> Thanks for sharing.

[00:50:32]>> Yep.

[00:50:37]All right.

[00:50:37]>> I think you can move to the next part.

[00:50:39]>> Yeah. Yep. Okay. So we we kind of touched on this engaging across sessions. You know again those technical stuff are super important but how do we think of orchestrating so many capable agents? Uh that's something again carries through no matter uh what you use. Um, so yeah, so after getting a report, right, here's what I would do.

[00:51:05]Like one is more exploration focused, open-ended, just to get a sense of what's what's out there, right? I I haven't read all the papers and I don't aim to be factchecking here. I just want to understand the conceptual landscape.

[00:51:20]what are the theoretical discussions that are happening and then kind of use that to navigate the next steps to dive in so to speak and find those papers, read the papers again and then run those uh later agentic workflows on top of those papers.

[00:51:37]Um and the other side is more critical uh focused uh you know step out of the field you know in this case you know if we read it through AI researcher's eyes uh do what management research has found makes sense to them uh are there any gaps in the methods that management researchers are using uh that would be push back uh so and I wouldn't do it as you know you blah blah blah as kind of like character prompting. Uh really it's kind of using the collective perspective if we are right. Uh so I actually try to think uh from that perspective as well alongside AI and then that will help me catch what I might be missing as well.

[00:52:28]So yes, so that keeps the agent honest but also keep us in the loop. Again, we're not trying to automate everything. If that's the case, uh again, that that's epistemically hollow. Uh and you can already automate everything on GitHub.

[00:52:46]There are enough packages out there.

[00:52:48]Okay. So, take away for for this configuration is discovery and interrogation uh as a design sequence uh engage engagement design across multiple sessions and models and productive suspicion of any single output. uh and have in mind a workflow with faces each with its own purpose. Uh so with agents like you know Kimi agent swarm you might have to write it down and give to agent in one go. Uh if you're using a more conversational AI approach then you know you may have a note on your side and just remind yourself that you you are here and here's where I'm going next.

[00:53:27]Um, and also uh notice that what will always stay opaque, right? We touch on this because often times the agents work from titles, abstracts, secondary writeups.

[00:53:39]When PDFs are accessible, the agents will try to read them. But again, um, not all the time. And uh, some PDFs are not accessible. Uh, and because PDFs can be and and to be honest, our papers are quite long. uh you you give five 10 papers to AI agents then they they start losing losing track of the the minor details uh which compounded over time or long longer sequence of tests can can result in bizarre findings. Um and also notice that whatever we try to automate so far here it doesn't always carry our bowlerly voice. um we get competent pros but the argument at least to me doesn't feel like my argument yet. Uh so throughout this process I feel like I'm curating insights through me reading the reports reading the original work uh and and take out what's not needed and really find that direction I'm uh moving towards uh and and from there you can iterate right the the whole workflow again uh and that's what um I guess to me that's what AI systems really enable us right to to automate some minute stuff and really uh engage more deeply and more broadly.

[00:55:06]Okay. So lastly uh I want to touch briefly on the full bandwidth. It seems like we have a lot of clock code and codeex users. I'm I'm happily surprised.

[00:55:18]Uh so we we can I can share more as uh discussed earlier um uh afterwards through Ibra um but uh for those who are not familiar with uh those uh tools uh let's get a sense of uh how how I work with them. Um so basically uh this is a nice rendering of the terminal environment. Usually it's quite intimidating for for folks who who haven't done coding in their life. Um but what's nice about this is because it's it's operating um in a terminal environment.

[00:55:58]So you actually see each step what it's reading what it's doing uh what files it touched what agents uh is spawn it has spun up and you know even the messages between the agents you can read all this right think of it as all like um without getting to clock code what you will get is just right tools that's it but maybe in the background there's hundreds or of of stuff that's happening. Um so in the coding environment you can actually get to inspect and importantly you can stop it. Uh so you you are actually actively participating in this research process.

[00:56:43]Yes you can automate but I would argue that before you can automate you actually have to work alongside the Asian to figure out what really clicks for you and your research question. then maybe you can automate some stuff and clock code and codeex they have internal features that help you automate things you know looking things um in sequence or schedule jobs uh those are all useful um but um again I think uh the same three principles hold through here um and like mentioned earlier You have clock code, you have Codex, you have Gemini CLI and Kim CLI. Uh, and except for clock code, every the other frameworks are all open source. Um, and you can actually swap all the other models into clock code. Um, but again, here's my usual setup.

[00:57:48]Um, on the left hand side you have, this is roughly my screen. on the left hand side you have the terminal open or for those who use clock code in the cloud desktop app you have that screen um but what important thing is like how do you check outputs right we're not necessarily writing codes that can easily be rendered as as a website or an app or or some analytical results especially for literature review oftentimes you don't like you could get some numerical results or metrics uh but oftentimes it's pets plain text. So I actually set up obsidian. So that's another part of my text stack, you know, in order to work effectively with AI agents. I use I write everything in markdown. Uh actually I did it before Chad GPT became popular like I started with using markdown when I first started my PhD but luckily it now became more mainstream and I highly recommend everyone try to look into it. Uh, Obsidian is free to download, free to use. Uh, and you get to read PDFs, markdown in there. Um, and yeah, so so you get to see what's happening through those markdown files. Uh, you you can talk to your agent on the agentic CLI screen. Um, and on the other hand, you have the markdown files that you can read and provide feedback. Um, and again, uh, I have a plug-in that can help you set up pretty much what what I use, uh, if you use cloud code, uh, and I will make that available to everyone afterwards. So, you can, uh, run it through clock, you know, cloud will interview you or whatever model you use, we'll interview you, help you set up your Obsidian environment, whether you use Mac OS or Windows PC or Linux. Um but yeah and and yeah some some stuff that are also relevant you know how how do you convert PDF to markdown uh it's in the plug-in there are some good tools that I have used um and also you know how do you save uh to have a good practice to save all your interactions uh on one hand it helps you to reflect and and to reference those conversations you have had so far uh bring them to different models. But I think increasingly more important is going forward, imagine if you have a study that you had an AI system um help you along the way. You want something auditable or at least can be summarized into a appendix about the AI usage, right? So save those transcripts.

[01:00:45]Um and you can go back and then whatever the editors and reviewers ask then you you you have this evidence that you can write from. uh and we we tried to do this in one of our studies and uh I have found it uh quite useful because honestly a lot of projects can happen over multiple months and you don't really remember what exactly happened uh you know for me like two weeks ago I don't know what happened so uh I I rely on this transcript to notice what what I did what I didn't do uh and also it's nice like after a long period of exploration you can talk to your agent of choice and be like hey let's look at what happened right what did you notice I miss or what did you notice you miss like is there any reflections on how it work together um so so in the end it becomes like a flywheel like you are generating your own data that you keep private private locally and you can create skills for your agents to work better along alongside you um so it's small stuff um that will pay off long term Um, so again I will I will have that plug-in available for everyone afterwards. So you can uh try it or or take it apart, pick whatever you like to integrate into your own workflow.

[01:02:09]Um, so we sort of already touched on this. Um, but again what was good about this agentic uh environments is the choices by every agent surfaced. uh you can intervene mate process uh and you can channel the three principles we have talked about simultaneously uh but like I said there's a setup cost you need to understand what files what tools what environments you're using and and to and to get in there in order for you to be able to work alongside um the agents I think it will become increasingly easier uh but to get the benefits now uh we we need to make an effort to to try out those things. Um, and it's easier than than many would think. I I would say especially uh I think it might be uh quite straightforward for for those who running who are running quantitative analysis, right? You you ask the research assistant to write some codes, but now you ask claw to write some codes. Um but as a qualitative researcher mostly I I would say actually it's even better for for qualitative researchers. You jump in and you you have all your text files there and you can read what the agents do and you don't need to talk about codes. You you you still are just having a conversation back and forth and you get to bring in literature, bring in data, you know, pending privacy concerns. Um and and you can iterate uh as many times as you as you like. Um so it does provide more bandwidth u but it doesn't always guarantee better work. Uh because we we get the illusion of productivity but I would argue that oftenimes it prompts us to engage deeper uh to to be able to challenge our own ideas and to discard the ones that doesn't hold upon the examinations. Um so to to kind of bring it back so we went through the conversational configuration then we talk about deep research uh the visibility changes uh but same principles and and same practice. Um so since this is what we did well we started out or I started out asking you know what when management researchers run all this evaluations or experiments on large language models what methods do they use and what kinds of claims do they make based on those methods it was a reasonable opening you know after a few rounds of conversation with cloud but didn't quite land um so it evolved right we we actually captured the theoretical construct that can help us bring in the relevant literature. So it becomes cognitive entities and evidence um licenses those claims and what kind of claims that can be uh that evidence can actually support. Um so yeah in the end I think the question you end with matters much more than the question you start with. Um and uh yeah three principles thinking through um engagement design uh and sometimes have that uh introspection inward um it's so overall uh I want to say yes we we will uh touch on and we will keep talking about the the specific texts and techniques Uh but I really hope that some of this has helped shape how you think about the process of engaging with AI agents. Uh and if you want to learn more, feel free to check out this uh paper that just came out yesterday.

[01:06:08]Uh and actually we we map out the different strategies that map to uh the configurations here.

[01:06:15]Um so I won't bore you with this right now but yeah so some resources again uh Ibra will be able to help share uh this and and any other questions you might have about the tech stack um and tips uh so yeah that that that's all for me.

[01:06:43]Thank you, Shal. Um, yeah, let's open up for Q&A. Um, there were lots of questions in the chat and I to be honest, I lost track of them. Um, but please bring it bring them out here uh into the conversation and so we can see you please uh raise your virtual hand.

[01:06:59]Um, I think Hovik uh had a question and he needs to go soon. Hovik, do you want to ask your question? And then Janelle, >> um yeah, what I was asking uh was whether using the agents um the CLI models in terminal >> and looking at what it's actually doing is a one for one of what's actually taking place because I think in reasoning models you'll see it say I'm now gathering literature. I'm now comparing this against that which doesn't necessarily reflect what's happening underneath. I'm just wondering if it's one for one of what's actually taking place.

[01:07:38]>> Right. So yeah, so again that that's similar to how uh you know in those conversational interface, right? It it's uh the the chain of thought is usually summarized by a different model, right?

[01:07:53]And it can be short, it can be long. So when we bring everything locally right especially when you run those uh multiple agents uh in those CLI environments you get to see the outputs of uh each phase right remember uh kind of what we did with the ki agent swarm I think it's a it's a nice example uh so we talked about uh we want uh the PDFs we want the notes and stuff like that so I think It's about slowing things down and not just trying to get a easy take away from from the agents but rather kind of build out the analytical processes in the form of folders and files so that you can bring in other agents to help you check but also you you can go back and check yourself. So rather than trusting everything in in a you know in a single step by by the agent in in one interface you break it down.

[01:09:02]>> Got it. Yeah. But it's but it's reasonably accurate. Right. Um I don't know what reasonable means in this case but >> uh if if Right. Uh well I guess that depends on what kind of questions.

[01:09:20]Uh but ultimately I think the epistemic res responsibility resides with you as a human researcher uh and and you you should be able to back it up, right? Um >> yeah.

[01:09:36]>> Yeah. Yeah.

[01:09:37]>> Thank you.

[01:09:40]>> Janelle, >> thank you. Um I'll turn my camera on.

[01:09:46]There we go. Oops. There we go.

[01:09:50]Maybe not. Okay. Thank you for doing this session. It absolutely has changed how I think about AI in this process.

[01:10:02]And my question kind of ties into the one in the chat by I think it might be Jorge Sound. Um >> so it's not a technical question. It's more of a um ecosystem question.

[01:10:17]>> I'm curious about Well, I'm no uh the question um the question I have is how prevalent is this AI research ecosystem approach for engaging with the literature within academia and uh you know is it being taught how how frequently is this being taught at university? Um, I I run the writing center at my university and so I've been working at a place that says, "Nope, we we're staying away from AI. We want everything to be done by the students all the way through." And at the same time, I see what you are what you're showing and I'm like, "Wow, this is a completely different view and understanding of what AI can do for research, for literature than my university is currently holding." At the same time, I also see um Sandival's comment about the influence of AI increasing the volume of low quality research submissions, lower quality writing. And so I'm curious about what is the actual prevalence of this practice within academia? And I know this is a general sweeping question, >> right? Right. Is it do you see it being uh cultivated within universities with students as this is the way we're going to do this moving forward and what are you seeing in terms of the quality of the work that is coming out in general?

[01:11:58]Thank you.

[01:11:58]>> Right. Right. So I guess to to uh yeah reference for his uh comment uh that's coming out of uh uh or science substack right? Uh yes, if you mindlessly just let AI models handle everything, yes, you will get lower quality. And unfortunately, we are seeing across disciplines, across countries, we're seeing the flood of publications coming out. And I don't have a you know I I I don't have visibility in how others are using it per se. So I can say yes the observations I would say are quite consistent with what I have been reading online you know what others have published um but I would say how prevalent it is across the universities I don't know but I have I have helped train the new K students at Imperial and what I noticed is that you know I I worry about hey I I went through my PhD doing things the old way. I spend so much time creating my Zotterero and I worry about hey I got like a thousand papers I never read.

[01:13:16]Those days are gone, right? And then and then so I start thinking well how do I how do I help those students to learn something that they may not even get a chance to to to experience that that that's worrying. Um and I don't know if I can force them to not use AI. So here here's my take look look at so pretty much I told them what what I have shared with you today and I work with them you know we we try to do the literature review in clock code right using different models so they get a flavor of different models capabilities um yeah we can get to something reasonably sounding quickly but I'm like in order to get better results um that engaging and interesting for your for yourself yourself to read then you actually have to go through the whole process like I said earlier like you can automate everything but in order to make it truly yours like carry your scholarly voice you have to work alongside the agent uh and I would argue that by going through step by step in order to automate things you might have learned more than doing things by and because you get feedback so fast. So I think there there there's a there's some hope here and I think it takes um I guess more senior scholars to try all those things to notice what works what doesn't and and help guide the the students right they they are curious and and they are using it um but they may not be thinking about the right questions or or the relevant standards or criteria that should go in into working with AI and producing um good quality work uh and and still have that epist epistemic responsibility.

[01:15:13]Um so yeah that that's my take.

[01:15:19]>> Thank you so much.

[01:15:21]>> Thank you Janelle. uh if I could build on Janelle's uh question um and Mita will come to you in a moment um and in particularly in view of the paper that just came out in ORC science uh the the intention for this webinar was really to facilitate uh you know how we engage with AI in a responsible and reflective way um and surely I was wondering maybe something you could comment on particularly in light of the comment you just made about the new generation of PhD students including at uh some of the elite universities.

[01:15:58]How do we cultivate reflective reflexive and and responsible approach to using AI so that >> we still produce high quality research and it's our voice that counts. Uh I mean again I feel like I uh I would point to these three principles that I'm actively developing and trying to uh instill in my daily practices. Right? So think it through rather than have AI automated for you. uh use AI to as a aspiring partner or thinking partner and then be I would say notice your own engagement with the AI and try to design it so that it helps you to safeguard those epistemic criteria and and you know responsibility or scholarly rigor that kind of stuff um and and capture that so it becomes uh your your own mode of working with AI agents uh and take time to reflect uh don't it when you spend too much time working it can be great but uh people have burnouts and we have seen this among the the programmers too they now they can run 10 if not a 100 or 200 agents at the same time they get so so much output but how many new great apps do you see right? U so they have the same problem. So I think it's often times good to be able to take what we learn from engaging with AI such as in literature search uh again read the papers, read your notes, write your notes and then you can bring it back uh if you still choose to uh engage with the agents or or you don't have to right you can you can now have a better understanding and and do things the the traditional way. Yeah, thank you Mumita.

[01:18:01]>> Um, hi. Uh, thank you so much, Mr. Len.

[01:18:04]Um, I just have a few questions. Uh, while I do know about the different LLMs, I'm a new a little bit new to using agentic AI and incorporating them.

[01:18:14]So first of all if you are like just so that I understand it correctly if you are to include agents in any kind of um LLM platform be it Kim be Chad GPT the literature review or whatever is being the responses from the chat from the LLM do you think it's like 99% or 98% accurate or is there still a slimmest possibility of hallucination >> um I think I mean I don't think it's a soft problem especially the way we work we we if you look at a typical literature review there are maybe a 100 papers and they may be coming from different disciplines and there are so there's so much nuances uh specific to the type of claims and arguments you make in your own writing right so I would think of it as I I never trust just say you know treasure or claude or or some writes a report and I take it at face value fellow no like I I take it as okay here's the broad arguments and then I start writing and then I will be like okay here are the actual papers go read it challenge me if it makes sense and in the end I will also be like hey you know maybe this is something claw produced fact check right like I have these papers check it because again like I said uh a lot Lot of times what we get from those schools they they are not reading a full paper and when they don't read it they only make the broad claims they make connections you know rawly at a conceptual level. Um but to get to the deeper like factual stuff I would say you always go back to the the actual papers. So, I would say those chatbot interfaces, they're great to to have be the place you start to to brainstorm um to learn about different areas, but once you get uh down to the details, I would uh say you almost have to move into uh the CLI environment so that you have the papers and you can read the papers together with AI to make sure you you're getting everything right. I see. Um, just a one follow-up question. This is more of an advice. So, do you have any like video or a tutorial where you kind of walk through how to integrate these agents and use CLI for the literature review?

[01:20:53]that uh could could be I don't have that yet you know um I think really we thought about it doing it for for this session but we hoped that we we start with something so we get everyone on the same level start with the right principles and hopefully uh you know in the future uh either either Ibra inviting other schoolers or I can do it in my free time to to kind of do you know those live stream kind of Um but we don't know but I will be happy to if there's you know enough demand >> thank you so much Mr. some wrap.

[01:21:38]>> Hey, hey, thanks Ibraat for giving the chance and uh congratulations first of all Zulle for this paper and uh for your work >> 25 you have another five minutes um to wrap this up.

[01:21:51]>> Sure. So I've been following your loom series and uh part of the question is from what you discussed and maybe the other part is the future session you can hold is more about the more I look at how engaging we are and you spoke about uh engagement there is this process through which probably a new identity is evolving one which kind of fuses the human and AI and I wanted to take some uh thoughts from you on how we can craft an research agenda if you would like to look at this process of evolution because uh you know this is this is something which is interesting and this is not a collaboration this is more like fusing two identities together for creating something >> um yeah that I guess I'm in the same boat with you right I'm curious and then oftentimes I notice I think it starts with uh uh trying to talk about it like Ibra and and Kevin they all have helped me uh to to get to have this opportunity to share what I've learned. I think this self-reflection putting things down uh in ways that others uh can at least understand. I don't expect everyone to to to adopt well what I do. Um I think that that has provided tremendous value for myself and explaining uh what what's possible to the PhD students uh and also looking looking outside of academia see what other knowledge workers uh even you know the researchers at those AI labs what they are doing and oftentimes they are sharing their stuff online too right so you you get to see the patterns right emerging but in terms of research agenda I think To me, I I think it's it's exciting, but also things are moving way too fast to have an anchor and say here's what I will study because we are living in the phenomena and which is exciting. Um, so so yeah, I don't I don't have a research agenda answer for you. Yeah, because I I recognize and I struggle to to to come up with something that that I would say uh would be evergreen in in in in just based on what I have gathered now.

[01:24:16]I think it will take maybe a few more years to to see how the dust have settle settled and then reflect back because you know last year I mean even even uh Ethan and and and Hilas you know they have studied great organizations adopting AI and by the time things come out you're like okay like people are using different things now right but those are still great insight so that that's what's exciting but also So it's like okay how do you set a research agenda for that? So maybe that's a better question for for Hila and then those uh scholars.

[01:24:56]>> Sure. Thanks. Uh I'll keep track of it and uh great work. Thanks a lot for sharing it with all of us.

[01:25:03]>> Thank you Sarah.

[01:25:09]>> Yeah many well no many thanks. I I just wanted to well first huge thank you for organizing this session really great and I would just like to to know if there is any sources any videos with a stepbystep almost like step by step how to use AI for dummies uh while doing literature reviews and uh and also what you recommend in terms of LLMs do you recommend same LLM for different parts of literature reviews some are bet best for some parts others are better for others But but yeah >> yeah u uh I would say I will try to answer the second question first I would say always uh so again like speaking back to what we said earlier open router it has all the different models I would say never lock yourself into one single model uh they all have quite different uh training data training techniques right was working on this paper like the hot copy. I'm going to give you another five >> and uh so so the outputs can be quite different. So I would say um yeah try to try to you know not lock yourself into one single choice. Uh but you know stuff like the so the app you're using they they can all be hooked up with different models, right?

[01:26:38]So you have a consistent interface uh but you can interact with different models. Um and to start I would say cloud chaptt they're all great deepseek I'm sorry I'm just I'm just giving you all the names. Um but depending on the type of work you are looking for right some are great at coding some are better at you know summarizing longer texts uh some are better at adopting specific theoretical framings uh so if you have something more specific I can then I can I can say say yeah give you a a narrow set of recommendation >> I think it's just for now it's for me I haven't used it I tried It's very basic but very extremely basic. So that is why I was wondering is there any place like how I could and maybe others in the call have the same doubts about where to start kind of very very basic how to start what are the different models how to actually do it.

[01:27:48]Okay, that I will I will take that uh as part of the feedback for for whatever we will send out afterwards and and try to include some uh steps and resources on that.

[01:28:00]>> Okay, thank you so much and again thank you so much for this presentation.

[01:28:03]Definitely the best I attended uh in terms of AI and so thank you.

[01:28:10]>> Um surely I'm not seeing any more hands raised but maybe um we could close with one more question for me. Um I I sense reading from the chat conversation today which was very active um that there are some people who kind of using very actively and seem to be on top of the game so to speak but you also have seem to people who just like Sarah mentioned you know kind of studying on this um what does it mean for for those who haven't yet quite grasp you know how to use AI at advanced level. Do we need to so to speak jump on this bandwagon in order to compete in the research process or yeah what what is your view on this?

[01:29:04]>> Right. Um, like I said, I want to what I what I'm coming up with uh earlier on is we have different configurations.

[01:29:17]Um, and I wouldn't worry if you're only using a conversation interface option. Um, for example, um, with cloud, right? you even clock in their desktop app has become much easier to to interact with. So I think it's knowing what's possible and what how how you can carry out this process uh alongside agents. uh I think this kind of knowledge matters uh less about the technical skill for example uh uh for for with Kevin and and my supervisor you know I I try to convince them to try out PL code right and it's daunting like I I I'm not sure how I could do it but luckily we have the the user interface in plot code that's quite friendly so I I work with cloud and build this plugin to interview them just just typing right answer questions and in the end quad would create the folder install the tools uh and and and take notes of their preferences and everything is there right so I think uh you shouldn't be too worried because uh the user interfacees are getting easier and easier easier to use every day. Um, it's and you have open source communities who are making it easier uh for researchers like you, like me to to to engage with these AI agents. uh and so yeah and you know be just I think the biggest hurdle I I see actually is just be curious uh to see what's possible uh rather than taking a you know flat out which sometimes can be impractical stance uh whether you are for or against AI um so you know just yeah like what you're doing right like once we we have talked about these principles we we look at what's possible then just take one afternoon off try it out.

[01:31:38]Uh you might be surprised what you can achieve in that one afternoon. Um and and yeah, you can get free trials, right? They they are trying to get more customers, you know, those labs. So, you get free trials. You try it out. No, you just spend some time and think of a research question you can throw at it.

[01:31:57]Um then, yeah.