Agentic AI in Higher Education:
Transforming Research, Data & Campus Administration

00:39Thank you for joining me today. My guest is Robert Miller, and we're going to be talking about artificial intelligence in general and his work with it. Robert's background is in information technology, as well as the implementation of Agile and Lean within the workplace. Past roles include director of information technology at Marmon Holdings, a Berkshire Hathaway company. City Retail Services business, global product manager for GE Capital, and now he's using his expertise to help us out at Datatelligent, where we're working to implement artificial intelligence in meaningful and responsible ways to help support our partners' growth outcomes. So Robert, great to have you.

01:23It's awesome, Debbie. I'm super excited to have this conversation as, as everybody knows, everything is AI and AI is everything these days. So here we go. Let's, let's dig in.

01:34Let's talk. Okay. So let's start out by defining some basic terminology for our listeners because you know, I've got colleagues who they've kind of Shut down the information flow because the idea of implementing AI is very Challenging let's use that word for them. So can you briefly explain to our listeners? What is a Gentic AI? What do we mean by generative AI? What is an LLM? And what do we mean when we say we're using machine learning?

02:08So I think the first thing we need to do, Debbie, is actually to take one little half step back before that, which is a lot of us have actually been using AI and experiencing AI for many, many, years. A lot of us have been in situations, whether it's in the workplace or with our bank, where we go in and we do a transaction, on our phone. And that transaction then causes something else to happen. You know, I set up to schedule a deposit that's going to happen at a certain time or transfer or pay a bill. That was actually the very early days of what today we're seeing as more sophisticated AI solutions. And that was with a concept called robotic process automation. So the idea there was, is that you could train systems to receive instructions and then follow those instructions, whether it be in a scheduled fashion or whenever triggered by some sort of an input. That really led us down the path to machine learning. that's ML as most people hear it these days. Machine learning is really the next evolutionary step there. And it's a simple algorithmic trick. That's all it is. It's a feedback loop. that trains and fixes and improves that automation that we set up with something like an RPA. And it's no different than what we do in our own lives. Someone comes back and tells us, hey, know, that thing that you're doing, it's great most of the time, but it fails when this happens, or it gets this thing wrong. And then we all step back and we go get into our conference rooms and we go, my gosh. Our process is not working when X happens. All these machine learning tools are doing, and I'm simplifying here because obviously just like in the workplace, some processes are simple and some are complex, is doing that algorithmically. So it gets that result back. This thing failed, somebody rejected it. Somebody flagged something is incorrect or they chose not to.

03:59Right.

04:21To use the end result and then it explores why. And then it tries to come up with ways to improve that, whether that is, you know, trying a different option, selecting a different product, whatever it may be. And that's all it is, is a constant feedback loop that is retraining that automated process. That is machine learning. It seems very simple because it actually is very simple until you scale it up into these big. giant complex things. Then of course you have what most people think about now with AI. So, you know, we've been using RPA and machine learning since the early, early 2000s. When I was at GE, when I was at Citibank, you know, we were using a lot of complicated processes related to this business processes and things. Large language models then started coming in really with voice chatbots, those dreaded terrible phone tree chatbots that we all experienced many years ago and loathed even to this day. That was the beginning of large language models. is really where all of this started, was trying to understand and interpret language in a natural way and then translating that back into logical text that then a system could understand.

05:23Right. you Right.

05:42And that's really all it was. that evolved over time. you know, OpenAI kind of hit their little, they were the first ones across the finish line, but of course they were chased pretty quickly by several other competitors where they finally got that hook to where they could get to around 80 to 85 % accurate all the time, consistently. And that's really what exploded into what we now see as generative AI. And generative AI is really just really sophisticated large language models that take our natural language prompts, interpret them based on how they've been trained, and then recreate responses based on that data. And that data is not, I wanna pause here just for a second. This is not robots. This is not androids from Star Wars. There is no actual intelligence here. This is augmentation and algorithmic logic. This is no different than when we do data analytics and we create a model in academia, you know, to predict student outcomes or, you know, when we're doing research in our social sciences and we're taking survey data and then we're coding that data and then we're using that to interpret results. This is the exact

06:40Yes.

07:01Same type of thing just being done at scale and really, really fast.

07:08So Robert, what you're saying is all of the forms of AI cannot operate intelligently without human input. Each one of them has to be trained and the quality of information that you train it on directly impacts the value that you then receive back out of it.

07:15That's correct. That's correct. That's correct. And so the public models that people have been exposed to, you know, just playing around online, you know, chat GPT and things like this, they're what are called generalized models. They're just trained on tons and tons of information, both publicly available and private, but they're kind of general use models. And that's why you can do a lot of crazy things with them, but you also get a lot of crazy answers from them. Right. And that's because They're trained on this huge thing and they're not specific in industry, right? Whether it be in higher ed or in research or in private companies, what you're getting is refined and tighter trained models that use very, very focused data sets to train them. That's why they can be more accurate. That's why they can be more reliable. That's why they can get into areas that maybe, you your public... Chat GPT just can't because you might have, you know, trained data on them that's unique to your business or your function or your area of study. So the training is big, but also it's important to recognize that it becomes just as important what the user is prompting the model for. It is because it is using that prompt to drive the entire algorithmic response, how you frame their prompt,

08:39That's right.

08:48How general it is, how specific it is, the tone you use, the references you make will also greatly influence the response. And that's why you've heard in the popular sort of zeitgeist in the news media lately where you're finding people that have started to use them as kind of pseudo-therapist. And why? Because these models will start to feed back to you what you're asking them to feed back to you, right? If you're consistently prompting them to respond in a certain way, they'll give you that type of response, which obviously can be very fulfilling for people. And so it becomes a pseudo type of, you know, I'm doing air quotes here, but therapy, it's obviously not real therapy. It's not clinical therapy, but it is in a sense, form of this sort of social feedback that they're looking for. Yeah.

09:16Bye. So yeah, I mentioned to you in earlier conversation that we've had that I've been reading Ethan Molyk's book Co-Intelligence and that is exactly something he points out, that the chatbot will want to please you. And so therefore, if it cannot find, or the AI, not a chatbot, the AI will want to please you and if it can't find what it thinks is the answer you want, it will give you what, it will just ran for it.

09:53That's right. That's right. That's right. And there's a concept. That's right. And there's a concept out there called hallucinations. It's a very real issue where it will again, through these algorithmic processes end up creating a false answer. And now there's a lot of things that you are being done to reduce these, to eliminate them. But again, in these very general models that people are

10:07They will reinforce the cues from you. Yes.

10:33Using just kind of freely out in their daily lives, there's not a lot of constraints and it's very easy for them to sort of pull the model down these paths that will then start to generate these types of responses. Because again, to your point, it's just trying to feed back to you what you're asking it for. And that's what they're designed to do. so, you know, so that's kind of what most people are exposed to. And now what you're starting to hear is this concept of a gentic. And particularly in the higher ed space, this is really the area where they're hearing from at conferences, they're hearing at by their vendors that this is really the secret sauce that's going to open up the use cases for higher ed. And the reason for that is the idea behind a Gentic AI is stop trying to be a generalist. Don't try to solve all the world with one thing. Don't take the hammer and then assume everything is a nail sort of approach, which is what most people are consuming right now with these big open source LLMs. So with agentic AI, the idea here is to build and design very, very focused solutions that combine that robotic process automation that we talked about earlier, the machine learning concepts. and the generative AI LLMs, whether it be generating words and or images, and then build very tightly focused, highly accurate solutions that solve one tiny little thing. And then you can chain those together as you need to solve different business issues, to help in research, to really augment. the human that's actually doing the thing. You know, if you go to a tech conference, one of the terms that you now hear by everyone, and it doesn't matter whether it's a big consulting firm like an ENY or it's the vendors like, you know, OpenAI or Microsoft is they'll hammer to you. Human in the loop is the critical success factor to using artificial intelligence solutions in the commercial and or nonprofit space. And the key there is the human part. A lot of them are even trying to rebrand AI to mean augmented intelligence as opposed to artificial intelligence. And that's a really critical mind shift to think about this more as tools that help you achieve and accomplish goals.

12:56Right.

13:07In a more rapid fashion or by expanding access to people that maybe aren't as deep in the technical space to allow them to access solutions that before they may have had to have waited, you know, maybe even paid for outside help to do. It's really an augmentation of the experts within a given space. A great example of this is in research. One of the big things that's happening now in medical research, and I still have a lot of network for my time at GE Healthcare, is image reading. So radiology. Reading images is incredibly difficult. And it's one of the areas that has one of the highest error rates within the medical industry because It's hard. These images, even if they are technically detailed, the human body is so complex, every human is unique, that it's very easy to misinterpret these images. And then when you're a radiologist and you're reading hundreds and hundreds of these images a day, the human fallibility comes in, you you get tired, you miss little things, and those things can become critical. They can either lead to missed diagnoses or false positives that then lead to unnecessary procedures, et cetera. So a big push and they've, they found really great success already in the last three years with having an AI assistant that's been trained on millions, millions of these images and how they've been read by, by officials and then tying it to the actual clinical results, you know, when they open up or when they do a treatment, you know, whatever. They've been able to help to guide radiologists. now they're not the systems they're not even allowed to by the FDA. They're not allowed to actually make the call themselves. That's just to be clear in case anybody's scared that their MRI was read by a machine and not a human. The human still has to make the final call. But what it does is it'll highlight on the screen, hey, radiologist person, something's weird here. Something's out of the norm and based on, you know, millions and millions of prior images, this is something you need to pay attention to. And that to me is really right. That is right at the core of what this is all about. This is about augmenting people that are experts and enabling them to do more, to be better, to get through their work in an easier way, to allow them to apply their expertise without being burdened. with all of the sort of boring aspects of their jobs.

15:43So let me see if I understand what you're saying. It sounds like in the use case that you're talking about with reading x-rays, it sounds like what the AI is doing is recognizing an anomaly that maybe the healthcare provider never saw before. But because the AI has been trained on maybe thousands of images, of course, they're not all going to be the patient of that healthcare provider.

15:59Correct. Right.

16:11And so, you know, so basically it's seeing a disruption in pattern and then just, like you said, tapping that practitioner on the shoulder and saying, hey, did you notice this? So.

16:22That's right, that's right. And in clinical studies that they've done using this, they've seen false positives in particular drop by over 70 % and misdiagnoses improve by about 35%. So it's really interesting to see that, you know, they're seeing very real relevant results with this. And the feedback they're getting from practitioners, is that it made their day easier to get to. It made them, you know, feel like they can get through and be more accurate without the stress of their job. They feel like it lowers that stress level because they feel like things are going to get caught more likely and, you know, all of that sort of stuff that goes with that. You know, so, but the real point here is that it doesn't really matter the area you're applying AI to. or these concepts to the, what matters is how you do it. You need to be thoughtful and plainful. And I know plainful is not a real word, but plainful is something that I love to use because you can be thoughtful and still reckless in the way you do something, right? And we see this in institutions constantly, right? You you'll have a lot of meanings, a lot of smart people, and they'll say, you know, it'd be great if we do this thing. But then they don't really do the hard work of figuring out the best way to do that thing. They just kind of toss it out there and then it may or may not work based on sort of dumb luck, right? But so to me, it's you have to be thoughtful. You have to think about where it is right to apply these solutions. What are the constraints you want to put on it? What is the governance that's important? What's your end goal? Do you have the right experts to do it? Do you have the right experts in house to actually run the process once it's there? And then you need to be planful about how you go about introducing it into that. How do you test the waters? How do you do limited releases? How do you expose it and compare it to the live and run in parallel for a period of time to really build up confidence? What sort of change management are you going to do within your organization to educate people around what it is and isn't doing? and how this does or does not change their jobs, their activities, downstream processes, whatever it may be. These are all incredibly important things to think about because you have this potential to unlock your best employees, your best researchers, your best educators, let them focus on the things that they love to do that you hired them for. and take all of the noise out and empower them, but you have to do it in a way that's smart because powerful tools can also create powerful problems.

19:05This that is true. This is great So let's move on to something that man many of the data professionals who that I know are concerned about and that is the risk of using a variety of types of AI in our work in the data world What concerns do you have? about AI and its use in analyzing data or maybe you're beyond the concern and you have discovered ways to mitigate those.

19:35So I think my biggest concern and I've seen this out in industry and I think everyone broadly is now sort of coming back to recognizing this concern, which is when you get a new shiny tool, oftentimes what happens is there's a rush to use it outside of the normal controls. of an environment, whether that be a company, a school, whatever it may be. And so things like data security, know, testing and quality assurance, things of those nature tend to get bypassed or minimized in the rush to use something. This is cool. This is fun. Let's just get it done. Just use it. You know, there's sort of that hype cycle. Everybody thinks it's more than what it really is. Then you have the crash of the hype cycle when they realize, well, it wasn't quite that. Okay, now, now let's figure out what it is and where the value is. And I think it's that, that third point in the process is kind of where we're finding ourselves now. More broadly out in the industry as people are starting to recognize that these are tools, they need to be applied and controlled and, and be thoughtfully used. They're, not panaceas to just solve every problem, you know, without much effort or work. And so, you know, that really is the big thing for me is making sure you still have, you know, your data security people involved to make sure that you're not exposing data to the wrong people. You have your quality assurance, you know, processes in place to ensure that whatever you're doing is meets whatever standards are required, right? Whether that's 80 % accuracy or 100 % accuracy, it depends on what you're looking for.

21:15Well, and you have to have a clean data set.

21:16Well, that goes without saying, Debbie. The number one problem in the world is that nobody has a clean data set, but you need a clean data set for everything.

21:17I I know that, But you did reference the hype, everyone trying the new tools, even my colleagues outside the data office. And they think that they can just grab a data set and go for it. And that's not really true, right?

21:47Right, right, and a lot of it has to do with the nature of the data, right? What tends to happen is they'll grab a very, very simple use case and they'll find success because it is a very, very simple use case. And then they get sort of an overconfidence that occurs where then they think that they can just apply it to a much more complicated use case without doing the work that's required and then that's where it blows up in their face. And that's kind of when you then get the downside of that hype cycle where they go, oh, none of this works, right? And then they don't trust it for a little while, right? Because they misused the tool. And we've seen this through earlier technology before, right? This isn't a new thing as far as the cycles go. New technology for sure. But I think once... people realize that this is just the next evolution of data science and data research and analytics. This is just the latest tool set, right? Remember, it wasn't that long ago when in order to do visualization, someone had to manually do them, right? You got your data in a big file and then someone had to go and create visual graphs and things like that. Then we started introducing these visualization tools. We democratized data analytics to a certain extent and because those tools allowed us to create curated data sets, curated dashboards with drill downs and sliders and all these sorts of things. You know, there was a mistrust at that time. How do I know that, you know, Bob in accounting understands the data deep enough that when he makes those selections that he's getting an accurate answer? Well, you don't until you train Bob, right?

23:32Right, right.

23:32And then you put the controls in your tool to keep Bob from hanging himself because now he has a tool that he never had before. And the same thing is true now. These tools are going to amplify and augment and push those capabilities further down the path, but they don't take away all those same concerns. Do people understand their data? Is the data architected and cleaned and curated the right way? Are the users trained to understand what the tools can and can't do? And is there a culture of trust but verify in your organization where you don't just blindly accept anything that's passed in front of your face? You need to understand lineage. Where did this come from? What's the source of this data? And these things are important, particularly when you get into

24:12That is... Right.

24:21You know, complicated data sets or regulatory type things, right? Then you need to understand that lineage. You need to understand who's doing it. You still, and it's important here, and you kind of hinted at this a little bit, you know, in your question, I think, which is, hey, those data professionals that you have, they're still here. They're your bulwark against the wild west use of this.

24:40Right. Exactly.

24:46It's natural for them to resist new things, because all humans resist new things. They are actually going to become the critical users of these tools in educating and training and putting those guardrails up and helping people and then being your experts that can use them for more complicated or more sensitive use cases where you need that deep expertise to get to where you want to go.

25:05Yeah, I have to admit one of things I've never really worried about with the advent of AI was the loss of my job because I've always recognized that you need a data professional in charge. This is, you and I don't think that a lot of my colleagues nationally hold anything. any different views. I think they all realize that our businesses, our campuses, whatever organization we serve, they still need us at the helm to lead.

25:38Absolutely, right. And remember, right, we had this same fear when things like Power BI and Cognos, remember Cognos guys? You know, when these tools all first came out, right, it was, well, we're gonna, you know, we're just gonna hand this out and everybody's gonna do their own data. And it's true, there were team members that were able to do a lot of really tactical, functional, cool things.

25:48Yeah.

26:03It was awesome, right? But it didn't eliminate your data team and your data professionals. Really what it did is it increased the value of your data because you just expanded the user's user base. And really that's what this is going to do, but just to the next level, right? You're going to be able to push this where instead of having on the finance team, the one or two people that learned a complex tool, you're going to have maybe five, six, seven, eight people.

26:08Right. Yes. Well...

26:31That can understand how to use natural language prompts in the smart way and do that. So it's again, it's further just expanding the value of your data, not so much making it or replacing jobs or anything like that.

26:46No, that is right. If anything, I saw the possibility that as a one person office, I would have more time to do what I really wanted to do, which was to have those collaborative relationships and working one-on-one with colleagues outside the data office so that they were more confident in their ability to understand data. to write a good research question to really understand what the visualization told them. So let me ask.

27:14My graduate student's son, he's in political science. He's working on his PhD in political science and he's in the comparative study side, which is the more data-driven side of political science. And so they have a big issue of you have to code surveys and the coding process that they use is manual. I was talking to him just the other day around how, can't you just set these parameters and go through and do this coding in a more advanced way where you're just sort of validating the coding as opposed to manually doing it line by line for thousands and thousands of rows. And he was like, that's possible. I'm like, well, yeah, it is. It is. But that tells you, right? He's at a tier one research school, and yet they're still manually coding their research to this day right here in 2025.

27:53Right. Yeah. Yes. coding. Yeah. All right. Let me ask you one final question. If you could use artificial intelligence to solve a national or a global pressing need, what would you direct your efforts toward? What would you solve?

28:22I think the number one challenge that we have right now that is an easy target for artificial intelligence and data science in general is food distribution and logistics. There is a real problem of the fact that we produce more than enough food on this planet to feed everyone easily. It's not even close. We overproduce actually. And the reason we overproduce is because of this distribution problem. We have a very serious issue in understanding how to move food products around the globe in a timely and efficient manner and then store them. And I think these are incredibly good targets for the use of deep data analytics and artificial intelligence because it is so big, it's so complicated. It really transcends beyond any one expert's capability. Probably whole teams of experts capability. the world's a complicated place. There's a lot of different factors in it from politics to physical logistics, et cetera, et I think that's a problem that we could surge this resource towards to create at least a strategy that then could be advocated for by NGOs around the globe to try to get this distribution problem at least improved.

29:33Think that would be a terrific use of artificial intelligence. Robert, it was great talking with you. I know I learned a lot more than I did before we started 30 minutes ago about artificial intelligence. And it's given me some thoughts about paths that we're taking here at Datatelligent. Your time is really valuable and I'm glad that you chose to share it with me and our listeners today.

29:43Yeah

29:58Have a great rest of your week and I hope you and your family have a happy holiday.

30:02Absolutely, Debbie. anytime, I'd love to come back sometime and talk about, you know, whether it be artificial intelligence or whatever you find interesting in the world. Have a great one, guys. And to everyone out there listening, have a wonderful holidays yourselves.

30:16Thanks Robert.