Transcript 0:00 2024, have we seen the gen AI at least partially fulfill, or to some extent fulfill the promise of democratization? 0:09 The deployments are happening, implementation is happening, but has there been an improvement in workforce, um, or workplace productivity? 0:18 Each industry is different with healthcare, retail, finance, each one adopts technology in a different way. [on-hold music] Hello, everyone. Welcome to one more episode of Unlocking the AI Advantage podcast. 0:34 This is your host, Ramesh Danta. Today's topic is taking a review or look at twenty twenty-four, what happened in the AI space. Thank you for joining. 0:44 And, uh, s- and you really wanna know, um, what happened and what's in stock for twenty twenty-five. All right. 0:52 So, um, the generative AI continues to dominate the tech world, uh, but the AI has become more nuanced and, uh, companies have become, uh, mature in how they look at AI, uh, taking it from experimentation to actual real-life deployments and, uh, uh, implementation. 1:10 So as a result, lots of, uh, things have, have come into the picture and discussion, whether it's ethics or regulation, along with the, uh, the technology, uh, improvements itself. 1:21 So just to take a look at all of them, so we have, uh, really fantastic speakers, uh, panelists for today. So let me now introduce the, uh, panelists for today. So first, Paul Singh. 1:33 He's the principal cloud solution architect with Microsoft, and Kavitha Tipnis Rasal is... she's the CTO of MyTown AI. So Paul, if you could introduce yourself, please. Sure. Hello. 1:45 Uh, my name is Paul Singh, and I'm a principal cloud solution architect, known as a CSA, with Microsoft going on twelve years. 1:54 I ac- I'm an external-facing resource focused on helping large customers deploy solutions such as gen AI and AI generally, um, also around data. 2:06 Uh, s- I'm also helping support regional, local, um, nonprofits and startups as well on... as a, you know, so a hobby. Also, I recently authored a book on generative AI and cloud solutions. Thank you. Great. 2:22 Welcome, Paul. Kavitha, please. Um, it's hard to follow Paul after twelve years at Microsoft. [chuckles] Kavitha Rasal here. I am the CEO and founder of Light Route Ventures. 2:34 We are an AI application advisory firm, and as part of our work, we join forces with, um, a company called MyTown AI, which I am the co-founder and CTO of MyTown AI, where we bring AI to small municipalities across the US. 2:50 So we have about fifteen cities in our pilot program right now, and we are rolling out digital twins and AI assistants to these, um, government officials for planning and decision-making. 3:01 Uh, before that, spent almost close to a decade at Workday and Oracle combined, um, and then I had to jump on, you know, learning to do and start with AI. So that's why I'm here. All right, Kavitha, welcome. 3:16 And then, so I know the ventures leads into a startup CTO, so I think that cycle continues. So all right. 3:24 So with that, fi- let's start the discussion by looking at significant AI advances that actually took place in twenty twenty-four. 3:30 We thought twenty twenty-three, uh, was the momentous year, but twenty twenty-four continues to, uh, make advancements in AI, right? 3:38 So, uh, there is no stopping there, and just Sora was released yesterday, so it's, it's, it's happening on a daily and, and actually in some cases an hourly basis. 3:47 So but we can look at these advances, um, in multiple angles. But, uh, to frame the discussion for today, so we wanted to discuss it from two angles. 3:57 One is the infrastructure and the platform level, so what went on in twenty twenty-four, and then secondly, the application level, so what went on. 4:05 So let's, uh, start with, so Paul, so from a infrastructure and platform level, twenty twenty-four, what kind of advances have, uh, and advancements have taken place in twenty twenty-four? Sure. 4:18 No, uh, that's a great question we could start off with. And, you know, it depends on what type of business or organization you are in terms of platform. You know, mostly everything is cloud driven, as we know. 4:30 Um, but generally, I think for, from infrastructure point of view, really, uh, the areas that I've seen my customers and people that I work with closely in the organizations, they want to scale, um, scale out and be able to use different models for different tasks. 4:46 And some of the scaling requires things such as, you know, how do you load balance across either multiple regions in the cloud service or even across multi-cloud as well. 4:57 Again, no one organization i-is dependent on single models. Uh, you know, we'll talk about a lot of different models in the future, but, you know, again, 5:05 uh, on this call, but there's, you know, models by Meta, Google, Microsoft, you name it, you know, everybody has, uh, just large language models as well as in, you know, other public domain models as well. 5:17 And really, how do you take advantage of them? And it, it really go, boils down to leveraging your cloud infrastructure. 5:24 A n-nice thing is you're no longer having to run these models locally on, you know, laptops or local servers. 5:31 You could use the power of the cloud and specifically around GPUs, which, um, as we know, that's, there's a huge market in that area as well. 5:39 So infrastructure includes the, the GPU, the processing behind AI, as well as being able to load balance and scale, uh, either regionally or globally as well. 5:52 Um, so some of the, some of the st-strategies I've seen is using an API management layer in between to say, "Okay, now I'mI'm intersecting these, uh, actions, prompts as you would call, and then being able to distribute those across many different data centers across the world and, and doing so in a low latency manner. 6:13 So... Hmm. Interesting- Yeah... that you're already started ta-talking about deployments. That means actually no longer experimentation, so that's good, uh, so Paul. 6:20 So that's a key, a key observation, uh, that you're making there. So Kavitha, so what have you seen in 2024 from a platform and infrastructure perspective? Uh, great. 6:31 I think one thing that Paul mentioned about the multi-cloud deployments, right? 6:35 So when we are coming in from the application layer, I think we are innovating so fast, we don't wanna get bogged down by, um, specific infrastructure requirements for these hyperscalers, whether it be AWS or Google or Azure. 6:49 We expect a level of abstraction in that layer so that we can deploy our applications, LLM applications in a multi-cloud environment. Um, so from a... 7:00 Like let's say the way we are seeing these AI architectures evolve is you have the hyperscalers that are providing you with the compute. Mm-hmm. 7:09 Then you have these LLM vendors that are providing you with, uh, multiple LLMs and different flavors of the LLM, whether it's large language models or small language models that are increasingly efficient, such as your Microsoft's, uh, Phi-3 or, you know, GPT-4o Mini, 7:26 that do give you a lot of, um, you know, compute power also and reasoning, uh, capabilities- Mm-hmm... that are coming out. 7:34 So one of the key innovations I would say is probably in the ability to run any model on any platform, and I, I feel that is accessible. 7:46 You can easily switch different models, um, you know, at that layer, infrastructure layer. Um, the new models, as you said, they keep just coming out. 7:56 So you have the reasoning model that OpenAI just, uh, you know, upgraded, I should say, at a different price point. Mm-hmm. Then you have the multimodal models that are 8:07 being re- You know, still in its infancy, but they are getting better. So you saw Sora's release yesterday. It is getting better. So a lot of the AI so far was text-based. 8:18 Now it's gonna be image and, you know, audio, video and, um, other modalities, right? And then finally, uh, you know, once you have that infrastructure and platform ready- Mm-hmm... 8:31 you're able to stack your, you know, workflows and convert them into agents, right? So that's in the pipeline, I would say. So if 2023 was a lot of awareness around, "Hey, this is coming. Let's experiment." Yeah. 8:47 I feel 2024 you're seeing a lot of these discussions around how do we scale? How do we actually productize this? How do we actually, you know, load balance, to Paul's point, about what does the user volume look like? 8:59 Got it. What is the user experience looking like? Hmm. Great. So actually taking that discussion further, you already started, uh, getting into that, Kavitha. 9:10 So, um, one of the things that I have seen, um, is that you talked about the text video, so multi-model, not the MO, you know, not the MODEL, but MODAL and the different modes of, uh, inputs, right? 9:22 So that is, uh, definitely has come through. And the other one that I have seen is the multi-agent, right? So we are talking about, uh, the customer service agents and employee agents. It's a data agents. 9:35 It's, it's like a, th- seems like, uh, 2024 is where this multi-agent... So Kavitha, if you could, uh, talk a little bit about, um, this multi-agent, whether we-- it's application level or platform level still. 9:47 So what are your thoughts on that? Uh, definitely. I think agent is definitely the buzzword that probably, uh, of 2024. 9:56 But leaving the buzz aside, right, I think historically automation was you could automate a series of steps in a workflow- Mm-hmm... using software. 10:06 When you add AI to any of those steps, in my view, that becomes an agent. When you add, uh, any agent or any AI that can reason or think, that becomes, you know, a reasoning agent or an AI agent workflow, right? 10:23 Um, so multi-agent, it, you have one agent that can just, let's say in a customer service context, is just looking at incoming tickets and providing you with the relevant knowledge base in the article by triaging the ticket flow. 10:40 You use an LLM to do that, which it has the context, and over time it realizes that, hey, it can direct a call either to, um, a service agent or it can just provide an answer from the knowledge base. 10:52 So that's your simple form of agent. Can you add another agent that is basically going to ship a replacement product? 10:59 That is where you'd start seeing these multi-agent use cases where, okay, I really see a customer who is unable to solve a problem- Mm... through my first level of support. 11:12 Can I ship, if in the context of e-commerce, can I ship with a replacement product to take that experience to the next level? So a lot of, um, multi-agentic frameworks are coming up. Microsoft has its AutoGen. 11:27 OpenAI, um, has something called its Open Source Swarm, where it's starting to release that out for, um, you know, developers to start using it. 11:37 There is CrewAI, which is well known in terms of easibility and ease of use from a development perspective. So you can-- you're able to create these, um, agent frameworks using CrewAI, uh, as well. 11:52 LangChain is always in the mix. And then you have a lot of low-code, no-code vendors also starting to operate in this space- Yeah... 11:59 where it gives some of the business users the ability to play around with like multi-agent frameworks.Great. 12:07 So Paul, um, taking this discussion further, you could, um, give your own insights into multi-agent, um, but also go into any other areas like I know open source AI is another thing that's coming around. 12:19 Uh, and so any other, uh, whether application level advancements or whether it's a platform level so, uh, that you have come across. Yeah, absolutely. 12:29 You know, the multi-agent or agentic AI, as I refer to as well, is on my list of things that happened in twenty twenty-four, like top three. Mm-hmm. 12:38 And also expansion in twenty twenty-five, which we'll talk a little bit later as well. 12:42 But you know, for those that may not know the term agentic AI, it really is r- it refers to this autonomous system AI, basically being able to plan actions or adapt to changing situations. 12:55 Um, really the good purpose is to achieve a goal. 12:58 And what I've seen is now these agentic a-AI agents, they're able to call other agents as well to now expand, try and f- do a particular task, or if that agent cannot complete that task, to determine, okay, what other model or what other, other additional agentic agents can I use to deploy within the application or service to achieve a goal or result? 13:22 So I've seen that happening in twenty twenty-four, and then I'll add some more later when we talk about twenty twenty-five. 13:29 But really again, it's dis-distilling complex tasks into singular tasks, smaller tasks that these agents can complete autonomously. 13:36 I think that's a, that's the main keyword there is to do it autonomously with minimal direction or, uh, guidance there. So Paul, um, then let me switch topics a little bit here. 13:48 Um, so we said at the beginning that Gen AI has been the dominant, right? So since been-- So that is still the, you know, eye candy for, uh, lots of people out there. Sure. 13:58 But one of the promise of the Gen AI has been that democratizing the access to AI, right? Anybody can do it, at least for some use cases. 14:07 Uh, you don't have to be a very sophisticated enterprise IT customer to be able to use. 14:13 So taking this discussion further, twenty twenty-four, have we seen that Gen AI at least partially fulfill or to some extent fulfill the promise of democratization? And, and your thoughts on it, please. Yeah. 14:27 No, great question. So we have seen the democratization of Gen AI to some degree. 14:33 However, with the proliferation of different tools, a lot of organizations, like, you know, building out o- their own frameworks, not only around agents, but entire tooling set that, uh, we haven't seen some sort of standardization yet and adoption as widely as, uh, I think we can in twenty twenty-five. 14:52 So it's, um, for example, a lot of the agentic AI, they can't work well with people with disabilities, for example. And there's some limiting factors there. 15:03 However, you know, I'll, I'll talk about what, where I see some improvements in that area. Where what I've seen in twenty twenty-four is the ability to create these agents and use different tooling and tool sets. 15:13 But integrating this to applications in a seamless way and doing it at scale is still something that's a work in progress. And I'll see that growth, definitely w-we expect to see that in twenty twenty-five. Sure. 15:26 So Kavitha, through your ventures, uh, Light Ventures, I'm sure you're coming across lots of, uh, startups and companies that are pitching. 15:34 And then there's also, I mean, you are very active in Women in AI as well, and you're seeing lots of, um, access to, I think, uh, different groups in the society as well. 15:45 So what are your thoughts about the democratization aspect of it, whether it's a Gen AI or any other AI? Yeah. No, I, I think that's a good segue into the current startup, the MyTown AI, right? 15:58 So the reason the startup exists is to provide access to a very powerful technology to these small city officials for planning. And we are seeing the effects firsthand. Mm-hmm. 16:09 So when we released our product out to these customers, the first initial few thoughts were like, "Wow, I have my Q&A knowledge base, and I can ask any question. 16:19 It just knows my city's addresses and parcel records and my knowledge base, right? Whether it's municipal code or ordinances." That access we wouldn't probably would have been difficult to provide several years ago. 16:35 So I'm seeing like the firsthand impacts of like, okay, when you, you know, make something so powerful that is Gen AI and package it in an easy-to-use way 16:46 for, you know, um, people that wouldn't necessarily have the access. Yeah. And they would have to go to the bigger vendors or, you know, consultants at this. 16:54 You almost have like your own, um, consultant in your pocket, right? That's how I think of some of these access to technologies. So that is helping out some of the customers with MyTown AI. 17:06 As far as the Women Defining AI, that is a community that, you know, educates women around use of AI and learning by doing. 17:14 Like we have, um, you know, classes around and just, um, you know, sessions around like, "Hey, can you just get together and build an agent so that you can solve your email marketing problem, or that you can solve your, um, you know, homework or calendar scheduling problem for your children?" 17:32 And that gives, you know, women and other minorities just access to, okay, I can build something on my own because you're already using AI to, as your coding assistant, you're already using AI as your, uh, thought partner. 17:46 So then you're taking it further to actually use in, um, you know, your daily use cases or daily life use cases. So that access, I do feel, yes, it's not maybe as ubiquitous. 17:59 You might see that it is only holed up in the researchers' hands, but no, we are seeing on the application side that, hey, anybody can use it, um, with a little bit of what you say, this is how we get started.Hmm. Great. 18:14 So again, so now, uh, with that, let me go into, uh, the next section of discussion, which is about, um, the reality check, which is, you know, have we really, 18:26 uh, you know, changed the playing field with respect to implementation? Um, so we talked about, you know, some major trends that have happened, and as a result, you know, the playing field definitely has changed. 18:38 But the key question is, twenty twenty-four was supposed to be the year when, uh, uh, lots of AI would, uh, come off of experimentation, kicking the tires phase into actual deployment. 18:53 So, um, has the playing field changed where it, it's-- What are you seeing? Is it still experimentation? Uh, are, uh, pe- I mean, are companies are actually getting into the mature deployments of AI? Yeah. Yeah. 19:07 Great question. So I-I-- It's a combination of both, honestly. I, I see both a-applications and organizations, companies creating app-applications that have gen AI ability. Mm-hmm. 19:18 Um, I've seen, you know, mature systems and services out there already. For example, there's a company called Kira Systems who does, uh, they do legal software, legal AI- Mm. -and across organizations. 19:31 They help with marketing, sales, customer service, and also, um, do contract review as well. So they're already having, uh, gen AI capabilities built into their product and services. 19:41 Uh, you know, even you go any chatbots and Q&A, like Kavita mentioned, that's low-hanging fruit. 19:48 We-- I see that in organizations where they may do a Q&A against their own data using RAG implementation and saying, "Okay, I, I, I want to have questions ar-against my healthcare or, or human resources questions as well." 20:01 Definitely, I see it's that, those low-hanging fruit. Um, some of the areas where I see kicking tires is still around the cross-industry solutions, um, such as, uh, RPA, ro- you know, robotic processing automation. 20:16 That's still in its infancy early on. Hmm. RPA has been around for quite some time, but, um- Yeah. That's quite-- been quite some time, so that's why. Yeah. Yeah. 20:23 However, combining that with AI or gen AI is still e-e-ear-early on. We're in the early phases. Um, there's some companies out there, for example, in the restaurant business kn-known as Miso Robotics- Yeah. 20:36 -who in Southern California, they already have, uh, a automatic RPA frying station in o- in some restaurants. 20:44 And so, so we're starting to see early adoption, uh, still kind of kicking the tires, but testing out phase of, okay, how do I implement combination of RPA, so process, robotic process automation, with gen AI or AI and melding the two together. 21:01 Hmm. But definitely, uh, that'll be great growth in twenty twenty-five. I'll cont- we'll continue seeing that, um, trend. Hmm. However, to your, point to your question, I, I think it's a combination of both. 21:11 It's still, uh, people are testing the application and services. 21:15 Some already are, you know, with mature, mature processes and mature AI ops, we call it, um, they're able to, uh, implement a little bit more complex solutions. 21:27 Again, gen AI and AI, it's a very broad field- [coughs] -different aspects of it that are already being implemented across many organizations. Hmm. 21:36 So, Kavita, I want to take this a little forward and then say, okay, fine, the deployments are happening, implementation is happening, but has there been an improvement in workforce, um, or workplace productivity as a result of this, you know, the deployments? 21:52 You know, you gave examples in the government sector, people can get answers, all that stuff. But as a result, is there a measurable productivity improvements, um, that AI has brought forth? 22:06 Uh, so great question, and I can give you maybe two examples. So by trade, I'm a software engineer, right? So then by-- as from a software engineering perspective, and if you see one of the 22:16 most notable use cases of AI is generating code. And if you're reading report from Microsoft or Google or AWS, almost twenty-five to forty percent of the code is AI-generated, and that is in use in these systems. 22:32 So that gives you some of these, I would say, your, you know, senior and senior software and beyond, they are definitely more productive. 22:42 I think I'll have to see the numbers, but definitely, at least I can give from my experience, ten to twenty percent more productive with the, you know, use of AI for writing software in comparison to not using AI to write software, right? 22:56 And then if you take that to different almost job functions- Mm-hmm. -what are some of the functions that have seen the most benefits? 23:06 So apart from the software engineer, your researchers at, in the sciences field and healthcare and medical field, right? 23:13 Those researchers, they have seen their research time drop, uh, significantly by using some of these reasoning models, and they are able to put out more experiments out in the wild. 23:26 So you're seeing those productivity gains for, um, the tasks that are more laborious, more manual, you know, and more, you know, I wouldn't say analytical. 23:38 Analyticals are still, like, do a lot of the analytical and code reviews piece. Mm-hmm. But more of the, you know, "Hey, I don't need to refactor my code every time. For that, I can use AI. 23:50 I don't need to figure out, like, how many, you know, protein combinations that would come up as a researcher. For that, I can use AI as a thought partner." Yeah. 23:59 So those are the productivity gains that we are seeing early on. 24:03 Um, there is probably, I'll link it, you know, after this, but a paper by Ethan Mollick on the earlier productivity gains that we are seeing in the space by using AI as an automation tool, and it's similar to the printing press, uh, revolution that we had.So it's just the beginning that we are seeing. 24:24 There is definitely a challenge in terms of measuring those productivity gains because we are just in its infancy. Less than twenty-five percent of the use cases are in production- Yeah... 24:36 at the bigger, uh, companies, right? The smaller startups, the smaller innovators, yes, they are doing a lot of things natively using AI. But the bigger enterprises, there's just more risk involved. 24:50 There are more checks and balances, so to speak, before you put something out there in the wild and in production, right? Yeah. 24:57 Um, so that we will see continue to like, hey, people putting more efforts into the risk mitigation plans, risk frameworks, the evaluations of these AI systems 25:09 to kind of give them the assurance that, hey, this is actually right, this is working, and then I can see the productivity gains. Yeah. Great. 25:18 So this question is actually probably, Kavitha, we'll start with you, but Paul, I'll get to you as well. 25:23 The question is: How has government, uh, agencies, uh-- how have government agencies adopted and leveraged AI compared to the private sector, and how are the challenges different? 25:34 So Kavitha, I know you mentioned small cities and towns, and that's what your MyTown is. 25:40 But also, let's also discuss in on the federal level and the state level, uh, is there, uh, talk about AI deployments, implementation, and how easy, difficult it is? 25:52 Because each industry is different, whether it's healthcare, retail, finance, each one adopts technology in a different way. So how about the government? Uh, for sure. And I think this year was an eye-opening. 26:04 After spending like decades in enterprise, right, right, and on the government tech side, uh, it's quite refreshing, to be honest, right? 26:11 In the last six, seven months, every government official or every government conference, they have been at the forefront of actually leading innovation. 26:20 And they have-- California has, um, you know, a Gov AI coalition that runs out of San Jose, and then it's, you know, a membership, um, organization where different government officials come around and, you know, discuss ideas around how to adopt, um, AI in these use cases. 26:38 So in many ways, some of these government agencies are quite forward-looking and innovative, and they are looking for solutions that are easy to deploy, easy to manage and maintain, uh, without additional overhead. 26:56 So they are looking at ways of where some of the small towns that we are working with, um, they are staffed by volunteer, you know, uh, folks at the, you know, um, board of trustees level or even at the municipality level. 27:10 So they are looking at ways to bring economic development to their town with the use of this technology to bring more, you know, foot traffic to their, you know, shopping centers and malls and just business to their cities and town. 27:23 So a lot of discussion in the, you know, Sacramento region in, um... 27:29 I was fortunate enough to go to a tech conference in Kansas City, and even in Kansas City, they're all discussing innovation that AI can bring, and it's more about doing rather than having plans. 27:41 So a lot-- from a budget perspective, quite a few agencies are deploying budgets to accelerate AI innovation. So you will see that happening. 27:52 A lot of these, you know, um, state and local governments will, for sure look for AI solutions that are built natively on AI, um, and, you know, help solve some of their, you know, paperwork overhead. Hmm. Great. 28:07 So Paul, uh, you probably are seeing, I'm guessing, uh, again at, at the large government agencies, right? So on the cloud deployments. 28:16 So can you talk, uh, from that perspective, what's going on, uh, with respect to AI in the government? Sure. Sure, Ramesh. Yeah. 28:23 As Kavitha mentioned, yeah, we are seeing lots of in-innovation at, at federal government level. 28:28 I have a lot of colleagues who, who do work in that space, and one of the differences, one of the main concerns they always have i-is security. So one thing we did not mention was security. 28:38 Of course, if the government's gonna be managing any type of AI and, and that data, n-number one priority is always, okay, how do I keep it secure so public can't access other countries, et cetera. 28:48 Um, so, so in that regard, I know a lot of the hy-hyperscalers have dedicated areas just only for the federal government, for example. 28:57 That means your average people cannot deploy application services that co-mingle or, or are co-hosted in that same space. 29:05 So security is a big, big area that, um, the federal government and our, you know, state governments also utilize AI for. Okay. This is great. Yes. Yeah. That's, that's exactly right. Okay. 29:18 So but I think in general, the perception is out, um, out there is that government is slow to react in terms of adopting technologies, so I think people are curious. Yeah. 29:29 Um, and again, and setting the budgets and the proce- the process and all that stuff, it, it might take. 29:34 So but I think both of you hit upon that point, um, uh, well, understand the question is more about the costs associated with running advanced AI models and the resulting equity, um, you know, based on that one in the sense that only large companies can deploy AI models and AI because of the cost involved. 29:57 So instead of read-- Let me, let me see if I can read the: How can institutions address equity concerns when implementing AI technologies, particularly in managing the substantial costs associated with running advanced AI models for applications and workflows? 30:11 So again, to ensure accessible and inclusivity. So Paul, uh, you, you are helping companies deploy, you know, on the cloud, uh, and then helping them get, uh, the AI there. 30:23 But, uh-There is-- I mean, I myself have seen in the, in the cases with the cloud, the cost escalates, you know, could escalate significantly unless I plan properly. Right. 30:34 So what can you advise the companies about managing this cost as they look at, uh, AI and AI in the cloud? Yeah. Uh, first I want-- I do wanna differentiate what cost means. 30:47 So the-there's the hyperscalers and the, the organizations that build these large lang-language models, you know, like Facebook, Meta, you know, Microsoft. 30:56 That requires a lot of cost to say, "Okay, I'm gonna build a foundational- Yeah -large language model to be able to run, you know, thousand GPUs." 31:05 I don't expect everyday users or small organizations to be creating these foundational models at all. I mean, it takes deep pockets, millions and millions of dollars to do so. 31:13 But instead, we are now the users of that service- Mm-hmm -you know, using, using API call, for example, to say, "Okay, I'm sending a prompt, process, and give me something in return." 31:23 So I see the, in that regard, there's me-areas to optimize and keep your costs low. 31:29 And that's where people like myself as a cloud solution architect come in, is to say, "Okay, these are areas where we could reduce your cost potentially against your AI models, depending on what type of tasks you want to do." 31:39 So definitely there are strategies to use. Uh, I, I, we do have a best practices guide by Microsoft on that type of strategy as well. And additionally, there is a report that's coming out next week. 31:52 It's called The New, uh, New Future of Work, that's really designed... It came out, uh, it's the fourth, uh, fourth release. 32:02 It started during the pandemic, and it- Mm-hmm -was really designed for, okay, how do you do remote work, hybrid work, and optimize, you know, existing, leveraging existing AI, is now morphed into now, because AI is so prevalent in integrating AI into your work solutions. 32:18 Things like AI assistance and Copilot that Microsoft, you know, is putting into all applications, those are, you know, free of charge, almost reduced cost, if you would. 32:27 It already has AI built into all application services. Now, if you're creating your own applications and your own services, yeah, the cost may be a little bit higher, but again, there's, there's ways to optimize. 32:37 Not-- You don't necessarily need to create a small, s-small language model or large language model from the ground up in that regard. So... Okay. Got it. 32:47 So, Kavita, so looking at from your perspective, from your experience, uh, what you're noticing, how are people, um, thinking about managing the costs in using AI? 33:00 We're not talking about the building the foundational models as Paul, you know, you know, made the distinction, but about, uh, the question is more about the using this, uh, AI. Right. 33:12 I think one thing that is coming from the field itself is does everything need to use AI? 33:17 I think that's a question you'll see coming up more often in twenty twenty-five also is like certain things are just an Excel formula. Excel is good at it. Does it really need to use AI? 33:28 So institutions or businesses or, you know, uh, they do have to think critically about their workflows or their use cases, in other words, right? 33:38 Which part of those workflows and use cases are, you know, manual and intensive and error-prone that can be offloaded to, um, you know, by using a generative AI call? 33:52 So not everything, I think our view is to keep your costs down is not everything needs an AI. So we do think hard and critically about like, does everything need to run using an API? 34:04 Can you actually, is it just plain old simple Excel formula that you can run, right? Um, that's a sure shot way of kind of keeping your cost down, um, and use it judiciously. 34:17 I-I mean, it is a powerful technology, so then you want to use it for the higher order task, right? For the complex, uh, problem-solving tasks. Okay. So, uh, now we have, 34:30 uh, one last question in this section, and then I'll get into, uh, the concluding section. The question that I have, um, is, Paul, I think you alluded to this one. 34:40 There are best practices that are out there, and then there's also a future of work, uh, you know, document that's coming ahead. 34:47 So what have you seen as the common threads, um, in-- with respect to best practices where the companies that have successfully implemented AI solutions? So they've gone from experimentation to deployment stage. 35:01 So but what are the threads that you're seeing that they have done right in getting past this stage and getting to the deployment stage, actually have a successful deployment? Yeah. Great question. 35:13 So one of the things I've seen large organizations do is implement, you know, age-old things like caching, for example. 35:20 And what that means is if one user submits a prompt to an AI model- Mm-hmm -goes on the cloud, there's a cost associated with that, right? 35:28 However, if you're caching that, and the next person asks similar or same question, you're no longer having to make that API call. S-And so thus, now you're looking at your local cache instead, thus reducing your cost. 35:40 I think implementing a good caching strategy, especially around, uh, prompt caching, is, is both the cost savings as well as reduces latency as well. So that's one strategy. 35:52 Another strategy I've seen successfully deployed in organizations is the, the ability to, again, load balance. And that's mostly for real-time, real-time responses required. 36:02 Not everybody needs real-time responses, so you want to i-i-in that case, you could batch as well. So there's something called batch inferencing. That'll reduce your cost as well. 36:13 So what you're doing is now you're sending prompts or, you know, inter-interaction with ML, uh, or AI models, send it to the cloud based off of a, a reduced time, based off of a schedule, likeTake, you know, could take a potentially twenty-three, twenty-four hours to respond. 36:29 That's okay. It might be part of a report that you may not need real-time or immediate action on. So that's a combination of, again, reducing your cost as well as just optimizing the o-overall service. 36:42 So those are two strategies I've seen that work successfully. Right. Hey, Kavitha, you started talking about, hey, use cases. 36:51 Identify the use cases and the workflows that are repetitive and that are, uh, you know, more, uh, prone for labor, and then maybe start, uh, automating those. 37:01 So Kavitha, is that, uh, one of the, um, things that you have seen successful co- uh, companies have done? 37:09 Start with your, you know, the business case first and identify them and then start, uh, you know, using technology towards... 37:16 So could you talk a little bit about what are the things that, uh, the successful companies have done? Uh, definitely. 37:22 I think in my view, the successful companies that have really built something out there, they identified a pain problem, right? 37:29 With the customer or whatever that their customers were requesting of them, and then decompose that into a set of tasks that they can, you know, uh, repeat and automate that workflow without going for off-the-shelf solutions. 37:44 Mm-hmm. 37:44 And they were able to kind of create, you know, um, parts where the parts that could be offloaded to an AI model could be offloaded, and then the parts that they could keep it in-house, then they could keep it, right? 37:56 So one example is around, um, there's this company that is using, you know, AI sales agents. Okay. 38:04 And then AI sales agents are great for making your cold calls and cold emails, but then the parts where you can do the research about somebody and then generate a transcript so that you can, as a sales, you know, account manager, you can craft a better email by yourself, not using AI. 38:21 Mm-hmm. But you have the research at your disposal to craft a better email. So those are the parts that you can, you know... You always have to start with the business use case and what problem you're trying to solve. 38:33 A lot of what we see in the earlier stages of technologies, you go around with the tool and say like, "Okay, I can apply it everywhere." 38:41 And now I think with the budgets being tight and then the businesses trying to understand the ROI of all of this effort in this technology, 38:52 uh, you know, you have to go back to the drawing board and saying, "This is where it, you know, helps, and this is a low-cost experiment to show that it helps." 39:00 And then the access to low-cost experimentation is, you know, the best that it could have been right now in our lifetimes, right? 39:08 So that low-cost experimentation is possible to prove value that, hey, I'm able to save time or- Mm-hmm... save money by, you know, um, solving this repeatable use case using AI. 39:22 So continuing that thread, uh, so Kavitha, so we'll get into the last section here, looking ahead, uh, into the future. This is always dangerous. 39:31 Um, so in twenty twenty-one, I remember like, uh, October or so when, you know, started looking at what's going to happen in, uh, AI and then, you know, we could only do linear predictions and then, you know, lo and behold, uh, you know, that end of that year, so the ChatGPT came OpenAI and said just exploded the whole GenAI, so all predictions went out the door. 39:53 So that risk is there. So I'm gonna tell you, both of you, right? So, but y- you're, you know, thought ex- thought leaders in this space, so you, you, you should take the risk. So Kavitha, let me start, uh, with you. 40:04 So what should we expect in twenty twenty-five? Great question. I think security is a risk, and I will say security in terms of when I'm looking at the model results daily and what these models are serving customers. 40:19 Mm-hmm. There are times and occasions where the model just spits out something wrong and you know it is wrong because you're the human and you're the subject matter expertise. 40:29 And how-- What keeps me up is like how do you continuously train or fine-tune the model to make sure it aligns closer to the truth. 40:39 So in twenty twenty-five, I-- we will see more of these models or more of the frameworks trying to adhere, like get the models to stay closer to the truth. Okay. 40:51 And you will see reduced hallucinations and so on and so forth. Um, from a security standpoint, these risks are manageable, I would say. 41:01 So then you will see a lot of these evaluations and checks and balances before these models go out to production so that somebody cannot just, uh, you know, get a bad user experience or, you know, worst case, like you don't get into, um, you know, fraud use cases. 41:17 So those are the two things that I would like to see is like, okay, your security hardening around the models and your, um, evaluations. 41:24 Like, okay, how can you make this model actually be confident in its answers and not just act confident in its answers. A lot of time these models are acting confident, and you have to be a little bit, what do you say? 41:38 It's not-- You're not going to be, uh, able to outsmart these models, but what are the guardrails that you can place, put in place to evaluate, um, the outputs of these models? 41:48 So just minimizing the risk of, uh, something going wrong- Yeah... because of a confident model just so that, that it's more, uh, you know, manageable. That's what you're saying. Okay. Got it. Yeah. 41:58 So we'll, we'll keep, uh, going back and forth between you and Paul. So that-- Okay, that's one of the things that you, uh, uh, you think we'll see. 42:06 So Paul, so from your perspective, so what are the, uh, you know, top, uh, things that y- you are expecting to see in twenty twenty-five? Yeah. No, absolutely. 42:15 I wanna echo what Kavitha just said because that was one of my top items as well. Mm-hmm. But I'll just-- Uh, o-one thing I-- we will see more of is- If two people agree on one thing, that probably will not happen. 42:24 I'm just saying. [laughs] Well, well, we already know even for n-next ChatGPT five, for example, we already know that it's gonna have better reasoning skills. 42:32 So that is better logic and better conclusion, but it's, it's a double-edged sword, I think, in that regard and as Kavitha alluded to.It w-we can't also over rely on AI, especially when AI then gives us, uh, references to possibly, you know, papers that where it thinks the information has been interpreted in a certain way, then, you know, normal typical user might think, "Okay, this might be factual." 42:58 Mm. So I think that's something we have to be careful of. Although the reasoning is going to expand and be improved, at the same time we always have to be careful to have the appropriate reliance on AI. 43:09 Like, don't over rely on AI. Yeah. Yet, you know, also be, be vigilant on and, uh, pay attention to the output as well. So- Great... that. 43:19 The other item I wanted to mention, if you don't mind, was, uh, more against the agentic AI expansion. 43:25 Again, the multi-agents that we previously discussed in 2024, I see that's phenomenal growth in 2025 as well, where these agents may also have new capabilities such as more empathy, possibly, so more, you know, personable traits as well. 43:42 Um, the ability to, uh, have... under- identify your individual habits also. Mm. We're starting to see some of that with, you know, Apple mentioned, uh, with their Apple Intelligence now. 43:53 A lot of integration of AI in, in, in the mobile space. Yeah. And really trying to understand your behaviors and h- and then the AI is gonna react to that in that regard. So- Mm-hmm... yeah. That's great. 44:06 So again, uh, to audience, uh, it, with the last section, so this is your probably final chance to send in your questions, uh, if we can, you know, address them and catch them in time. 44:17 And otherwise, if you cannot, of course, we'll, uh, I'll get back to you with answers afterwards, but this is the time for you to put in your questions. 44:24 So with that, uh, the cynic in me says, right, so with AI, the way, um, I don't know if it's driven that way or not. If I look at evolution, initially we said these are the AI tools, right? 44:37 So a few years ago now everybody went and looked at that tool, this tool. I can, you know, create content, I can publish content, I can text to image, text to video, all that stuff. 44:46 So then afterwards then came the phase of GPTs. Hey, you know, you can custom GPT, you know, uh, so you can customize the GPTs and then companies and people started spending, uh, you know, spending, um, on GPTs. 45:02 Now, we are saying agents, right? So it's like, hey, you know, you can do that agent, and you can do this agent. So Kavitha, so i- the, the cynic in me says that we always keep saying tools to GPTs and agents. 45:15 Is agents another buzz thing that everybody will get on 2025? Or, um, is there a real benefit to this, this multi-a-agent, um, evolution that's happening right now? 45:29 Uh, great question, and I feel like [chuckles] I think we are in the way where every year you need, um, like a, you know, marketing term to go around with it. 45:38 But keeping the under-- you know, what the tool is called, we are in the middle of a revolution or an evolution from a technology perspective where 45:48 there is a lot of compute that is being pushed on, uh, from the cloud to your devices to automate your daily tasks. We had like a massive, like last ten, fifteen years proliferation of software, right? 46:02 Everything was software. But you have to just get to one task, you would have to use ten different tools to get to the task. 46:10 In my view, what 2025, these agentic workflows or agentic AI, if you may, is going to reduce that surface area, and my wish list is that you would have one, you know, on device, on a, you know, your mobile phone agent, whether it's Siri, whether it's, you know, uh, who it is, to be able to finish that task for you. 46:33 That's how I see the, you know, evolution from a technology perspective. 46:37 Um, as far as from a, you know, broader question, to address your broader question, yes, there is a lot of buzz in, in the market about what's the next technological change going to be. Mm. 46:47 And even if we kind of almost say that AI has hit a wall and we pause, the rest of the world outside of the big tech has a lot of three to five years of just pure adoption of the stuff that has been released in the last two and a half years. 47:04 Yeah. So I don't see there is any time like, you know, um, a dearth of like jobs or workflow, you know, any kind of work not stemming from this just, you know, evolution that has happened. 47:17 So I see a couple of questions around the job side, right? Yeah. There's plenty of work to go around in the adoption and application space. Um- Actually, that's good. 47:25 Let me state the question, and then you can continue on that one. So somebody was asking, "For a non-technical person like me transitioning to tech, will 2025 have more or less jobs available for me in AI?" 47:37 So Kavitha, so proceed. Yeah, I think you're getting to that question. Yeah. So great question, right? 47:43 I mean, we are seeing a lot of this tech just come out and spit out from like a fire hose, but there's not a lot of people on the other side being skilled and retrained and upskilled to actually start adopting and using the tech. 47:56 And I do feel that's a promise of 2025, where organizations do have to think and actually invest in retraining their staff, right? And every job is going to become an AI job, if it hasn't become already. Mm. 48:11 You do need to understand how to work with these models. You do need to understand how to use these models, not only in your daily use cases, but also your professional use cases. 48:22 Um, and I state true the barrier to learning has reduced drastically. So yes, there will be, you know, more AI jobs, uh, whatever that means, because I feel every job is going to be an AI job at some, to some degree. 48:36 Um, even if you were a farmer, yes, you do need to use some kind of capability to understand your yield, you know, given climate change. Um, so-There is hope, um, definitely, and, uh, it's time to learn. Yeah. 48:53 So Paul, um, that question as well as the next question, which is: What advice would you give to a person who wants to be an AI analyst? So if you could address both of those questions, Paul. Yeah, absolutely. 49:04 So to echo Kavitha's comments, I definitely see continued growth in AI. 49:09 I mean, it, it's, it's quite transformational at this point in the last couple of years we had, and that's only gonna continue in twenty twenty-five. Um, you know, the... I referenced New Feature of Work. 49:19 It's a report that Microsoft will be releasing on Monday that-- And I'll put the URL, maybe we could share. Yeah, please. That does go, go into details about just how existing labor is being challenged and modified. 49:32 Challenged to a degree, for example, web developers, application developers, how that their jobs are being morphed. 49:38 It's not necessarily being replaced, but now you're augmenting your work using AI, so that means faster to market, you know, speed, uh, reducing costs as well. 49:48 So definitely there's approaches that are mentioned in that report. Um, so again, growth in AI, definitely if you want to be an AI analyst, there's jo- lots of job opportunities now. 50:00 I remember just, uh, earlier this year and late last year, prompt engineering, that was one of the top five jobs, uh, you know, that were being seek, seeked on Link-LinkedIn. 50:09 And so if you're a prompt engineer, put that in your, in your profile, you're definitely going to be reached out by a [chuckles] recruiter. Um, so again, that space. Some areas that we do see growth, uh, changes is, uh, 50:22 such as, you know, report writ-writers, um, people that are, uh, summarizing documents. 50:28 Because a lot of, lot of those tasks, lower-level tasks maybe perhaps as well, are being replaced by AI, but there are still people that need to manage those applications and services. 50:39 Like, okay, how do you properly prompt, or how do you summarize documents, or how do you create these agents that we are hearing about? So definitely continued growth. And what was the second part of that, Ramesh? 50:51 The second question was, uh, about, uh, um, what advice would you give, um... And actually, uh, would there be more jobs? I think you already addressed both of them. Yeah. So that's fine. Yeah. Okay. Okay. 51:02 So now, um, so let's, uh, so start wrapping up. Kavitha, so we covered, um, with, with respect to, you know, what has come forth in twenty twenty-four, um, it almost like a drinking from a fire hose. 51:15 A lot has, you know, uh, come forth. And if I especially combine twenty twenty-three and twenty twenty-four, it, it's a lot, right? So has it changed the playing field? Yes. 51:25 I think both of you agreed that it has changed the playing field. Um, and so Kavitha, if you could just summarize the discussion and then provide any, uh, new thoughts that you have going into twenty twenty-five. 51:39 Um, definitely. I think it's been a great discussion, right? There is a lot of, what do you say, um, cynicism in terms of around like, will AI hit a wall? 51:47 But I think we all agree that no, we are at the, in the middle of this transformation, and it's just at the right, you know, stage where it's about to take off. 51:56 So as an individual, what you can do right now is to really take things into perspective. 52:02 You know, do what it takes for you to kind of, uh, you know, learn about the technologies that have come out, how does it impact your area of expertise and your area of, you know, uh, interest, and then start planning your future to learn and self-invest in your career to be future-proof using this technology. 52:23 Did we do that as humans before? Yes, we do that. We did that with the internet. We did that before, uh, before the internet. So we can do that. 52:31 So I do expect a lot of, you know, certification programs, training programs, that even corporates will invest so that people can start adopting the technology. 52:41 And I do feel the next one to three years will be a lot in the adoption and the rollout of this technology to actually uplift and, and get the productivity benefit that it claims to have. Great. Great, Kavitha. 52:56 And, uh, with that, uh, to Paul, um, so if you could just bring it all home. Yeah. I mean, for businesses, uh, uh, we're gonna see growth in AI, definitely. There's things, improved tooling that, that I've mentioned. 53:10 Uh, every organization is looking at, okay, providing tools for everybody to democratize AI and leverage it. Lower the cost as well. 53:18 Every time we've seen new models come out, the generally, the inferencing costs have been dropping, uh, as well. So I expect that trend to continue in twenty twenty-five also. Um, not to mention enhanced security. 53:30 So one, an important aspect, everything is, again, enhanced security. With, with AI, now there's AI-powered threats also. So that means- Mm-hmm... like AI, you know, potentially c-can be, can be a threat. 53:44 So how do you combat that threat? It's with AI. So you're gonna now... We're gonna see more modeling and tools that are have the security, AI-enabled security built in out of the box. 53:54 So I think all of that combined together will help business and organizations be more efficient. Um, going back to the agentic AI and multi-agent topic, I know this is a hot topic. 54:03 I did wanna mention that there's a few authors out there that have already stated that the next future billion-dollar companies are going to be created by a small group of individuals who are running these agents for their product or service. 54:17 That is, the multi-billion dollar companies are gonna be created based off of agents [chuckles] and not, not necessarily the large number of personnel. So that's something to consider. Great. 54:29 I think that's maybe five years down the road, but that, that is the future that we, we envision. Yes. Hmm. Yeah. So that's great. So thanks, Paul. 54:37 Um, again, uh, to people who are trying to get into AI and then looking at it, yeah, it might seem very overwhelming. Uh, one of the things that I have learned is whatever job you're doing right now, right? 54:51 So you are more than likely you're competent and you know more about what you're doing, whether it's tech or non-tech. So that is your starting point, right? 55:00 So just take that expertise and then build, you know, one thing to it, right? As opposed to trying to completely shift, incrementally add, um, you know, some... 55:10 Like for example, if you're, you know, a marketing person and see if you could use AI tool to, you know, uh, do abstract, right? 55:19 So then that way you get to know the tool, you get to know AI, what it can do, and then keep building on it, uh, as opposed to, uh, trying to change completely. That could be very overwhelming. 55:31 That's, uh, one thing that I could say. But, uh, there's no looking back. I think both Paul and Kavitha made that very, very clear, is so much is happening and so much investment is happening. 55:40 Uh, it's there, you know, for real, and then at different stages with experimentation to pilots and proof of concepts to deployments, but it is happening. Um, so, uh, learning is, uh, something that we have to do. 55:55 Uh, there's no going back on this one. With that, again, um, so please give your feedback to s- today's session. So, uh, just literally takes two minutes. 56:03 And, uh, thank you very much for attending and look forward to the next session. [upbeat music]