Hala Nelson on human machine coexistence, ontology first, AI driven digital twins, and bidirectional connections to reality (AC Ep45)

Transcript

Ross Dawson: Hala, it’s wonderful to have you on the show.

Hala Nelson: Hi Ross, how are you?

Ross: Very well, thank you. So, we live in a world where we thought there are many, many things that humans were uniquely capable at, and AI has proven that it can do quite a few of those things. So, what do you think remains uniquely human today?

Hala: Well, it’s upended the belief, right. So, let’s start with what we thought was uniquely human. It was always language and mathematics and reasoning across disparate sources of information, for example, data modalities, knowledge domains, and we were the only species on this planet that was able to do that. Some people argue that mathematics evolved together with language, even though I stand on the feeling that it is actually out there and just a constant in nature.

But then machines came, and AI came, and it upended the whole belief of that, and our whole systems and institutions and processes and jobs were built around the fact that these three things were only human, and machines were there to help us accelerate, but not replace those functions. So that’s where I stand.

When I first wrote my first book, I thought a lot about the nature of intelligence and what aspects make us intelligent, and which of these aspects machines can replicate really well. So I started going through them one by one: perception—machines can perceive now; communication—machines can communicate; vision, as part of perception—machines can do that; reasoning—machines can do that; memory—we’ve had memory forever, actually much bigger than ours.

So I went through everything one by one, and we could mathematically model with a machine and create machines that can do all of these aspects. And then, finally, the only two things that I was left with that machines have not replicated yet are consciousness—we don’t know its nature, and we don’t know if they’re conscious or not, and I don’t think they are—and reproduction. These were the only two things. Even empathy, machines can mimic now. So that’s where I stand on the human-machine coexistence at this point, especially when they become embodied with robots, a lot of them.

So I’m standing on how do we structure our world in a way where we have human-machine fluency, where we’re augmenting the human condition as opposed to threatening it and eliminating it, or upending millions of jobs without the ability to replace them. So that’s how I went down the journey of human-machine coexistence.

Ross: So, you talked about the complementarity. Given there’s so much that AI can do, or has the potential to do, which humans have done, how do we design a world where they are complementary, where humans are augmented, where humans have their role in conjunction with the machines?

Hala: I am on the optimistic side of that equation, in the sense that I never believed that humans were meant to do the kind of work that the post-industrial revolution put us in. Entire generations were trained to serve the machines, whether it’s on a line in a factory or doing data entry for all the software and ERP systems, CRMs, the SaaS applications—everything we’ve created. It’s us trying to mold ourselves into the machine, but now we have the ability, the fact that machine speaks human.

So now we can reverse the paradigm and have the machine serve us and accelerate most of these mundane tasks that we don’t want to do, that we were never meant to do. About 20% of payroll jobs never really serve the goal of the organization; they’re just there, and nobody knows why, and maybe it’s time to reassess all of that.

How do we start with all of this? We start at our own level, look at our own workflow, and see where AI can help, and we start there. Then, on the organizational level, we can start bottom up with the teams: what kind of use cases do you want to own with the head of the department, what kind of AI tools you’re going to allow, what kind of data you’re going to allow to touch. At the executive level, what kind of policies about AI you’re going to have, what kind of budgets you want to allocate, who’s doing what with what tools.

So my view of how we propagate AI into society and the workforce in a productive way is to start with the most beneficiaries—us—owning our use cases and planning around them.

Ross: I think that’s the frame. In a way, it’s all in our attitude, and if we start with what is beneficial, I think that’s where we’re likely to be heading in the right direction. One of those key distinctions is: what are the jobs we want to do, what are the jobs we don’t want to do? If we don’t want to do them, then we can work out how to automate them, and then look at how the organization changes.

You have deep expertise in enterprise modeling, and I think there’s a lot there I’d like to dig into. One of the old paradigms—actually, Karl Weick, who just passed away, unfortunately—looked at organizations as sense-making mechanisms. I think that’s one of the parallels: humans, individually and collectively, are sense-making; we make sense of the world, and enterprises as well, with that combination of humans and AI. So I think there are analogies to the ways in which humans think, which aid the AI systems built.

But I love this phrase you use, which is “ontology first.” I think that’s just a nice entrée into thinking about the structure of how enterprises function. So, what does ontology—well, actually, just for those who may not be fully familiar with it, what’s an ontology, and what does “ontology first” mean?

Hala: Ontology is the definition of what exists and how it relates to each other in any system that you can afford. So if your system is an organization, it will be the entities, the people, the departments, the processes, the tools—everything that there is, and how it all talks to each other. It’s a beautiful concept. I think our intelligence is based on us building ontologies in our head, and that’s why we’re very fast in categorizing context and getting it right most of the time. So that’s what an ontology is.

In terms of enterprise modeling organizations, the reason I go with an ontology-first approach—it wasn’t an immediate decision. I had to think through what is the best approach to have a robust organizational digital twin and model that actually serves the organization, helps it accomplish its mission through the work and execution that’s being done on the ground.

So I looked at the AI paradigm, and I noticed that it has different levels of maturity. For example, the documentation, the language processing part, the summarization, the mapping of documents to compliance to laws, the document generation—all of that is really good and is being used all the time, and in a quite impressive, and I dare say reliable, way at this point. But then I thought that to get actual real-world decision making, you have to have that AI not just on top of some documents that you release it on, or some knowledge base that you guide it to. It has to be connected in real time to your systems, and that’s a different level of maturity, a different level of engineering that needs to go in it.

So I thought that for that, the AI needs to understand your organization in two ways. One way is static: exactly what its structure is, what’s the organizational chart, what are the deployments, what are the functions in each deployment, what are the roles of each function, what are the tasks. That is static—of course, it changes, but not as fast as the real-time dynamic part, which is the actual work that’s happening, the processes, and how all of these talk to each other.

So I realized that the more structured parts are easy ontology—that is your organization: who’s in it, how do they interact, you have to create that ontology. But then you also have to, in the ontology, include the people of the organization—the workers—and the workers could be human or digital workers, like the AI.

Now, before AI, we could not model processes and actual work that’s happening—the dynamic part, the execution part of the organization—very faithfully, the way we do now with agents. These digital workers are able to model an entire process, connect to all the input and data, whether it’s people, operators, signatures, authorizations, uploading the budgets—all of that can be modeled very nicely with an AI agent, actually easily. The nice thing about the AI agents is that they will end up as entities in your ontology.

I’m going to say the difference between ontology and knowledge graph in a minute, because it’s actually a knowledge graph when you instantiate it—that’s the technical difference, and I need to clarify that for the audience. The thing about the agents is when you model them, when you deploy them in a governed way in your organization, you can immediately start measuring signals that connect to KPIs, that connect to the department, to the department outcome you want, that connects to the mission and strategy. So basically, this top-down static view combined with the bottom-up agentic view of an organization, combined in a computable way, allows you to really see through your organization—it’s like an MRI image at any level—and allows you to do simulation, to highlight duplication, to highlight inefficiencies, to compute these things in a way that you were never able to do at a large system level with live data and at scale.

So we are at such an exciting time that we can connect the mission, vision, strategy, functions, roles, tasks—all the way down to the execution layer—which removes all of the inefficient middle layers that exist just to communicate between the top-down and bottom-up. All those inefficiencies live in the middle between where you need to be and how you want to get there, and the actual execution on the ground.

Ross: So the ontology, because it links concepts or entities through relationships, then provides the link between what is static and what is dynamic. But one of the aspects, as you suggested, is the agents are part of what is able to track and monitor and then to populate and then modify the ontology, which remains at the heart of this system.

Hala: Exactly. And I do want to make the difference between ontology and knowledge graph. A knowledge graph instantiates the ontology, meaning it could be an ontology for the whole industry and the knowledge graph for this one organization in that industry, another knowledge graph for another organization in the same industry. So basically, the ontology is more on the meta level, the higher level, and each instance—each organization—has its own knowledge graph. So the agents and everything are in a knowledge graph, usually talking to that specific organization’s instance entities—what exists and how it talks to each other.

Ross: You’re suggesting the ontology is generalized rather than organization-specific.

Hala: It is industry-specific, I would say. Think of it as, if you have the energy industry, say you have the power stations, so you have this facility has its own knowledge graph, this facility has another knowledge graph, but they are similar, they’re governed by a similar ontology. Their knowledge graphs that are guided by the ontology might differ a little bit between each other, but they mostly follow the ontology of the industry. So, a knowledge graph instantiates an ontology, but it could be slightly different. This facility’s knowledge graph is similar to that, but they follow the same ontology. This is bigger, and these are the same.

Ross: Yeah, it’s worth drawing out, I suppose. This idea of the structure of the ontology is the relationships. So in the energy industry, for example, you’re mapping the entities of all of the ecosystem in which the organization sits, the activities, the flows between them, the relationships between them—the whole structure of how value is created.

Hala: Yes.

Ross: And it’s, I think, so Palantir famously provides their ontology as a layer, and a lot of what they are selling—and maybe using the language a little bit differently—is, “We will create the ontology for your organization,” and that becomes structure, and obviously then they are embedded at the heart of the organization. So it’s actually worth thinking about that process, because ontology is extraordinarily complex, the knowledge graph for an organization is itself complex and unique, so it requires creation. What is the process then for an organization—and that’s not something you can do immediately—so what is the process to build the ontology or the knowledge graph for your organization that will enable it in the way that you describe?

Hala: I would say, do not boil the ocean all at once. Start with what you want to twin first, because ultimately the ontology is twinning what exists, and then start with your most pressing pain points, and then go from there. These ontologies connect to each other, so you can have a mini ontology of where you can allocate resources to twin, and then the next one, and the next one.

Now, with AI, you can populate these much faster. You can scan some environment and the AI already has categories of what might be there, and then it recognizes it. So AI automates a lot of that and makes it much easier—instead of it being a one-year engagement with the organization, we’re hoping to bring it down to a few days. Ultimately, you would have the ontology of the industry, and then you just go and see what part of that ontology exists to populate the knowledge graph for that particular organization, and ultimately it should be automated, this discovery process.

Ross: So this will, of course, relate to any existing systems for the organization, which include the ERP or any other systems, because you are pulling their exact existing data structures and then being able to map or relate your ontology to those.

Hala: Correct, correct. And even from public information on organizations, like org charts, what they do, their job roles—you can ingest those into AI and see how it’s structured. So there’s a lot that we can do with AI now to accelerate so much of these engagements, and the whole idea is that you don’t want each engagement to be a million-dollar exercise. Otherwise, you don’t have a product. You don’t want bespoke engagements; you want to make your digital twin product repeatable and scalable.

Ross: So there’s another phrase which has been very popular in the last month, which is “context graph,” and I think part of what that point is, you know, the full context of the organization, but particularly pointing to what is tacit. So, of course, anything which is current digital structures, or which can be listed as fair, but the idea of the context graph is you are eliciting the actual ways decisions are made, or the actual context in which work is done. You talked about this before. So, how do we get that reality of how work is done, as opposed to the way it’s documented or captured in software processes, in order to build an ontology which does reflect the full organization?

Hala: For sure. So, I have two views on that. One of them is, in the past, before AI, everybody would be stuck with entity resolution—like, what’s a customer? Is it this person, or is it from this database, is it the person who bought this, or so on. With AI, because of the knowledge graph, because of the context around it, AI is able to disambiguate a lot of these things that used to be done manually in the past. The graph gives you so much context that it helps the AI, it helps even a human, on what’s what.

The other thing is how work is being done. Again, it’s the AI agent. If you have a process that touches multiple departments at the same time, multiple databases at the same time, multiple systems as well—like ERP and CRM, and some SaaS application—the nice thing about AI agents is that they can connect to all of these tools, and they can reason on top of these tools, and then they can produce the output and the signals that you want from these tools. It’s as simple as connecting the agent and allowing it certain privileges. Of course, that exposes a lot of security risks and governance issues that have to be engineered into your agent: identity, authority, whether to use public facing or not—all that has to be identified. Then the agent is able to emit what you want it to emit, and also quantify how much time is saved, how much money is saved, or how much risk is reduced.

There’s a lot that, because you’re able to connect so many things and reason about them that you weren’t able to do before, it’s actually—if you think about it—what changed after. So ERP is like a digital twin of the business operations and the finance systems, operations, processes, etc. That’s ERP. The CRM is a digital twin of customer relations. The SaaS is a digital twin of actual physical processes. You might have 14 or 15 in your company, but each of these moves the bottleneck from one thing—before we had them, like manual, like paper, maybe documents, etc.—to something else. Now you have to enter the data. Now the data exists, but now you have to generate the reports, you have to get it from the different databases and generate the report manually yourself, and you have to write to the customers yourself after you’ve seen all of these CRM tickets and all of that. So that moved the bottleneck from one place to another. That improved a lot, but it moved the bottleneck. But then the agents come and connect all of that very easily, and then they can reason with it, and they generate whatever you want in split seconds. So in a way, it completes the story—from digital twin to AI-powered digital twin.

Ross: Okay, so you have a book coming out. What’s the title?

Hala: AI Powered Digital Twins: A Guide for Humans, Engineers, and Enterprises. And then a lot of people tell me, “You make it sound like humans are not engineers, engineers are not human.” I said, well, soon a lot of the engineers will not be human. So this is a guide for both humans and machine and enterprise.

Ross: So just coming back to that digital twin idea, but in this case, because it’s in software, it is still not the full twin, I suppose. So what is the gap between the reality and the way it’s instantiated in an AI model, and how do we reduce that, or is that important? Obviously, there are differences across what you are modeling, but what is that relationship between the digital twin and the original, as it were, or the real-world instantiation?

Hala: Excellent questions. I would say, when people think of digital twins, they think of an AutoCAD rendering, like a three-dimensional rendering of an engine, or they think of a dynamic simulation of, say, weather models, or they think of a process—like, how do you pay with a credit card, and how do you get a reimbursement through a certain system—that is one process. When they think of digital twin, they’re thinking like one of these three things. Or actually, they do think of digital twin of the human, the actual application of mules, digital or the robot—that’s also another way. These are all the answers I’ve ever gotten from people when they think of digital twins.

What binds all of those on the data architecture is that what we want is a virtual replica of an object, a human, a physical process, an organization—and here’s the very important point—that’s bidirectionally connected to reality. That’s exactly what makes the difference between just a rendering of an engine, as opposed to seeing the engine in real time, how it’s functioning, what’s happening to it, is it overheating, or is there a blade that’s out of place. That’s the difference between a digital twin and a live digital twin.

What’s different, engineering-wise, is the live back-and-forth data feedback that we have to engineer through APIs and standardized protocols and all of that. So that makes it live. What makes it AI-powered is putting the intelligence on top of it—the reasoning, whether it’s reasoning through knowledge graphs or reasoning through language or both together. That makes it AI-powered, and that makes the feedback cycle hopefully faster between the two.

Ross: If you’re looking, for example, at an enterprise process—some work has been done, involving human and AI agents, decisions being made—when you have the digital twin of that process, which is happening, that has access to the full knowledge graph of the entire organization. So it has that full context of the organization, which can come back and feedback to reflect in that specific process, which is happening at that point.

Hala: 100%. It’s a back-and-forth connection, and you have to be careful that some agents should be less powerful than others—they can read, they cannot write, they can access this part of the graph and those databases, but not the others—because you treat them just like you treat human workers with different privileges. It depends on what your goal is; you get access to what you need.

Ross: So where can people go to find out about your book and about your work?

Hala: Oh, I am on LinkedIn, Hala Nelson, and my first book is Essential Math for AI, and the new book coming out in August is AI-Powered Digital Twins with Wiley 2026. They’re both on Amazon, they’re both available now.

Ross: That’s fantastic. It’s really, really important work you’re doing. So, thank you so much for sharing your time and your insights.

Hala: Thank you, Ross.