Melissa Clark-Reynolds on AI in the boardroom, improving judgment, creative friction, and foresight for agile strategy (HAI Ep59)

Transcript

Ross Dawson: Melissa, it is wonderful to have you on the show.

Melissa Clark-Reynolds: Thank you. I'm so excited.

Ross Dawson: So you work with, well, you work on boards and with boards and decision makers, and everyone's touching AI in various guises. So I'd love to hear at a high level what you are seeing and how board directors or boards are using AI.

Melissa Clark-Reynolds: Yeah, I'm kind of seeing three different things. One, I'm seeing the directors use AI themselves—often badly—to summarize board papers or to look for insights in those board papers, and we can come back to that.

The second one I see is that they're governing AI, but mostly what they're doing is stopping the use of AI by putting more and more rules in place or not funding the appropriate tools to the appropriate teams at the right time.

And then the third thing that I'm seeing is that they have an overconfidence in the ability of AI to reduce headcount in the organization, and they don't understand that AI is a capability multiplier, not a headcount reducer. I wrote a Substack on this a little while back, where I looked at whether or not AI was genuinely creating productivity, and whether that productivity led to job losses. And what I could find from all of the available data was that basically boards had bought a story around productivity ahead of that productivity. The senior team kind of pitched to them that we could lay off 10% of the staff or something and do it all with AI, and then that hasn't worked out very well.

Ross Dawson: Yeah, there's a number of recent cases, including Klarna and Spotify and Commonwealth Bank, where essentially layoffs have resulted in rehirings, and you know, those are—it's interesting. What I find is that board directors get a whole bunch of professional service firms swanning in and telling them all these stories about AI, and that's their source of knowledge. So it's not necessarily the best source to understand where your business can go,

Melissa Clark-Reynolds: and I see the same at the government level. I see ministers of parliament who suddenly get Copilot because it was kind of what the government bought—it was free in their Microsoft package—and they're now using AI to do their emails and to answer each other's emails and maybe to summarize cabinet papers or something. And so they think, well, I'm being much more productive, so we can roll this out across the public service and kick out 10 or 20% of the public service.

So we're seeing it not just at company director level, we're seeing it at national governance level as well. They've bought this hype from vendors or from consultants, and it's just not working out so well.

Ross Dawson: So let's go back to the governance piece. And to be fair, when a director—you know, you have all these fiduciary responsibilities, and you try to say, well, what are all the things that can go wrong and can stop that? And it's not quite—it's easy to say, but also not quite so easy to do—to say, all right, well, we are going to be enablers of transformation. So, how do you think that can best be done in establishing parameters or governance that actually does allow the useful use of AI while still acknowledging and recognizing and putting guardrails around the risks?

Melissa Clark-Reynolds: Yeah, I think there's a couple of things. We could think first up about the use of AI in the boardroom, and we've had numerous cases—and you can think about those fiduciary duties—numerous cases in pretty much every jurisdiction in the world, where there have been board members who didn't read their papers, and then something's gone terribly wrong in the company, and the board members who've ended up being convicted for not fulfilling their duties have been the ones who didn't read their board papers.

We have a very famous case in New Zealand where an entire board got convicted except for one, and the one board member—not only did he read the papers, which the others didn't seem to have, but he documented the questions he was asking around those board papers, and he got them minuted. Now, he wasn't doing that to try and create a paper trail. He was doing it because it was good governance.

So step one to me is—I sit on a number of boards, which I think makes me a better board advisor because I do the daily decision making. And so I want to read every word in those papers, and I want to be all over them, and I want to ask great questions.

Now, I don't think there's anything wrong if you've got protected data and those board papers are not getting out of a sandpit, in asking the AI some really useful things, like: pretend that you are the best marketing manager in the world that specializes in financial services. What are the three questions that I should be asking the marketing manager from that point of view on these papers in the meeting?

To me, that makes me a better director. But what I often see is I just see my colleagues summarizing the papers and then reading the summaries, or I see my colleagues—one that did my head in the other day—where a fantastic, in my opinion, technical marketing team with great experience put up a marketing paper and an AI marketing strategy. A board member who's got great skills in finance and nothing in marketing put it into ChatGPT and said, "How could I write a marketing plan for this company?" And what he came back with was nowhere near as good as what they had produced, but he felt very confident about it because he knew nothing about the subject, and ChatGPT to him had answered his question.

So, long answer, I know, but step one is: agree in the boardroom how the board itself will use AI, what is acceptable, what's not acceptable, and I would think train board members in how to write fantastic prompts. Most of them are not getting that training. I don't see the professional associations for directors running prompt training. I think they're running policy sessions, but not actually how-to's.

And I'm probably never going to be a director without AI ever again. And the AI is never going to be worse than it is today. So as a director, it should be available to me and on me to learn how to use the AI myself.

Ross Dawson: Yeah, well, there's a lot of data which has shown that AI is extremely helpful for experts in their area of expertise, and far less so outside of their areas of expertise.

Melissa Clark-Reynolds: Yeah.

Ross Dawson: So just, you know, possibly I think, just to your point—whether it's formal or informal—that directors should be applying their expertise with potentially using AI to be able to pull out further insights or areas to explore, but not trying to use LLM outputs to try to be experts in domains which they are not,

Melissa Clark-Reynolds: or to redo the work of the executive. You know, the executive's there for a purpose, and the governors are there for a different purpose. So maybe there are ways of asking the AI how I could be a better director, rather than how the management could do their job better. And I think that would be the first step that boards should think about.

The second step comes back to your question about guardrails, in a sense. How do we determine for the organization what is acceptable, what isn't acceptable? And for me, that requires some quite big conversations.

I think that boards have got way too focused on cybersecurity, and cybersecurity is essential. And look, I've spent 35 years of my career in tech, so I am not diminishing the importance of cyber. But I feel like what's happened is that cybersecurity has become one of the main conversations around AI in the workplace, and it's the wrong leading question—not that it's not important.

So boards need to be having deep conversations with the senior team, and the senior team needs to be having those conversations with the organization around what acceptable use is, how we're going to pay for sources that we're using, what the privacy requirements are around data, what's the interoperability, whose data are we using, how are we using it. Those are principal questions to me that we need to be having long before we start to write, "You can and cannot use this."

And then I think we should also be spending some time understanding what specialist tools the organization uses. AI is not a monolith. It's not just one thing. I started working on AI in the '80s. I've been through five different kinds of code approaches to it. Working on the next one—you know, LLMs will have their day, and we're already working on what's called world building. There will be other tools after these current AI tools, so we need to be thinking at the philosophical level in the boardroom first, not the rule level. What do you think?

Ross Dawson: Well, in terms of all of those things that surround this—how do we build this complementarity between essentially the human judgment and the AI inputs? And as you say, AI has many different forms. I just don't do it so much now, but one of the introductory things I always say is, "This is machine learning, and this is generative AI, and this is how they're different, and they apply in different ways and different things," and so on.

Because I've seen so many people talk AI, and then they're just talking generative AI, whereas of course there are many other well-established domains that are far more relevant to many aspects that can be valuable to the business. So again, this is foundational knowledge, which enables those who are leaders to see where there's potential value. And as you said, the focus on cybersecurity—it's hard to disagree, but as you say, if it drowns out all the other things,

Melissa Clark-Reynolds: yeah,

Ross Dawson: which are also significant, then that starts to take you somewhere else.

Melissa Clark-Reynolds: I love what you raise there because I think again in the boardroom, that competence piece—there aren't enough directors who have deep enough knowledge. I think about these distinctions of what is precision agriculture versus physical AI, robotics versus machine learning. We're still using neural networks technology, which I worked on really in the '90s until now, so we're still using neural networks. I've still got some bits of the old switch code, which is where you wrote millions of lines of if-then code to make intelligent decisions.

So these old forms are still actually in place in many organizations, and often the governors don't know. And so there's something here too about governors getting really skilled up in the language, which is continuing to evolve, and understanding the distinctions, so that they're not just putting everything into one bucket.

Ross Dawson: So, broadening out from the boards to just more generally, leadership and expertise. The word "judgment" is used a lot these days, and I use it a lot myself. I think it's a really valuable word in terms of understanding what is the role of humans in the context which we're bringing to bear, and how we communicate and make decisions and so on. So human judgment is critical.

So I'd love to hear your insights into how can we use AI to complement, improve, or enhance our judgment, and how do we apply our judgment to understand when and where and how to use the AI tools?

Melissa Clark-Reynolds: I think that's a great question because it feels to me like there's a big difference between getting the right answer and asking the right question. A lot of discussion has gone on at the moment about how we prompt, how we engage with the AI tools, and if we're looking for an answer, I feel like we're really wasting the capability of these tools.

There's one tool I use a lot called Undermined, which only searches academic databases, and I really love it because built into the tool is a coaching tool that coaches me until I've asked a really good research question—not just a good question, but it has to be a research question—and I'm getting better at them because it's coaching me.

Then I've learned by using it how to prompt it to tell it not to synthesize the answers, to give me all of the points of view around a particular topic. Then I ask it to do me a table of the top ten people who have refuted the theory that I'm investigating, with no synthesis. And so what I can see is that that is improving my judgment because I'm canvassing maybe twenty points of view on a topic, and I'm asking it not to synthesize those, and so it spits me out tables of arguments and refutations to those arguments. That genuinely improves my judgment.

Whereas what I often see is kind of average users go and ask, "What is the best way to organize a national science system?" rather than saying, "Can you find me all of the research that's available on different models for a research system? Who has argued for and against them? How have they been evaluated? What are the major theorists? Who's the most published, and who are the people who argue against those people? And again, has it been evaluated?" So I could start to really dig in then and bring my judgment, but often I feel when people talk about human judgment, it's a bit like the Kahneman thinking fast and slow thing. We're using AI to think fast, not using AI to think slow.

Ross Dawson: Yeah,

Melissa Clark-Reynolds: makes sense. And so I want to improve my judgment by having the wisdom of the crowd, knowing I'm using the wisdom of the crowd, and looking for better questions to ask, not by trying to get to the right answer faster.

Ross Dawson: That's great. I've been running a series of masterclasses for leaders on applying AI to leadership capabilities, and we've had all these practical exercises. And what I've had to just keep on repeating is: the machine wants to give you an answer, and your job is to not go to the answer; it's to be able to say, all right, let's start somewhere, let's frame what is the question first of all, let's use the AI to help us understand what is the right question, and then we can start to move to the next step and so on. The machine always is trying to get to the end,

Melissa Clark-Reynolds: yeah.

Ross Dawson: and in some cases, that's great, but in anything complex, that's not the best way to go.

Melissa Clark-Reynolds: Totally, I love that so much. And the other thing I love with that is that we know that all of the LLMs are programmed to be amiable, and so what happens is that they're like, "Oh, Mel, that's a genius question," and then it finds ways to agree with me over time. And then the same—if I catch it out in something, I go, "Actually, I looked up that reference, and that reference doesn't say that," it comes back and it goes, "Great catch, let me revise that."

And I think that's the other thing: people don't realize that the AI is designed to suck up, you know. And so, when we come back to human judgment, if we have a friend who only ever agrees with us, there is no creative friction. We're not learning anything in that relationship. And I feel like this bit of how we use AI is we want to use the AI to challenge our own thinking in order to be better at doing judgment—not have the AI, which is what it's programmed to do, agree with me and find reasons to agree with me. And most people don't understand that it has a bias for agreement.

Ross Dawson: Yeah, or if you understand it conceptually, but still, when it tells you you're wonderful, then you tend to accept that. And I guess the "but" part is that this is an attitudinal thing. So you've expressed very well how you go about it, and a wonderful description of these ways in which you're using the tools to enhance your judgment, but it really needs people to have that attitude of not wanting to find the easy, cheap answer,

Melissa Clark-Reynolds: yeah,

Ross Dawson: and instead to want to enhance their judgment. So I'm not sure—how do we get to that root cause, as it were, of that attitude of how we want to use these tools to improve ourselves?

Melissa Clark-Reynolds: I love that you're running the courses that you're running because it seems to me that we've got too many organizations that have just sort of rolled a tool out and gone, "Kind of knock yourselves out, figure out how to use it." And so what I see is that organizations need to be having these discussions at every level. We need to be having them in the boardroom. We need to be having them with the senior team.

I think the most important people are actually the tier threes, and the reason why I think the tier threes are the most important is that they're the culture carriers in any organization. They're the people who actually manage the teams that manage the teams. And so, if you think about it, the CE generally engages with the board and their team. The level twos engage with another small group of people. But the size of a level three, even in smaller organizations, they're the ones that have the engagement with the most staff.

So, to me, while we've got to educate the board and the senior team, this day-to-day practitioner work, I'd start with the tier threes because they're the culture carriers in the organization. Does that make sense?

Ross Dawson: Yeah. So you have deep, extensive experience in foresight, and I think one of the good things about foresight is that we don't have answers, and hopefully people get it that we can't necessarily know what's going to happen, and it is about being able to improve our acuity of sensing and responding. So, maybe not necessarily tying this to AI, but how do we enhance that capability for foresight? Is it in running scenario programs? Is it being able to bring broader thinking? What are ways in which we can use technology or tools to enhance the foresight capability of leaders and organizations?

Melissa Clark-Reynolds: Yeah, there's a lot in there. So the first bit is there is this real kind of rubbing up edge of forecasting and foresight, and I think that's quite interesting. I came out of a more quant background, and I spent, as I said, like 35 years of my career being in tech, and a lot of that was in predictive analytics and in the math side, and so I was really interested in forecasting. I wrote actuarial software, and I worked with insurance schemes, so really interested in forecasting, which often you could do a 50-year forecast, but a lot of it was really more about what might happen in the next two or three years.

A piece of my practice still does that, so I've been working with logistics companies on supply chain disruptions, and so we saw that through COVID and through the disruption in the Red Sea at the moment, but also with climate change and all sorts of stuff. So there is this edge of forecasting. I'm going to put that to one side.

The next piece of foresight for me is really about how do we engender deep curiosity in the organization about how the world might turn out. I studied strategy with everybody—MIT, Harvard, Stanford, Cambridge, all over, right? And when I come down to it, what I see is that many organizations pick a strategy; it's their preferred strategy, and it's going to create the world they want. And they think they can somehow set out on a journey towards their ideal future.

What we discovered is that that kind of worked post-industrial revolution and pre-digital, and for only a very small number of companies does that kind of strategy work today. It seems to work quite well in FMCG companies, and in companies that have high volume, low margin, where their world doesn't really change much. It's pretty much the same year on year—meat companies, dairy companies, maybe even banking. At some point, that stops working for them, but it's worked for a long time.

So other than them, the rest of us have to have much more agile approaches to strategy. We have to be able to zig and zag as the world around us changes. And so, to me, as a foresight practitioner, what I'm interested in is how do we prepare for shocks, so that when those shocks come—which they will—we are agile enough to be able to deal with them. That might have been COVID, might have been the closing of borders, it might be supply chain disruptions, it might be the lack of staff, it might be a cyber attack. But I think now if boards and senior teams aren't prepared for shocks, they're not paying attention.

I love signals analysis, which is looking for weird things that might turn out to be the future. One of my favorite things is to run scenarios with organizations, and I have a workshop where I teach how to do scenarios using a couple of different methodologies. I think that the organizations who run scenarios and then design their strategies for however the world might turn out are much more resilient organizations than the ones who just go, "We're taking that hill, advance." Does that make sense?

Ross Dawson: Yeah, bringing multiple perspectives to bear and understanding that we cannot predict—we can explore uncertainty as well as the trends that are playing out.

Melissa Clark-Reynolds: Not only can we not predict, maybe our ability to make the future we want—like all those '90s posters—turned out not to be true. We have less agency in some areas than we might think, which leads me to the use of the AI tools.

I've pretty much tested every foresight futures AI tool I can find, and there are things that I love about them and things that I don't. One of the things I don't love is that we're still at a point with AI where it's kind of fancy autocomplete. My worry is that what I keep getting out of those AI tools I see is a replication of other people's thinking. It's not the deep creativity that we can do in a scenarios workshop.

I ran one recently with 90 people from an organization. We had multiple scenarios running, and then we took the wisdom of those people and put them together and created four futures for how the company may need to operate. Those people had very different points of view. We had a janitor, we had a family owner—the third generation family member who was there—and kind of everyone in between, and so the perspectives in the room were so diverse that we ended up with something really rich and useful.

My worry with a bunch of the AI tools is that they go hunting for other people's answers to the question, and then they give you back the average of other people's answers to that same question. Coming back to your judgment piece, I think that the human creativity bit, the human imagination bit, the skill of humans to pattern-make in novel ways—AI's not there

Ross Dawson: yet. Yeah, absolutely. It's one of those complements, though it is also a way in which you can—I mean, AI is very good at scenarios in the sense of being able to imagine future worlds, how you got there, day in the life, all of that stuff. It's a wonderful tool, for example, if you build scenarios, then to say, "Okay, well now create images of what it would look like, videos of what it would look like, stories of what would happen for particular people in those things," and it's brilliant at that.

Melissa Clark-Reynolds: It is brilliant at that, and I use Firefly—that's my favorite tool at the moment for the creation of those images—and I find that I can tweak them, I can get some really good results, and they make the scenarios that we created richer and more relatable for people.

I also find that the AI is fantastic for listening to workshops and listening to what people are saying, and then being able to give me summaries that, as a facilitator, or if there's someone there taking notes, we still filter through our bias, or we don't hear everybody in the group. So I think there's also a lot of threat and wealth in those insights that can be captured by AI that a human would just not have the time to do, or even the mental capacity to do.

So I'm a big fan of using those, but I'm not a fan of just sort of outsourcing foresight to the AI, which is what I am seeing quite a few pitches for.

Ross Dawson: Yes, it's the old classic thing: if you give people scenarios as opposed to taking them through the process, it's just like, "Oh, here's a few things," as opposed to—and even more so if it's AI generated—in which case, just, "Oh, look at it and say that's interesting, that's plausible." Doesn't really change anything.

Melissa Clark-Reynolds: Yeah, and I'm with you because it's not the output—again, it's not the answer that matters. It's the thinking and the conversations and the exploration and the discovery that matters.

Ross Dawson: So to round out, what would be your advice to leaders in this world on how to amplify their capabilities for leadership?

Melissa Clark-Reynolds: I think the first one is that we are dealing with complex systems, and my favored approach for dealing with complexity is to find a team. So one is: don't be on your own. If you're on your own in your workplace, can you find a team outside your workplace that is interested in exploring and pushing your capability? Might be a cohort that you're being coached by, it might be a masterclass that you join or something. But can you find community? And if you can't find it in the organization, find it outside. But don't do it alone. And doing it with the AI is still kind of doing it alone because you're in an echo chamber. So find some others.

The other one is I would really think about this piece about judgment and capability amplifier. How can I learn to use AI to be better at my role rather than to get better at? I think that's quite important. So it may be going on a—yeah.

Ross Dawson: Very important.

Melissa Clark-Reynolds: Very important. But a lot of people aren't doing that. They're just trying to be faster. Like I keep saying, they're doing less in less time, which is not really making the most of the tools. I think go to some courses like yours. Go and get yourself educated about how to use the AI better.

And then at the board level, there's all that education piece. Have really good conversations and drive conversations with the CE and the senior team, and then through those tier threes and beyond, about the principles of the use of AI in the organization—ethics, privacy, what we'll use it for, when we'll disclose how we use it, all of those things—rather than trying to start with the guardrails. And I am a fan of guardrails, let me be clear.

And then the last one I think that is super important is that we evaluate ourselves, that we find ways of assessing whether we have become faster, better, making better decisions using our judgment. Have we improved company performance?

Actually, I have one more, which is that the tech requires new operating and business models, and the conversations at the board and senior team level need to move from it being a technology conversation to being an operating and business model conversation, and that's where it hits up against strategy as well. And again, I'm seeing them kind of go, "Here's the car keys, learn to drive," rather than, "What difference will transport or the ability to transport our product mean for our organization?" And we need to be having those conversations in the senior rooms. What does our organization look like if it's really AI enabled? What does the business model mean?

And you see this threat to all of the consulting industries—law, engineering, accounting—because people expect something different from an AI-enabled accountant at a cheaper price. So the business model has to change, and how we're thinking about business model change.

Ross Dawson: Yeah, absolutely. Absolutely agree. It changes the nature of business, and that's happening very fast now. So where can people go to find out more about your work?

Melissa Clark-Reynolds: So I have a website, melissaclarkreynolds.com, and you can contact me there. And the other thing that I've just recently started doing, and I'd love some feedback on if anyone's interested, is I've started a Substack called The Boardroom Futurist, and I'm doing a mix of things at the moment. I'm finding signals from the future that I'm really interested in, and thinking about how they're playing out, and getting some data around them, like job loss.

But I'm also reading a bunch of books at the moment and reviewing them for a boardroom audience. So today I woke up and I've just finished Future Shock, which was 56 years old, and so I was reflecting quite a bit on your question about can we predict the future? Because I was kind of surprised at how good he was on a bunch of stuff—quite impressive.

So yeah, I do a mix of book reviews that I think should be on a boardroom bookshelf, and signals that I'm spotting. So I'd love some feedback if anyone wants to join that conversation. The Boardroom Futurist on Substack.

Ross Dawson: Fantastic! It's been a real pleasure. Thank you, Melissa.

Melissa Clark-Reynolds: You too. Have a great day.