Joseph Jay Williams on adaptive experimentation, positive behavior change, AB prompting, and one minute experiments (HAI Ep62)

Transcript

Ross Dawson: Joseph, it is a delight to have you on the show.

Joseph Jay Williams: Thanks for having me. I’m excited to talk about how we can get AI to make us smarter, not dumber.

Ross: That’s exactly what we’re focusing on, so let’s dig into that. First, much of your work focuses on the idea of adaptive experimentation.

I’d love to get the big picture of what that means. Then we can dig into how it can be applied to get us some wonderful outcomes.

Joseph: Adaptive experimentation is 20 things, not one, but here’s the broadest explanation. Everything you do has alternatives.

Adaptive experimentation is a way of saying, “I’m already doing one thing, option A. How do I test options B or C?”

That allows me to discover better solutions to my problems. I can give examples, but I see this everywhere. I look at how I’m doing something and think, “How could that be better?”

How can an organization communicate better through marketing? How can people talk to AI to get what they want? How can you change the way you exercise so that you’re more motivated?

For any behavior you want to change, there’s an adaptive experiment that helps you try something different and see whether it’s better.

Ross: I think one point, and perhaps something to dig into later, is that human behaviors tend to be very fixed. People fall into patterns.

We are habitual animals, and behavior change requires us to display different behaviors to see what emerges. We can hypothesize what a behavior might be, but humans also need to be adaptable.

We cannot be too stuck in our routines if we want to experiment with ourselves.

Joseph: Yes, I think it sounds like you’re getting it. Given that behaviors can be so fixed and habitual, how do we remain open to changing them? How do we approach that?

Ross: Yes. As I said, that may come a little later in the story, but I think it’s an interesting point.

If we’re looking to adapt human behaviors by experimenting with them, humans must first be open to experimenting with their behavior.

Joseph: Yes. There’s one paper relevant to this about one-minute interventions, or one-minute experiments.

The idea is that we’re probably not open to changing our behavior by default, but we are open to being happier, healthier, and wealthier. In my work, we ask: What is a one-minute experiment you can do to achieve your goals by testing something different?

One example might be a one-minute experiment with AI. Try saying something to AI differently to help you figure out, for example, how to earn more money.

Another example would be a one-minute experiment with exercise, and I have a lot of social media content about this. Exercise for just one minute and afterward praise yourself: “You did a good job.”

That’s an example of taking these adaptive experiments and making them really simple. They don’t require much, but they move you toward your goals. That’s one way we make people more open to changing behavior.

You can Google the paper or find it on my website. It’s called “One-Minute Interventions.”

Ross: Excellent. We’ll have the link in the show notes. As you suggested, there are many ways this can be applied.

One domain is behavior that affects health. That is deeply interesting to many people and extraordinarily relevant to individuals and society at large.

Why don’t we start with your work on helping people improve their health-related behaviors?

Joseph: Okay, I’m trying to compress about 35 papers into a quick, short version, but that’s a fun challenge. Let me give some examples.

We all get stressed, right? Think about the last time you were stressed and how we typically respond.

Option A is something like, “Calm down, it’ll be fine,” or, “We’ve got this.” Honestly, our experiments show those responses aren’t very effective.

One alternative we’ve tested is telling yourself that stress isn’t something you must solve immediately. Being stressed could actually help you perform better.

For example, perhaps you’re giving a presentation at work. Maybe you’re frustrated because you and your colleagues are arguing about what to do and how to use AI.

Perhaps you’re in a bad mood because AI might replace your job, and everyone is stressed because there isn’t enough time. How do we turn that stress from something that hurts us into something that helps us?

We do these short adaptive experiments. You might say to yourself, “How can I see this stress as evidence that I’m doing a good job?”

“I’m stressed because I care about this work, and perhaps that stress has helped me pay more attention and focus my energy.” You might then stop and think carefully about exactly what you want to say to your colleague.

That can help you tackle the problem collaboratively instead of causing conflict. That’s an example of taking a mental well-being behavior that is often personally and professionally helpful and experimenting with your self-talk.

I can point you to our paper and many others. Let me look up the title. It was “Designing Brief Online Interventions for Helping Students Identify Positive Aspects of Stress.”

We did this with students, but you can imagine it applying in a workplace. We sent them messages immediately before an exam and tested dozens of ways of framing them.

The right messages, delivered in just a couple of minutes, boosted their grades by as much as 4%. Let me tell you, even if I spent an hour or two talking to students, I might not boost their test grades by 4%.

I think this kind of intervention is powerful. It’s almost like finding the magic words. Often, what we say to ourselves can land in our brains in exactly the right way to change our behavior.

That change can then cascade into other effects. Explaining how to turn stress from a negative into a positive can help people sit exams, function in the workplace, and navigate their personal lives.

Does that make sense?

Ross: Yes, absolutely. That’s a great example. One point that arises is that what works for one individual may not work for another.

I want to start thinking about this as a human-AI system. Presumably, AI suggests different interventions and gathers data points that feed back into the adaptive experimentation process.

I suppose there are two parts to this. First, how do you optimize for an individual versus identifying the best interventions more generally across groups?

Second, what is the AI system’s role? How does it function within this learning loop?

Joseph: That’s a great question. How do we use AI to personalize what will help someone in a given moment?

I’m thinking of six different papers, but I’ll share one that is concrete and broadly applicable. If you Google this paper, its title explains what I’m currently going through perfectly: “Understanding User Perceptions on LLM-Enhanced Narrative Interventions.”

Aside from that title, here’s an example. Let’s use the stress situation again, although we have others.

I might ask, “Ross, what’s going on?” You might say, “Joseph, I’m exhausted. It’s early morning.”

“I want to help my child get ready for school, but they’re frustrated and complaining about different things.” “I also feel bad because I’m not helping as much at work, and now I’m frustrated with everyone.”

You can enter that into ChatGPT or Claude and see what it says. It might say something okay. We developed a framework that takes what you said and the typical AI responses and adapts them.

We conduct an adaptive experiment by using what you said, the AI response, and principles from psychology. We ask how to make the response punchy and understand what Ross might be experiencing in that moment.

We design different messages and then see which ones resonate with you. That is what AI is already doing, but in my opinion, it isn’t doing it well enough.

You say what’s on your mind, and our framework asks you to share more. What exactly is going on? We combine that information with what the AI would normally say and create something new specifically for you.

It might say, “This sounds like a reasonable problem, Ross. Here’s a story from someone in a similar situation.” Someone might explain, “I’ve been frustrated about work for several reasons.”

“I realized I needed to recognize that there are things I can control and things I can’t.” “Here’s one example of something I tried to control.”

We craft messages that give you a concrete story. People respond, “I can imagine other people going through this.”

Then it might say very concretely, “Here’s what you could do. Here’s another thing you could try instead—the adaptive experiment.” We end with a question such as, “How would you apply this to your life?”

We combine what people say with what AI says and apply principles from psychology and behavioral science. Then we give people a concrete way to feel better and create plans for action.

Does that clarify how we adapt it?

Ross: Yes. You answered that, but I want to dig into the data feedback loop. In this case, you might ask the individual, “Did this help?”

I’m interested in the potential ways we can gather feedback about the impact. The approach will differ depending on whether we’re addressing stress, health, cognitive acuity, or something else.

I’d be interested in exploring the range of ways we can gather data about whether interventions are effective.

Joseph: Yes, that’s a good question. How do companies, individuals, and scientists gather data to determine what is actually effective?

One of the most powerful things you or I can do is simply try different things. Once you have two options, two ideas are 10 times better than one.

Suddenly, you’re thinking, “Why are they different, and how would I respond differently to each one?” One way to get more data is simply to make the options concrete.

Consider marketing, for example. You design a campaign explaining why a gym offers great class options. You might spend hours writing one version.

Instead, spend 10 minutes making two versions, which AI is good at doing. Tell AI, “Act like the best marketer in the world. Create something people would be happy to see and give me three versions.”

That alone gives you data because you can compare them side by side. You can think concretely about how each email or social media advertisement would land with someone.

You already gain data by comparing options. I can give some other examples.

Ross: In that case, you’re presenting the options to an individual, and the individual says, “I like this one.” Other potential approaches could use brain monitoring, pupil dilation, or a variety of other measurements.

To what degree should we use the individual’s conscious and verbal response? How useful is it to explore other kinds of responses, from reactions to text to the physical performance of different behaviors?

Joseph: I tell my PhD students, whenever they describe how they collect data, to ask why eight billion people would care about it. We develop techniques everyone can use.

Everyone can look at examples side by side, but here’s how we can go deeper. I don’t normally just ask what someone likes. Let’s take the marketing email as an example.

If you received these two messages, which one would make you feel glad you received it? Which message would make you curious about the product? Which one would make its relevance to you clearer?

Those questions go beyond liking. They tap more deeply into people’s attitudes, thoughts, and responses. When you see both options, how likely are you to click a link?

How likely are you to forward one to someone else? This approach involves considering all the questions we could ask someone and testing which ones work.

We’ve collected data from, I think, more than 500,000 people now, but that’s because it’s our job. We allow people to participate in our studies if they want. They can apply to take part.

You can also gather data from yourself and others. Another approach, although it isn’t always reliable, is AI simulation.

My whole approach to AI is that I honestly don’t trust anything it says. I treat it as a hypothesis, not the answer. To be honest, that’s how I think about most of life.

Even if you were the most brilliant person in the world, Ross, I’d treat whatever you said as a hypothesis, not an answer. You can ask AI to provide hypotheses.

For example, “If I showed 20 people these messages about why I think this gym is cool, what different things might they say?” AI can begin to provide a kind of simulated data.

The key is to think of good questions about how people respond, what they feel, what they think, and what they would do. Then we can explore whether AI can augment human data with simulations of what people might say.

That isn’t something we do as well, and other companies are working on it.

Ross: Part of your work involves changing not only behaviors but also beliefs. I’d love to hear more about that.

Of course, that is a fraught question. We can adopt more useful beliefs, but the ability to change beliefs can also be weaponized.

Joseph: Let me give you a two-minute excerpt from the talks I give to organizations and my students. Why are incorrect beliefs a target of mine?

Perhaps I think something is a great idea, but it isn’t. Perhaps I think something is a bad idea, but it’s actually good. All I need to do is discover that.

Then I stop doing the thing that isn’t helping me and start doing the thing that is. I focus on belief changes that are in the person’s own interests.

For example, perhaps I don’t want to do yoga. Let’s say I tend to do functional fitness, such as F45, and I don’t think I like yoga.

That belief might be correct, but there may be short experiments you can conduct. You might ask someone, “What are three benefits yoga could offer you?”

“What do you enjoy about exercise? What makes it fun? Is it seeing people or having a class nearby?”

Sometimes, you can simply try it for a couple of minutes and see. These are examples of helping people change beliefs to get more of what they want: to become happier, healthier, and wealthier.

You might decide that you like functional fitness more than yoga but that five or 10 minutes of yoga is relaxing in the evening. You’ve now adjusted your belief.

Yoga is worth doing, and it has made you happier, healthier, and wealthier. You spend less money going to an exercise class and have more money for something else.

Does that explain how I examine someone’s goals and what they want to achieve, almost like a consulting model? Then I ask which beliefs are a gold mine.

If you adjust or tweak those beliefs slightly, you can get more of what you want.

Ross: Yes, the direction is clear. You were describing asking questions as one mechanism for helping people discover how they might move toward a different belief.

What other mechanisms or interventions can you use? Is it a series of questions or the presentation of hypotheses?

What steps can lead people to consider other possibilities and amend their beliefs?

Joseph: As you’re talking, I’m scrolling through the roughly 80 papers on my website and looking for a good example. Here’s one: Asking questions is always powerful, and we have several papers on that.

Part of my PhD examined how asking “why” helps people change their beliefs for the better. Honestly, asking questions is a broad approach. Almost anything can be framed as a question.

You might ask, “What beliefs can I change about interacting with people that would make things more fun?” You can formulate a question for almost everything, so it isn’t a bad framework to use.

For example, asking yourself about the pros and cons of something is an effective way to change beliefs. You might say, “I don’t feel like eating this particular food. I’m not in the mood for cucumber right now.”

But what are the pros and cons of eating cucumber right now? What do you like about it? The pros might be that it’s available, it’s cheap, and it will make you feel better.

Afterward, you can reward yourself with something else. Then you ask yourself about the cons, which you probably already knew.

You might list them quickly: “I don’t like cucumbers, and I don’t feel like being healthy.” You’d be surprised how much thinking through the pros and cons can adjust your behavior.

You ask, “Why would I want to do this, and why wouldn’t I want to do it?” Typically, we first decide whether we want to eat the cucumber and then justify our decision.

Instead, asking why you would or wouldn’t want to do it can change your behavior. Does that make sense?

Ross: Yes, that makes sense and is helpful. Let’s consider a person in their everyday life who is deciding between a chocolate bar and some cucumber.

Going through this process of posing questions is more likely to lead to a healthier, better outcome. But how do we surface these kinds of interventions?

How do people draw on the interventions themselves? Do we place them in an environment where they’re exposed to these questions?

How do we make these useful tools available so that people use them when needed throughout the day?

Joseph: That’s a great question and something I think about almost all the time. Suppose you come to me with a specific situation.

You tell me you’re married, work a certain number of hours, have a particular job, and care about certain things. I ask myself where in your life it would be easy to test a small intervention.

You’re already doing something. For example, Mary works on the 10th floor of a building. Mary walks in, takes action A, presses the elevator button, and goes up.

What is an easy action B that would help her achieve her goals? Perhaps she feels really sleepy after lunch.

I might say, “Mary, let’s conduct a small, one-minute experiment.” “Next time you enter the building and press the button, get off on the eighth floor and walk up two flights of stairs.”

“When you finish and think, ‘Damn, I’m huffing and puffing,’ tell yourself, ‘Mary, that was good.’” “You woke yourself up. Your body is asking, ‘What the heck just happened?’ and giving you energy.”

That’s an example of considering Mary’s goals and then reviewing the hundreds of actions she performs. We look for situations where she can replace an existing action with a different one.

Many good books by scientists and others provide principles rather than recipes, and that’s great. But remember that what works for Mary on Monday might not work for her on Tuesday.

I think of adaptive experiments as a method for testing which recipe will work now. On Monday, taking the stairs from the eighth floor might wake Mary up and make her feel better.

On Tuesday, she might need to get off on the sixth floor or go for a brisk walk. These are all things she can test very simply and observe their impact.

Does that explain how adaptive experimentation helps you think about this?

Ross: Yes, that’s good. As a brief parenthesis before discussing cognition, let’s think about the human-plus-AI system.

The human component is very clear here. There is also an interface, whether it’s a text message, screen, or something else.

Behind that interface are data and AI that try to identify the most positive intervention for that individual. Can you describe the nature of those AI-driven adaptive experimentation systems?

What do they look like, and how are they built, in a nutshell?

Joseph: That’s a good question. Let me think of a good example. What system conducts adaptive experiments to determine what will actually help you?

I’ll give one perhaps controversial example of a system that isn’t very good and one better system we created. Why are TikTok and Instagram so addictive? Because they conduct adaptive experiments.

Their algorithms test hundreds of messages to see what makes you click and what makes you linger. These systems already exist and are often used in ways that hurt people.

In fairness, I know many people who do this work at these companies, and I don’t think they necessarily want to hurt anyone. If companies can make money by helping you, they’ll do it.

If Facebook, Instagram, and TikTok could make money because you shared inspirational stories about helping people or exercising with friends, they would. If they could get people addicted to connecting, having fun, and being healthy, I think they would.

But they need to be shown how. Consider an adaptive experimentation system that simply maximizes dwell time on TikTok for any content.

That tends to lead to things we all recognize as doomscrolling. We’ve built perhaps 10 to 15 systems that take a different approach. I’ll give an example that might resonate.

We took the humble text message. You sign up for our text messaging service, and we send you one message each day.

We generate different versions of messages from people, AI, users, psychologists, and other sources. It might be a fun fact of the day: “Did you know X?” It could resemble a Reddit post.

Another could ask, “Who could you contact today and tell, ‘I was thinking about that fun time we had a while ago’?” You could add, “I hope you’re doing well.”

Another might ask, “When you feel tired today, what could you do to get a little more energy?” There are many such messages, and the adaptive experiment keeps adding more over time.

You are part of the adaptive experiment. You can say, “That wasn’t good. Here’s a better message.”

You can tell us what you might like. My PhD students, others, and I developed algorithms that determine which messages are most appropriate for you at that moment.

A key point is that we don’t only measure engagement, although engagement can be helpful. If people want the content, behavioral design experts provided it, and great educators contributed, then it’s probably good content.

In that case, maximizing engagement with good content can work. But we also examine whether people who receive this content go outside or exercise more.

Do they call friends instead of merely scrolling through Instagram stories? If they are using Instagram Stories, are they engaging meaningfully?

They might say, “Sally, that’s really interesting. I had this thought as well,” or, “Thanks for sharing that. It brightened my day.”

We develop other measures to optimize for the behaviors the person wants to engage in. Does that explain how we build these adaptive experimentation systems?

We take something you’re already doing, generate alternatives, and optimize for what will help you.

Ross: Yes, absolutely. That’s the underlying principle as you’ve described it, and it can be built into many other things.

At the very beginning, you talked about cognitive impact. That’s deeply interesting not only to the human-plus-AI community but, I think, to everybody.

We know that using AI or generative AI in some ways will likely erode our cognitive acuity. Using it in other ways could help us know more, become smarter, and be more effective.

How can we apply adaptive experimentation techniques to improve our cognition through our use of AI?

Joseph: That’s a great question, and this is what I love about adaptive experimentation. People will still be conducting adaptive experiments 200 years from now.

I wish I could apply it in every possible way before I die, but I won’t. Hopefully, other people will as well.

That’s why we have PhD students and why I work with companies and organizations. There are about three ways to answer your question. Could you echo it back?

Ross: Certainly.

Joseph: Let me check whether this is a good interpretation: How do we use adaptive experiments to test when AI is making us smarter or dumber? Is that a good formulation?

Ross: Yes.

Joseph: Everything you do with AI could be an adaptive experiment. For example, I could open ChatGPT and say, “What’s one interesting thing I learned from talking with Ross today?”

That’s option one. Option two could be, “I just had several conversations with Ross. Here’s a copy and paste of the transcript.”

“Can you identify three things I learned that I’ll want to share with people?” That’s an A-versus-B adaptive experiment. You start with your typical approach and then try another.

You can examine everything you’re doing and see how you could do it differently. The first thing you see from an AI influences everything that follows.

You can generate two or more versions and then ask which ones help make you smarter and which could make you dumber. That’s a little blunt, but if I simply ask AI a question and receive an answer, that might work well.

But asking a different question might force me to think more. That’s what I’m always asking myself. You can’t permanently bake into yourself or an AI how to become smarter rather than dumber.

You must keep testing it. I have one concrete example in my AI workflow that I can share at tiny.cc/abprompter.

Ross: When I follow that link, it opens Claude.

Joseph: Yes, and you can add it. I told Claude that whenever I ask a question, it should tell me what it thinks I’m asking.

Then it should provide an alternative version of the question that I perhaps should have asked. As soon as that happens, I have to think, “Which of those is a better formulation of my question?”

Then it gives me two answers. I now have to think about two questions and two answers. That will make me a little smarter because I must consider which option makes sense in my situation.

Which answer is better? What are the pros and cons? It forces me to think more about what’s happening.

I don’t always want to do that. Sometimes it makes sense, and sometimes it doesn’t. That’s where you conduct an adaptive experiment.

Should I do that in situations where it’s important to think about the proper approach? Should I do it when I’m feeling lazy and want to get my brain moving?

These are examples of testing strategies to see which ones help you learn from the process and accomplish more. My goal when interacting with AI is, in a perfect world, to spend less time, produce better output, and learn more.

I’m always trying to achieve all three simultaneously. Asking AI to provide two prompts sometimes costs me more time because I have to read both.

But sometimes it saves time because two ideas are 10 times better than one. By looking at the two options, I can instantly see the right direction.

Otherwise, I might have written something, thought about it, and repeatedly reformulated the question. That would cost more time because I’d still be trying to determine the appropriate approach.

This process also makes me smarter. I must think about the pros and cons, and my final decision is better.

Ross: Do you have a paper on these cognitive interactions?

Joseph: I have the link to that example prompt, and we’re working on the research. Interested people can apply to participate in the study, contribute, or offer ideas.

We call it citizen science. If you like AB Prompter, you can suggest ideas about what we should ask.

You can tell us how you use it and contribute directly to the work. That’s an ongoing project I’m excited about, and organizations can use the approach as well.

I can also describe what we’ve already done that led to it.

Ross: That’s great as a starting point. You gave one structure: Provide another version of the question and give me two answers.

But there are many different structures. The appropriate structure may depend on the user’s context, the type of inquiry, or what they’re trying to get from the system.

It’s a complex domain with many possible types of interaction. You could experiment with those possibilities to determine which produces the best outcome.

Joseph: We’ve discussed AB Prompter, but there’s also a paper called “AB Scribe.” You can Google AB Scribe or go to tiny.cc/abscribe.

In that paper, we let people start writing something, such as an email, a message, or something to help them manage stress. They could select a piece, such as the first sentence, and use AI to create another version.

Here’s a concrete example: “I’m procrastinating right now. What should I write on a piece of paper for the next time I procrastinate?”

The first sentence might be, “When you’re procrastinating, try this.” AB Scribe would let you select “When you’re procrastinating” and ask AI to provide better hooks.

You could ask it to draw on the best procrastination research in the world and Joseph’s work on adaptive experimentation. AI would then provide several versions.

For example, “When you’re procrastinating” might become, “It’s natural to procrastinate,” which gives you a different hook. You could compare those versions side by side and test which one is appropriate.

Then you could use AI to create a third version. AB Scribe addresses situations where there are many things you could say and test.

It gives you a way to use AI to design those alternatives, determine what’s appropriate, and manage the versions. That prevents you from becoming overloaded while helping you decide exactly what you want to say.

This could apply when you’re procrastinating or writing an email to someone else. We compared AB Scribe with the way people ordinarily used ChatGPT.

People found that it helped them organize their thinking about what they wanted to say. They felt they produced more creative, diverse ideas that conveyed their intentions better than their default use of ChatGPT.

Ross: Fantastic. As you mentioned, you expect adaptive experimentation to be applied in interesting ways for hundreds of years.

To round out our conversation, where do you see the most immediate applications? AI capabilities are accelerating, which may allow us to build better adaptive experimentation systems.

At the same time, questions about how we interact with AI are becoming more prominent. What is most important, and where do you see the greatest potential demand for these techniques in the coming years?

Joseph: I have about 10 answers to that question, but let me reduce them to two. Where could we apply adaptive experimentation to have the greatest impact while allowing people to start immediately, personally or professionally, with AI?

The AB Prompter I mentioned isn’t a bad place to start because eight billion people can use that text. Whenever they talk to AI, they can say, “Give me two interpretations of my question and two answers.”

That’s a good starting point because it helps people think about how they interact with AI. It also gets AI to test different ways of interacting with them.

I’m writing a book on one-minute experiments. Many of those experiments involve things you can do in one minute to change your behavior.

They prompt you to ask questions and modify your behavior, either independently or with AI. That’s a great place to start: Ask AI for two interpretations of your question and two answers.

Companies can also implement this now because they’re investing enormous amounts of money without necessarily knowing whether their systems work. I wish Claude and ChatGPT were doing more of this.

There’s significant financial value for companies that address this problem. Every time an AI opens its mouth and talks to you, it could have said something different.

It’s very easy to generate alternatives. What’s shocking is that AI companies aren’t testing those alternatives nearly as often as they could.

I see some of the experiments they conduct, and I applaud them. Some people are doing great work. Other experiments dismay me because I love experimentation, so I’m sad when I see an experiment that isn’t good.

You know the kind: “Do you prefer answer A or answer B?” Those experiments appear in Gemini, Claude, and ChatGPT, but they could be conducted much more effectively.

Existing AI companies and startups that want to outperform them could take a different approach. Whenever someone interacts with AI, it isn’t difficult to generate different versions of what the AI is about to say.

You need clever prompting, but you can generate alternatives and then conduct an experiment. You could test the alternatives with users or use simulated data.

Companies such as Simly AI simulate data that can help determine the best response to show a particular person at that moment. That’s an easy place to conduct experiments.

It’s easy to identify the opportunities, generate options, and test them with users, whether that means 1,000 users or only one or two. You can also use simulations.

A product team could ask AI, “How would 100 people respond to this?” That’s a great direction.

Every time AI opens its mouth, we should conduct an adaptive experiment. We should determine whether what it says is most appropriate for what that person needs to hear to accomplish their work or business goal.

Ross: Obviously, the direction and design intent are critical. Joseph, where can people go to learn more about your work?

Joseph: Definitely my website, although it’s quite academic. If you love reading scientific papers, go to tiny.cc/williamscv. All my papers are there.

Comment on the papers and say what you agree or disagree with. I also post a lot of content online, including on social media, YouTube, and TikTok.

People can also contact me directly. I love receiving three questions someone has about AI, behavior change, or experimentation. Just send them.

I can’t respond to everything, but I try to read them and identify questions I can answer. It’s like an “Ask Me Anything.”

I might answer a question in my book, on LinkedIn, or through another channel. You can find me on LinkedIn or my website, and you can email me.

Send questions about AI and behavior change, or ideas you think work and that people should know about. I’ll try to turn them into online micropodcasts, videos, or something else.

Ross: That’s fantastic. Thank you for your wonderful work, Joseph, and for sharing your time and insights today.

Joseph: Thanks a lot. I study how asking questions helps people learn, so I love being asked questions.

It forces me to rethink concepts in different ways. When I teach my classes, I have students ask me many things.

Before I answer a question, I have them write down their own answers too. I love seeing how they think about it.

Even when they ask a question, they produce knowledge I wouldn’t have thought of myself. I love hearing people’s ideas and their answers to the questions I address.

Thanks for asking great questions. It was a great discussion.