How I Tested That

David J Bland (0:1.144)

Welcome to the podcast, Chris.

Chris Hood (0:3.033)

Hey, thanks.

David J Bland (0:4.910)

I'm so glad you were able to join us. You have such a varied and interesting background, moving through movies to video games to digital media and AI. Maybe you can just give a little bit of context for our listeners. So a little bit about yourself, and then we can jump into maybe how you tested a few things.

Chris Hood (0:25.935)

Yeah, it always seems very maybe disjointed, but I've always said that I think there is a nice cohesiveness to my career. I started off working at a movie theater, fell in love with film, grew up watching television shows like the A-Team and all of that kind of fun stuff. Media was always really just something that I was gravitated towards.

storytelling, the escapism, and then of course, video games. I started playing video games probably around 1982, not to date myself, but to give you an idea. And same thing, it was an escape, it was the opportunity to just go somewhere else. And I stuck with that. I stayed in media and entertainment. I did movies, television, music, and really as I started to, you

get into my professional career, I started to realize a lot of this was really all the same. It was all about telling stories in different forms of media and in different contexts. In the 90s, I started building websites. And then I realized that this was just another form of storytelling. We get to current kind of what we know today as business and the technology that is around it. We first everybody needed a website.

then everybody needed some app. There's an app for that. Now everybody needs AI. But at the end of the day, if you look at business throughout that, when we think about marketing or we think about customer experience or we think about any ways that we want to actually engage with a consumer, there's some level of storytelling that is involved. So I've been able to take all of the things that I've learned throughout my entire career from television, video games, movies, music and business.

and technology and really just morph it all together into some sort of strategic initiative where I go and I can help organizations better understand how to tell good solid stories with that human experience that we all crave into their everyday corporate operations, marketing, whatever their customer experience is. And so yeah, it might seem like there's a lot of different things that I'm doing.

Chris Hood (2:52.207)

but I've managed to actually keep them all pretty aligned.

David J Bland (2:57.985)

I love how the storytelling is the thread here of the storytelling from movies to video games, especially being an avid gamer myself, probably started gaming about the same time that you did. Started on Atari 2600 and then went to my Nintendo 8-bit and kind of just never stopped from there, know, just always eventually got a new system. And I agree with you that the storytelling in games really pulled me in. And the music, I was surprised that now

you know, kind of being middle of my life and looking back and I'll listen to a track, you know, from early 80s. And it kind of just like brings me right back. And so these idea of stories, you know, what's maybe an example of, you know, how you've used something that maybe you learned in movies that you implemented in gaming or gaming and implemented it in strategy, you know, like what's where was that aha moment for you where you started maybe connecting the dots a bit.

Chris Hood (3:56.139)

I don't think there was really an aha moment other than, as I mentioned, I built my very first website in 1997. And at the time, there wasn't many websites out there. I built a blog before people even knew what a blog was. And what I learned though, as I was building these websites was that really people were asking like, what is your website about?

Why should I come to your website? What are you blogging about? And now those things seem second nature today, right? But those are still valid questions. And my answers were simply, well, I'm building a community. I'm building someplace where people can come and read content. What's the content about? And as you start going through that, you start to realize, well, really this is nothing different than a script.

We are scripting out the experience. You go to a movie, you sit down, act one, act two, act three. And if you don't like act two, you change it. If don't like act three, you change it, you rewrite. And websites were really no different. When we reached around the 2000s, I was having people come to me and say, I need to build a website, but I don't know what to put on the website. And I asked the simple question, well,

What are you trying to tell them? What is your story? And I think it was there where you began, where I began to see kind of this trend of websites were really nothing more than another form of entertainment. If you really think about this, I'm sure there's a lot of people who will say, well, no, it's our revenue generator, right? We go and it's shopping, it's where we make money.

Bottom line is you're not making money if you're not convincing them in some sort of experience, which is all storytelling. And so when I started to connect those dots, I really just looked at websites as just another medium of storytelling, no different than movies or television. It was just on a different screen.

David J Bland (6:18.477)

That makes a lot of sense. And we frame this often in my work as like, what's your value proposition and how would you communicate that value proposition? And I think so often, especially the folks that really have a deep domain expertise, they want to talk about all the features and all the products and all the platforms and all the AI. But in the end, it's about that story. It's about, how's this going to help me achieve a thing or how's this going to help me solve a job or

address a pain I have or we use jobs, pains and gains a lot in my work. So we talk about what's a gain that I'm going to get out of this, you know, and I do think we underestimate storytelling and that's how we learn. And I was thinking when you were talking about, how, how would you test a story? Right. So whether that be movies or in video games or in website design, or even now, maybe we get to later in AI, how do you help people work through

know, testing it versus I'm going to craft the perfect messaging and the perfect story and then unleash it on the world and on my customers. How do you help people kind of work through that process?

Chris Hood (7:28.613)

Well, let's start with the premise that you started with yourself, which was I have all of these product features. I have all of these ideas. I want to get all of it out on my website, which no one's going to read, right? Let's just be honest. You could have the most incredibly detailed product requirements and definitions. And honestly, no one cares, especially in today's market where people

have a tension deficit, we want quick bites, everything has to be readily understandable. So then let's think about this from a reference of movies. Most people learn about movies through either a trailer, right, or somebody tells them about the movie. And when you're talking with your friends, you're not going into the entire details of the movie, you're pretty much summarizing it and giving them what

we would call a synopsis. And so I've often told people, can you tell me what you do in 20 words? That's all you got. If you've ever looked at the synopsis of movies and well, a long time ago, the synopsis in movies or like a TV guide or like 20 words, right? You had to get exactly and you're not giving away any of the plot lines. You're just kind of overviewing it and.

Even in video games right now what's happening is, well, this is sort of like Pac-Man meets Space Invaders. And then everybody's like, oh, okay, I got it, right? So, who are you? What are you? Can you do it in 20 words? And then the challenge in the iteration on this is, okay, great, now can you do it in 10 words? And we'll use another TV reference. If you've ever watched the show, name that tune.

I can tell my product in three words. Great. You're probably not getting a lot of substance, but what that is doing is it's narrowing in what your story is and it's helping you define what your product is. So as you are pitching this, whether that's online or whether that is to a friend or to a networking event, whatever it is, if you're even pitching and you're looking for funding as an entrepreneur, can you get this down into three?

Chris Hood (9:53.445)

five words and make me understand exactly what the value proposition is and what that story is. And if you can do that, again, just like a synopsis of a movie, just like saying Pac-Man meets Space Invaders, then that's where you wanna be. That's your website, that's your product pitch, that's your presentations. If you've got 20 slides and 20 pages of nothing but words,

Like the whole script, if you've got the whole script, no one cares. No one's going to read it.

David J Bland (10:30.419)

It resonates with me. I was working with a toothpaste company and we were designing a landing page for a new product and I was trying to keep it short and I kept wanting to add this and this and this and this and I was trying to give feedback of our conversion rate is going to be pretty minimal here because this is a lot of overwhelming information for something that's completely new. And sure enough, launched it. Conversion rate was like 1 % or less and we came back and simplified it. How much do you think

And then we use this term voice of the customer or the words you use. We were talking before we jumped on our podcast today about who do you, who are you testing with? And I like that testing with and on. like that framing as well. So when you're testing with people, how important is trying to find the right words for the story? And do you find the very, depending on who your, who your target is.

Chris Hood (11:29.381)

Yeah, absolutely. think.

Chris Hood (11:34.125)

One of the phrases that everybody I'm sure has heard at some point in time, the customer is always right. Now, I'm sure there's a lot of people out there that disagree with it. I'm sure there are some people who do agree with it, whatever the case might be. I think, and I put this in one of my first books is that there's some nuances to that. Broadly, the customer is always right.

because you have to have a customer in order for you to have a business without the customer, you don't have a business. But I think what we're actually talking about is in percentages, right? If I pull a hundred of my customers and they all say yes to something, odds are we should be doing that. If only one person says yes and 99 people say no, then odds are we probably shouldn't do that. And what that percentage is,

varies based on what you're working on. But I think the same applies for any type of marketing. It's really interesting how many companies do not actually engage their customers to develop marketing campaigns. They spend a lot of time on doing it with product and, can we test this product? Do you like this product? Yes, I like this product. What do you like about that product? Do you like the shape of that product? Do you like the color of that product?

Do you like what that product does for you? Like all of these are typical questions, but no one is sitting there and saying, well, we've got this marketing campaign. Do you like the colors? Does the story resonate with you? Does it make sense? And I think if you went through that process and you broadly asked the customers, then theoretically a percentage of them are going to provide you with enough information to determine if you're going down the right path. And over the course of the last couple of years,

there have been plenty of stories, I don't have to go into any of them, but there have been plenty of stories of marketing campaigns that have launched that have alienated their customer base. And at no point in time did you actually vet that marketing campaign, vet the story, validate if it's going to be successful with your core customers, because those customers, broadly speaking, are always going to be right.

Chris Hood (13:53.495)

And yes, you're going to have some differentiation. Some different demographics are gonna probably think about it differently, but this is why you poll them. This is why you talk to them. This is why we have focus groups. This is why we do surveys. We bring that information together. We look at the data and then that data tells us, this story resonate? Will this campaign work for us? And if all the answers are all yes, then you go do it.

David J Bland (14:21.719)

So why do you think it is, like what's the underlying reason behind marketing departments not testing their campaigns? Like what have you heard? And you don't have to name any company specifically, but I'm sure you've heard some trends over time on similar responses.

Chris Hood (14:37.591)

it's usually personal agendas or egos. Like this is pretty basic. Sometimes you'll have a rogue marketing campaign that goes out there. I want to do this, I think it's great, I'm not gonna tell anybody, you get no approvals, you go off and you do it and uh-oh, oops, comes out. But I do think a lot of times,

you will have, marketing is a creative business mostly. And so I think what happens is you'll have egos like, I've got a great idea. If we put an ape on top of the back of a T-Rex and have them driving a motorcycle, this will work. I'm confident. And somebody says, it's not gonna work. Well, I outrank you, we're doing it. I mean, there's a lot of politics involved in there, but mostly,

It's a creative type of structure and creativity is often ego-driven. And so that's why, you know, that's more likely than not why it happens.

David J Bland (15:47.694)

I've seen similar things where the absence of evidence, its opinions going around in circles in that conversation and whoever ranks the highest usually wins. It's better for worse.

Chris Hood (15:59.545)

Yeah, and the opinions though, Like those opinions come from, I'll use myself. I've been in marketing for 35 years. I do have a knack of being able to look at something and say, that will work or that's not going to work, right? And the problem is, is that when you get into that kind of space where you're like, look, I've been doing this so long, I know it's gonna work or I know it's not going to work.

then that's where the problems usually come from, as opposed to saying, despite the fact that I've been doing this for 35 years, I'm still going to look at the data and I'm still going to ask you, how do you think our customers are gonna feel about this? And if you can't provide me with that data back, then I'm gonna say, no, we're not doing it, right? Or I'm gonna go off and I'm gonna do my own research, figure it out and then come back and say, no, we're not going to do that. So it's the data in addition to the experience.

But when you're void of that data, then most definitely it's all ego.

David J Bland (17:4.829)

Yeah, that makes a lot of sense too. I want to dig into a little bit about video games and you just because I'm very curious about this and it's something near and to my heart. So you get into gaming and maybe share a little bit about what came about as far as creating your own studio and how you've iterated that over the years. I just love to learn more about that from you.

Chris Hood (17:33.637)

Yeah, so I've now started three game studios. The first one failed miserably. It is what it is. The second one became hugely successful and is actually still in operations. And then the third one is brand new and we're still trying to figure it out. And it's a good example of iteration for sure at the third studio.

Gaming itself is definitely an iterative experience, not just starting companies, right? Like I'm gonna do everything I didn't do the last time and try to replicate the things that worked, or I didn't like that. Like I think we all have kind of a checklist in our head that says that worked, that didn't work, I liked that, I didn't like that, okay, I'm gonna do it this way this time. But video games broadly is definitely a very iterative experience. I think video games probably,

almost more than any other medium I've seen is let's actually try to build something. Let's test it. Let's test the market. Let's get people playing it. Let's see how they like it. Let's make adjustments. Let's continue to evolve that game. This is also why games typically take multiple years to produce. Unlike movies, which is really interesting because every single movie goes

through something we would call a screening, where they would have either a public screening, you may have seen this, it used to be that if you went to a movie theater, you would have people out there with like yellow sheets of paper saying, hey, would you like to come and see a movie for free? And here's the movie and sign up. Now it's all online, but it's still the same principle. You could go see an advanced screening of a movie, they get audience opinions, and then they take those audience opinions, and you know what happens? They don't do anything with them. Nothing at all.

They're just trying to gauge how successful the movie's going to be. I think there's only been one time, and in my career, I have been to hundreds of movie screenings. I've even watched movies in multiple stages before it goes live, and then I go to the theater to watch the production release to see if they've changed anything. And only one time, I think, ever was something changed.

Chris Hood (20:2.387)

So they just don't care. They've already built their movie. They already know this is really all just market research to try to determine how well the film is going to do and what markets they might want to release it in. And maybe they pull back some budget and maybe they don't market it as hard. Maybe they don't send it to certain demographics because they don't think it's gonna do well. And that's all that is. You think they're going to make changes. No, it does not work that way.

versus again, yeah, video games, it's all about that. I'll use one more television since we're talking about it. I mean, the great thing about television is it's take one, oh, I messed up, there's a blooper, great, take two, right? And you can kind of watch that iteration in real time of people making those changes. But yeah, video games, I think, has always been very interesting in terms of learning what the player likes.

making adjustments to that game and doing that continuously throughout the process until the game gets released. And even when the game gets released, there's usually very demanding communities who want different types of fixes or changes. And oftentimes those are delivered.

David J Bland (21:19.989)

Yeah, was thinking about playing a lot of World of Warcraft again recently and hardcore. I got my priest to 60 somehow, which I'm not sure how I did that without him dying. But I was playing Seasons of Discovery before that and I was like, this is kind of a test, right? They're doing a lot of different game mechanics in the game. You can only get to 20.

Chris Hood (21:24.602)

Yeah.

David J Bland (21:41.312)

And then they learned from the community and changed some things. I eventually, I think they went too far away and I didn't like the changes and I stopped playing. But they were pretty active about, this is how the community is responding to this change. And they kind of owned up to the fact that we're trying out a lot of things and we want your feedback on them. I also think we see that with a lot of the...

you know, early access releases of games where they even charge for them, which I think was almost unheard of in prior decades, right? We would say, well, we're going to do a test, but it's free. And now you have companies, you know, over the last five to 10 years charging for early access. And in a way, in my opinion, they're probably building up their coffers a little bit to help further the game development even as well. So it's really interesting.

I'm just really fascinated by video gaming and design in general. And so you're on your third studio. So what is it you're trying to test out? Like what are some big assumptions you have on your third studio here with the advancement of AI and everything now, and it feels like tech is just changing so quickly. What are some big assumptions you're trying to work through with your third iteration of this?

Chris Hood (22:54.567)

Yeah, you know, again, I think gaming is interesting because gaming just boils down to one question. Is the game fun? And if you can make a fun game, people will play it. You know, there's a reason why, you know, maybe there's a reason why I was gonna say Monopoly has been so successful for all these years is because generally people like it as a game, whether it's fun or not, I guess that's opinion. But overall,

You make a fun game and it works. And there is really no other metric that you have to be worried about. I think right now, a lot of times there are, unfortunately in the gaming industry, there are a lot of business folks, venture capitalist, big business, Microsoft, Sony,

and they are looking at profitability more than anything else. They're running it now like traditional businesses as opposed to entertainment. And so the technology is getting involved with the process, but same thing, whether there are certain things that people wanna build games for or whether there's a certain demographic that you wanna build games with them in mind, whether it's a specific type of technology.

At the end of the day, none of it matters because if the game is not fun, no one's gonna play it. And I think the hardest thing, I've talked about this in the industry before, which is I'll often ask somebody like, why are you building a game? What is the purpose of this? Like, why do you got a studio? know, anything along those lines. And they said, well, I love games. I'm really passionate about this. This is my passion project.

Well, video games are the wrong place to have a passion project because at the end of day, your passion is probably your passion. It's not everybody else's passion. And there's so many indie studios in the such that are out there trying to make their game and maybe their game will meet a certain audience, but it still goes back to, it fun? If your passion project is legitimately fun and you've got people playing it and they like it,

Chris Hood (25:20.751)

then great, but in most cases, those types of passions are not everybody else's and that's usually the problem. So yeah, you've really got to invest your time and energy and understanding into is what we are building fun and how can we make it more fun? And that's it.

David J Bland (25:43.726)

Yeah, I a lot of business consulting. I've noticed what you what you mentioned there, which the thread of OK, now you get the game and now there's loot boxes and now they're custom skins and now there's a battle pass and now there's all this other stuff. And it's like, does it ever end? You know, they just force it on you sometimes. And it's like, OK, this is fun. But there's all this other noise here that's detracting from it, which is the business, the viability of the game. So it feels as if.

In your third iteration of the studio, you still have this core assumption of is what we're building fun. So what are some ways you try to test for fun? Because I can imagine it's very subjective.

Chris Hood (26:25.217)

It is, but again, I think you have to just go back to put somebody in front of your game, have them play it and see how they respond. Like, do they wanna keep playing it? If they do, then there's a level of addiction, which, you know, a lot of games are addictive. You know, do they smile when they play the game? You know, are they enjoying it? Are they saying, oh, wow, that was cool, right? Like those little moments, if...

if you're checking those types of boxes off, then usually you know you're on the right track. You know, how do you get there? You know, there's a lot of scientific research and study in terms of, you know, what's the science of fun? And you can look at that and analyze that and put those things into play. But I do think you also have to be a little goofy, you know, in the process, right? Like I was kind of joking earlier, like what if we put Godzilla on the back of, you know, King Kong and had them riding a motorcycle?

Like, could we do that? Yes, we could do that. It would be like, right now I'm working on a character for one of our games and it's literally like a rhino that is a fan of Elvis and plays a ukulele while surfing. It sounds ridiculous, but you've got to kind of create these really...

colorful characters and if you have fun, like that's I think also one of the big keys is if I can have fun creating the game, then I hope that people will pick up on that and enjoy it just as much.

David J Bland (28:4.653)

Yeah, I that. It reminds me a little bit of why I enjoy drinking coffee in a cafe. So I should open a cafe. it's like, these are very different experiences. It's like, I enjoy playing games. So maybe I should go to studio. It's like, maybe. But also, it's very different when you're behind the scenes. You mentioned a lot about AI. it's kind of, we were talking about this also earlier. Where do you think AI is going in media? People are leveraging it to test things now. I've seen that in my work and my consulting.

What's your opinion on AI and where it fits into testing right now?

Chris Hood (28:40.618)

There are, well, let's start with this. AI and testing is not new. AI has been around and has been a part of testing for maybe 30 years. So if you have done any type of coding, test validation, even consumer validation, you can put your application into a system, it will run through that system, it will take...

all of your use cases, it will put those into a formulaic, you know, testing processes. You can have it pretend like it's 10,000 people attacking it. Like all of that stuff has been driven by AI for a long time. So that's not new. I think what is new and what is interesting is how people are validating ideas like through chat GPT.

And so if we kind of break this down and we say, okay, log into ChatGPT, provide me five business ideas that I can do for under a thousand dollars and kickstart in the next 30 days, right? Oh, great, here's some ideas. And I look through those and I pick like, oh, number three looks good. Let's focus on number three. Build me a business case for number three. And then it builds out the business case. And all of this is great. Like coming up with content and coming up with information is fine. You should know that none of it is anything original.

Chat GPT gets all of its idea from past ideas. So all it's doing is generating other forms of the same ideas. So it's not like you're get anything original. But now let's say we've got all of this and we start to try to get into it a little bit more to validate if this is a good idea. I see a lot of people right now using Chat GPT to validate their ideas. I was using Chat GPT,

a of weeks ago and I was working on some ideas for like a new book and I was coming up with some scripts and some storylines and just trying to brainstorm and ChatGPT responded to me and said on one of these synopsis said this is the best idea I've ever heard.

Chris Hood (30:53.883)

I had to sit there and think about this for a second and I was like, well, first off, you're not hearing anything and you're robot and I know this, like I know enough to say like, I know that's not true. As I actually started to do a little bit more research into this, I discovered that there is actually what we would call a bias for you. And it's an attempt to better understand and provide you with relevant information.

and personalize that experience and that data, it is actually being biased towards what you want to hear. And so if you were attempting to validate an idea and you're using ChatGPT, I would actually argue here are kind of the top three things that you would never want to use for validating your idea. Number one, your best friend. Number two, your mom. And number three, ChatGPT.

Like none of them are actually going to give you the validation. Well, they're going to validate you like, hey, this is the best idea I've ever heard. But that's not actually going to work when you're trying to iterate on a business. So don't use AI for this because that's not working. Still go out and get real people, talk to a human being about your idea, get people to actually play test it and then get data and use the data to validate your ideas.

David J Bland (32:20.941)

Yeah, I was reading a post the other day on LinkedIn where there's like the case against design thinking and somehow they in the comments, they're recommending the book series I'm a part of. And somehow made the logical leap that you should not interview customers, you should just talk to Chad. And I don't think people know under Chad, if you log in, right, you go into personalization, there's actually memory there and it's logging all those interactions. And so I think feeding into what you were saying, it is tailoring its responses.

Maybe go in and clear that memory and then ask or toggle it off. Maybe you'll get different answers. But the way I use it is almost like an extra team member. Help me think through some of the assumptions I have. Help me think through what the business model could be. But I don't have it substitute real people. And I think that's a bit dangerous because you go in there and you keep asking questions. Don't like the answer. Just hit regenerate until you get the answer you want or...

You know, it feels too easy to let all those biases creep in and then convince yourself that what you're working on is going to be successful.

Chris Hood (33:21.659)

Yeah, exactly. And it's not just the memory. The memory is a part of it. There's actually some research. If you find me on LinkedIn, I posted a bunch about this. But when you start a chat GPT session, and let's say you start it with no even memory, it actually provides a 50 % neutrality in terms of its response. But over time, the more you stay within a particular session,

the more that it will actually start to bias towards you over time. So a 10 minute, 20 minute, 40 minute, 50 minute session, as time increases, the more it will actually return what it thinks you want to hear. And usually in a positive tone, it's not going negative. It's not saying the more you talk, the more you suck at this, right? It's the more you talk, the...

It's purposely trying to provide you what it thinks you want to hear. And that bias, as you just said, if it creeps in, now I start thinking that I'm on the same level as Shakespeare because ChatGBT has told me so.

David J Bland (34:31.977)

I'm also feeding it coincidentally book ideas for a new book and it's just... I'm skeptical of how successful it thinks this book is going to be. Because it's like, oh, but you're not like this other famous author because you can position yourself this way and this is why it's going to succeed. I'm like, I really do want to believe that. But at the same time, I know there's so much more testing I have to do. But it's fun to play back and forth with on a new book.

but I'm a little skeptical about how successful it thinks it's going to be.

David J Bland (35:6.315)

So we've covered so many different things today. We've covered your background in movies and production and video games and AI. And I just really want to thank you for hanging out with us and sharing some of your stories. If people have listened to this and they're like, I want to reach out to Chris, I want to ask him about this. What's the best way for them to get in contact with you?

Chris Hood (35:32.771)

easiest way to get in touch with me is on my website. So you can find me at chrishood.com, C-H-R-I-S-H-O-O-D.com. And there you can find my social media profiles, you can find my blogs, you can find my podcast, you can find videos, you can find a way to reach out to me and whatever else, just send me a message. Happy to sit down with you and talk. And that's all right there, chrishood.com.

David J Bland (36:1.719)

Thank you. So we'll include that link on the detail page for all of you. I just want to thank you so much, Chris, for hanging out with us and sharing your background in storytelling and in marketing and in mixed media. It's just been amazing to hear you share some nuggets of wisdom with us. So thank you so much for joining.

Chris Hood (36:17.371)

Yeah, thanks. I appreciate it.