How I Tested That

Welcome to the podcast, Alberto.

Thank you, David.

It's great to be here.

I'm so glad that we can finally sit down and chat together.

I've been a huge fan of yours for so long.

I think the first time I became aware of you was a Google talk where you may or may not
have walked out on stage dressed as the Grim Reaper talking about the right it and all of

that.

And so I've been a fan ever since and I'd love for you to share some of your background as
well to the reader.

there was bit of an infamous stunt that was at the Google testing conference.

Now it's not testing product idea, it was just testing.

software and I went out to give a talk called Test is Dead, which was quite controversial.

you know, the video is still there and you see, you know, half haters, half lovers.

uh But essentially, I hate to say, but I was right.

The QA discipline, the way it was done before, has dramatically changed in the past 10
years.

But I suspect it's going to change again with AI because with AI,

writing code is much easier than testing code and making sure that code does the right
thing.

So I think in just a brief time of 10, 12 years, it's going to go flip back again to
actually human being testing to see what the computer does.

So it's fascinating.

So it's like a Lazarus of testing.

Test is dead, long-lived testing, fascinating.

But that's a different type of testing than the one we're talking.

True, yeah, the first time I think our paths feel like they cross a lot in online world,
although we've never really sat down and chatted before, so I'm excited about this.

But I actually asked you to be a part, a little piece of the book I wrote, Testing
Business Ideas, because of your pretend to own story that resonated so well to me and you

graciously accepted.

And so I feel like ever since then,

I've just been watching what you're writing and what you're speaking about and just just a
huge fan of what you're doing.

Thank you.

And vice versa, my fan of yours.

you know, you who brought us together?

Patrick Copeland, because Patrick was my boss at Google.

Patrick is the one that one day he wrote me in and said, Alberto, I need you to volunteer
for something.

I said, what?

You know, said, you have to be part of this innovation task force.

said, innovation task force?

You know, I'm trying to do other stuff.

Said, well, please do it.

Do it for me.

Cause they asked Patrick and Patrick didn't want to do it.

So he kind of put it in my lap.

So as it happens with most task force, think I was the only one who did any tasks or had
any force.

So, uh, and as part of that innovation, and we started, you know, how, can Google be more
innovative and what are the problems with innovation?

And one of the things that, uh, that came out is that, you know, we, can build stuff.

But so many times we build the wrong thing, right?

So if you go and look at the history of Google, right?

Remember Google Wave was supposed to be, you know, the evolution of Gmail, kind of Google
Glass, although that was actually an experiment.

But there were many Google products.

In fact, if you go online and you look for, you know, Google Graveyard, you find that
there's far more failed products than products that succeeded.

You know, it's like a 10 to 1 ratio.

So I started studying the problem.

And because I was an engineer and a director of engineering myself, realized, you know,
Alberto, I spent 80 % of my life building great products, testing them, making sure that

they work great, that nobody ever used.

So I kind of came up with this slogan one day, make sure you're building the right it
before you build it right.

Because I know we can build stuff.

but most of the time we end up building the wrong stuff as you well know and as you well,
and as you well teach.

So it's funny, I shifted my career.

said, look, I know how to build stuff.

The real challenge is how do I make sure I build the right stuff?

I thought, can I apply the scientific method and you know, my engineering approach to
identifying and figuring out which products are going to work before, be successful before

I go and sell them.

In other words, to use your language and my language.

I focus on testing desirability because that is the missing ingredient, right?

Feasibility, can we build it?

The answer to that is yes, 90 % of the time.

uh Viability, can we make a viable, sustainable, you know, long lasting business out of
it?

That also is, the answer is yes, most of the time, because I have a slogan, if there is a
market, there is a way, right?

maybe not today, maybe not tomorrow, but maybe in five years, it will become efficient.

So that's how I got into it.

And one day I thought, prototyping, I talked to Tim, said, we have to build a prototype.

And then they came back and said, well, yeah, it's gonna take us about six months.

I said, no, no.

Can you do something in like in six weeks, six days?

I said, no, no, no, that's not a prototype.

So I thought we need something before that.

So I called it preto tapping.

So I rushed to write on Google search.

Is anybody else using this work?

Cause it's pretty cool.

So just, just misspelled on some pages.

And so I decided, good, it's mine.

And so I decided, Hey, I'm everybody's allowed to

one neologism in their life and this is going to be mine.

So I came up with pretotyping.

90 % of the people love it, 10 % of the people hate it.

They think it's not needed, but I like it and I'm sticking with it.

Yeah, that resonates with me.

mean, I think that's still true today with feasibility.

You know, I do very few experiments around feasibility.

I still try to walk my clients through feasibility risk, but they already know that world.

uh Most of my focus is on desirability and viability because even if people want it, you
know, you still have to find a way to make money with it eventually.

And that's why I get pulled into business modeling and all of that.

But I would say most of my coaching consulting now is

almost all desirability and viability.

The company's high coach can build anything.

Well, that's right.

You know, I mean, somebody comes to me, Hey, Alberto, I have a time machine.

Can you help me, you know, test the desirability?

They uh said, well, first make sure that it works.

uh No, but see, in a way that's where your work and mine are slightly different, right?

I'm super concentrated.

In fact, exclusively on desirability.

That's the, the, the pretotyping part.

And because most of the time the companies I work with, they're great at executing.

Right.

So I figure out that they can figure, figure it out, but you're right.

A lot of the time, especially if you work with entrepreneurs and not with experienced
companies, they have an idea and then they can't figure out how to make any money with it.

Right.

So you're right.

Desirability without viability is a problem, right?

It's called a success disaster, right?

People want my product and I lose, you know, $10 on every sale.

So yes.

I think our work is very similar.

I remember when I asked for the pretend to own to be included in testing business ideas.

I was thinking, well, I can't, I can't.

So we had a professional illustrator named Owen Pomeri hand draw every experiment.

And I was like, oh, I can't have the, you know, the wooden block with the chopstick as far
as a drawing.

I need to find another drawing.

So this illustrator, I said, hey, can you?

This is what I'm talking about.

And I point him to kind of some of your work too.

And I said, can you just come up with a drawing that represents this?

And he ended up drawing this sort of like fake dog.

It was like a dog.

It was cardboard.

And you put a leash on it and drag it around.

And it actually, I kind of really enjoyed it.

So it made it into the book.

for those of you, I guess for those listeners who don't know what a pretend to own is, can
you maybe just expand upon that a bit and how you use it, you know, with your work?

Sure, so pretend-to-own is a type of pretotypes, right?

So just like if you have in your book, which I think is a beautiful, beautiful book, and
as I said before, with a perfect title that I wish I had thought of that title first, How

to Test Business Ideas.

So there are many techniques, and I call them pretotyping, and the pretend-to-own is one.

And essentially it's to test desirability, but at a very personal level, because many
times I'm sure it has happened to you.

You work with companies that they build products and they say, well, I'm not going to use
it, but for sure somebody's going to use it.

In software engineering, there are many tools that I work and say, well, I would never use
these.

I would just use,

a VI or just like a text edit, but I'm going to build this very sophisticated text editor.

um

The product in question uh is the Pound Pilot.

If you're our age, you probably remember when it came out, was the hottest thing ever.

And Jeff Hawkins, super bright guy.

So his product before the Pound Pilot was called the Grid Pad.

So the Grid Pad was the very first iteration of what today is an iPad.

ah So...

no, genius level, genius level guy.

He spent years developing and building the grid pad.

And as uh Time Magazine said, it was an engineering marvel, but a market disaster.

Okay.

So the grid pad did a lot of things, but it turned out it was too big, know, too
cumbersome, tried to do too much.

So it was essentially, it was a failure.

So here you have a very smart person.

Jeff Hawkins and a very talented uh team investing years to build a product that did
exactly what it was supposed to do and that nobody used or bought.

so when he, you know,

after licking his wounds from that failure, he thought, well, what can I do next?

Why did this product fail?

And he realized, well, maybe it was too big.

instead of making it, you cause it really was, you know, as heavy as a brick and the size
of a, know, of a paper, of a hardback book.

So said, what if I put something I can actually put in my uh pocket?

And uh so he thought, well,

I know I can build it, but I'm not stupid.

I'm not going to repeat the same mistake, right?

Fool me once, shame on me.

Fool me twice.

Fool me once, shame on you.

So he said, I'm going to build.

I'm going to, he went to his garage.

He got a block of wood about the size of a pack of cars, or actually the size of today's
iPhone.

a block of wood, he put some paper sleeves on it and on the paper sleeves he drew a user
interface, right?

That had a square where you had the text.

And then remember he had this of writing called graffiti, right?

Which if you want to write an A you do like a little triangle.

And it was actually very easy to learn.

I can still write in graffiti to this day.

But before he built all the hardware, of that, he just went around with this block of wood
pretending that it

was working to see if we would actually use it.

So would pull it out of his pocket, you know, and I can pretend that now I'm, you know,
I'm Jeff Hawkins and say, Hey, David, why don't we schedule lunch next week?

Does Thursday work for you?

You say, yes.

So he pulls out this block of wood with a paper interface, a toothpick, you know, and it
kind of clicks where you would do to schedule a meeting.

And after doing that for a week or so here, he learned

Right?

The valuable lesson, you you, you, you do pretotypes and experiments to learn.

So what did he learn?

Yes.

If it's this size, I would carry with me all the time to, I would only use it for four
things 90 % of the time, appointment books, to-do lists, you know, a little notepad and a

calendar.

So he just built something very small with those features and it turned out to be a
success and it's the predecessor to our today's smartphone.

Amazing.

Yeah, I love that story.

It's, it almost feels as if people would skip that step and say, well, why do all that
work?

I'll just go and build the whole thing.

But it sounds like he learned quite a bit from just the experience of carrying it around
that he may have missed some of that nuance of the four types of things if he hadn't have

done that.

That's right.

And I have a theory as to why people do not do this more often, right?

Because it's so logical, right?

If it takes you, if you know, by the way, you know, the first thing that I think I write
in my book, chapter one, the law of failure, which states most new products will fail in

the market, which everybody understands.

So that's the first punch.

The second punch is even if competently executed.

So competent execution, I mean, you built it well, it works great.

That doesn't matter if the product, if there is no market for that product.

So people know that, that most products fail in the market and yet they still build the
product and then they try to sell it.

And after a while I thought, Oh, I finally figured out why that is.

It's a fear of rejection, right?

So I kind of compare it to, you know, fear or romantic rejection, right?

You just like you are afraid to go and ask, you know, somebody out on a date or you want
to be perfect.

Say, well, you know, after, you know, after I, you know, buy a new dress and I look good
and, you know, can put some perfume, I can go and do it.

They want to have the perfect product before they go and approach the market.

And that's very hard to do.

because the market may reject you.

So they use that as an excuse to procrastinate and I'm going to build it, I'm going to add
this bells, I'm going to add this whistle.

And then eventually they bring it to the market.

They think it's perfect.

And then it kind of backfires.

That's my theory on the psychology, right?

Cause we're logical.

Human beings are supposed to be logical, but some things are psychological.

So the logic is, of course you test if there is a market.

Because to use your language, you do a risk assessment.

Am I correct?

You look at the risk ranking.

This is the highest risk.

So let's this hypothesis first.

Most people know, will people buy it?

Is it risky hypothesis?

And yet they ignore it.

So they're not being logical, but they're being psychological.

a lot of time, in fact, a lot of my work, I emphasize psychology.

uh over logic.

Human beings are not as logical as we would like them to think.

Yeah, I think that resonates too.

think there are so many ways that plays out in the experimentation process.

Everything from, we're taking a really long time to run this experiment, or we're taking a
really long time to analyze the results into insights to drive to action, or we set the

criteria so low that we just kind of hop over it and check the box and say, yep, ran a
test.

There are all these different ways that plays out, but I think you're right.

It comes back to, we don't want to be wrong.

We don't want to be rejected.

and therefore we'll sort of manufacture these scenarios where we don't really have to face
that.

We'd almost rather face the, we spent millions of dollars and launched it and then it
flopped versus the other rejections along the way.

That's exactly we like to live in Totland, right?

So in Totland, as long as the product exists only in the imagination and the market exists
only in the imagination, you cannot fail.

Right.

So, so this is the same of, the singer songwriter that never performs in public, but,
know, it's taking time to put together this amazing album, know, of 10 songs.

There is no failure in your own mind.

When you bring it in the real world, the chance of failure is very real.

And that's why I think most people live in what I call Totland with their

idea.

Yeah, and you mentioned um the risk ranking and looking at hypotheses.

I want to get your take on hypotheses for this because I sense some confusion out there in
the market.

I also sense confusion in people like Coach.

I think early on in my career, I used assumption and hypothesis sort of interchangeably.

And over time, I've sort of clarified my stance a bit saying, yeah, an assumption is
something that would, you you think it's true, but there's no proof.

Whereas a hypothesis sort of like a working assumption.

where I use the words, and Alex Westerwalder and I do this too, where it's like testable,
precise and discreet.

And then we try to refine something, because sometimes it's like, yeah, well, they buy it,
but how do you test will they buy it?

It's like, who's they and how much and what's it?

And we ask all these questions, but you have a specific format for the way you teach
hypotheses.

And I just wanted to dig into that a little bit.

So like, how do you approach framing a hypothesis?

Yeah, I use, uh it's funny because the words you use are exactly what I use.

here's how the X was the hypothesis.

That's why I it came up.

So at the time I was uh teaching this course on innovation and creativity uh at Stanford
with Tina Selig, you know, great professor of creativity and innovation.

And I had office hours, right?

So in office hours, students came to pitch their ideas.

And I remember the most memorable ideas we even did some video about it was the challenge
was how do you reduce food waste on campus?

And one of the idea was second day sushi.

And the idea is that, know, just as a sushi is about to expire, right?

There is a 24 hour window where it may not taste fresh, but it's not yet going to kill
you.

And so the idea is that, students have a strong stomach.

So, you know, if we take the sushi and, because it's about to expire, instead of throwing
it away, we sell it at half the price.

Some people will buy it.

And when they explained the idea, they use, as you said, who are these people?

are these words?

said, well, lots of people would buy second day sushi if it's cheap enough.

Right.

And, and I said, well, who are these people?

You know, uh what is cheap enough?

So what are all these details?

So two expressions that I learned at Google, data beats opinion and say it with numbers.

Okay.

So, uh, I was doing this office hours and on this board, it was in the engineering
building, you know, some, uh, some kind of math class or engineering classes, they left

some equations, right?

Some differential equations.

So saw look at X, Y, and Z, you know, and I thought, wait, I know how to get to these
people.

So I raised the board and I wrote at least X percent of Y.

will do z, right?

And x, y, and z, remember from your high school algebra, they're unknown variables.

And that's exactly what you're dealing with, right?

So you're dealing with an unknown product, an unknown market, an unknown price that will
actually work.

so that day, I even asked the student, can you please take a photo of me and the
whiteboard?

Because I thought, I suspect this is a very good idea.

And in fact,

Right now when I can teach something, just teach people the X, Y, hypothesis.

ah

And I also teach it to entrepreneurs.

I said, look, you have to do an elevator pitch.

So I taught some entrepreneurship classes and we have these things where students come to
a team has five minutes with you.

I said, when am going to teach you?

I said, just the X was the hypothesis.

So imagine you have an idea and then you have to kind of wave your hands and explain the
idea to some people.

here's my idea.

At least uh

20 % of students ah that want to buy sushi will buy second day sushi if it's half the
price.

ah Right, so it's very clear, you know, 20 % of your target market sushi buying students
and you have a price target, which also kind of gives you an idea of the size of the

opportunity.

you know, how big the business is.

So I started teaching that and it caught on, not just with entrepreneurs, also kind of
CEO.

You know, some people said, damn, you know, I got this Harvard MBA and nobody ever taught
me something this simple.

They just come with this 40 page business plans.

And this one explains it in one second, whatever idea you have, if you need to communicate
to anybody or ABC, if you give them the X was the hypothesis, at least 30 % of people use

taxes will use a Waymo, you know, if it's price, you know, less than 20%.

percent more than the taxi.

And then you know what the size of the taxi market is, you know what the price is.

It's a very clear way of explaining your idea.

Yeah, I like that.

um One of my first big um engagements in San Francisco was a travel company.

And the whole travel company is based on the idea of you'll buy a room if it's discounted
enough.

So we had to have come these hypotheses of, what's that rate and who's the target and uh
what are they going to convert at?

What percentage is all that?

And so that really resonates with me.

So you're kind of using XYZ to...

focus it feels like or help them refine what they're trying to say in a way where they
could go test it.

That's right.

It accomplishes several purposes.

First of all, if you have multiple people in a group, the moment you put numbers to it,
you realize the differences.

So I had another team that said, an idea for portable air quality monitor, right?

So, you know, if you live in a very polluted city like, I don't know, Beijing, you know,
the air quality can get horrific.

And if you have kids, you have to be careful.

So they thought, well, you know, if it's reasonably priced, et cetera, et cetera.

When asked them to write the X-ray hypothesis,

I realized that for one of the team members, reasonably price meant $100 or less.

For another one was...

about $200.

So just the fact of saying it with numbers, you know, they said, what do mean $100 or
less?

We cannot build it for less than $100.

And the other guy said, what do mean $200?

Nobody's going to pay this for $200.

So that's the first thing it accomplishes.

Second, it's a great communicator, right?

If you say 5 % of parents who live in highly polluted cities will buy a personal hair
quality monitor for, you know, let's assume the grid on $150 for $150.

You also have an idea

of the size of the market.

And finally, it comes to what you and I are kind of famous for.

It is designed to be tested, right?

The job of a hypothesis is to be tested, right?

It doesn't exist there in isolation.

If you have the hypothesis, you've got to test it.

So you have all of the parameters.

So if it's true that 20 % of the people who live in Beijing and who have children, who
spend $150 for a portable air quality monitor, how can we test it?

And there what you do?

Do you test with every one in Beijing, which is like 10 million people or more?

No, you kind of you, you, you, zoom in.

I call my next step is called hypothesis zooming.

Right.

So 20 % of people who live in polluted city, said, but where are you now?

Well, I'm in Beijing.

Okay.

So 20 % of the people in Beijing, where in Beijing are you now?

I know this, I mean, this district.

So in this district, is there a place in this district where there are parents that, well,
know, my kids go to a school and there's a, thousand parents and there, so there's about a

thousand kids that, perfect.

So you zoom in from the entire planet of all the polluted city to Beijing, to a particular
school in Beijing.

So run your test there.

Because just like when uh they tested em the fact that gravity could bend light, the
effect of gravitation, that applied to the entire universe.

Yes.

gravity, know, uh bends uh space, but they couldn't test the whole universe.

So what did they do?

They waited, remember, for kind of an eclipse where, you know, the position of a star and
when, you the eclipse was at the right point, they were able, by just looking at one star,

that it deflected by the predicted amount.

And so by doing this very small focus experiment,

the validated hypothesis.

So hypothesis make predictions and if your test validate those predictions, then that's a
very strong signal that ah your hypothesis may be correct.

Of course you don't judge on one test, but...

like how it gives you the ability to run multiple tests, to ultimately prove or disprove
or refute or support or validate or invalidate whatever language you're gonna use with

that hypothesis.

I do like zooming in.

I once worked at a company that was founded by this guy named Majid from MIT.

And he kept talking about how we zoom in, how we use statistics and use all that.

And he said, you know, if you don't believe in this next time you go in to the doctor and
they're testing your blood, have them take all of it just to be sure.

And that really stuck with me all these years.

I was like, yeah, yeah, just take all my blood just to make sure that, you know, we
already have this.

So I'm a huge fan of your book, The Write It, and I know you

You use a lot of this in your own work.

We're very alike that way where we don't just espouse these methods and tell other people
to do them and then don't follow themselves.

We use them in a lot of our own work.

I use a lot of these techniques in creating my book.

I'd love for you to share some of the ways you've used this in your own work with your own
book.

How did you prototype your way through it?

What did you learn?

I would love just any insights you can share on that process.

I love it too because it's very meta, right?

So I wrote a, I pretotyped a book about pretotyping.

Okay.

So.

When I was at Google about 2011, I came up with this word, pretotyping, and I kind of put
some of the concept together.

And I wanted to write a little booklet to use in-house with various pretotyping
techniques.

We cannot mention one, right?

We took the pretend to own, are some kind of the fake door, the facade, the mechanical
turk, there's a whole list of them.

But I thought, I knew, right?

Everybody knows most books fail in the market.

Okay.

And since you wrote the book, you know how much damn work it is.

It's very hard.

It's no fun.

You're there alone.

The feedback takes months before it's published or a year before you get it.

So it's really drudgery.

So I thought, I'm not going to go through the drudgery unless I know that there is a
market for my book.

So I thought, oh, perfect.

I'm going to write a pretotype book about pretotyping, which is very...

Meta.

So I talked to my boss, Patrick Copeland, know, best boss I ever had.

That was one of your previous podcasts.

Yes, on the Amazon dashboard and he was amazing.

Yeah, he's great.

He's a great guy.

A lot of fun.

We've shared the office and we always had a lot of fun.

So said, okay, Alberto, I give you one week.

to write this, right?

Because I don't want it to be a junket, so it disappears for three months on a sabbatical.

So I thought, okay, I one week.

rented, you one of the first Airbnb was an Airstream near California, here near Big Sur.

And I said, okay, no internet, nothing.

I'm gonna give myself one week to write, and however long I write in one week, that's
gonna be the book.

So it turned out to be 72 pages.

I didn't, the editor was our uh tech administrator in the office.

you know, thank you, Diana.

Right.

So I gave it to her.

She was very good and you know, found a lot of little typos.

So I, the 72 page booklet, you know, I put it online, you know, internally at Google.

And then I went and printed some, some hard copies, know, and folded them and stapled them
myself in a little booklet.

And I started giving them away.

And kind of it caught on and people kept asking, asking me for it.

So eventually it escaped the, Google firewall and ended up at Stanford.

And so, uh, people at Stanford started doing, asked me to, you come and give talks about
it?

Uh, to make a long story short, eventually, I don't know how many copies have been printed
or read illegal, cause I didn't protect it or anything, but I know there are a lot of

translations in all sorts of languages.

Uh, some people ask me for permission.

Other people just simply translated.

it.

And so I said, Hey Alberto, you know, your book has been translated, you know, in
Portuguese.

said, well, I didn't know that, but that's great.

Good, good for them.

Right.

I mean, Google paid for it.

So, uh, so, so when that happened and you, boy, people are going to like the book.

They like the way I write it.

So now it's time to write a proper book.

because I validated the desirability.

People love the idea, they share it with other people, they ask me for more examples.

And so at that point, it made it very easy to go to a publisher.

And that's the next step.

Right?

So this desirability, you know, I'm able to get an advance from the publisher that
they're, you know, means that they're interested enough.

So that actually worked out very well.

And, and also gave them the evidence.

I can write, right?

Here's the evidence.

So there's some feasibility.

Hey, Alberto can write and he can, can write in a funny way.

People are reading it because it's all this translation.

say, well, this is better than we have for most books.

And so then I invested the nine months, uh, it took to write the book, you know, and I've
never had to give birth, uh, to

baby but you know giving birth to a book uh it's probably painful in very different ways
but yeah it was really a no it's a it's a lot of work it's a lot of lonely work so but I'm

glad I tested it

I am too.

think I have in 2011, you had a second version of the PDF.

I don't know how I came across it.

It might have been from one of your talks or something you posted, but I have that version
from 2011, which at the time was still very insightful.

mean, you were onto things that, you know, the Lean Started movement was just getting
going.

And there were a lot of these things circling together.

I felt like we had your work, we had work from Eric Ries and Lean Startup, we had work
from my co-author Alex Osterwalder with the Canvas and the Business Model.

All that stuff started like swirling around this problem we had, which was we're launching
stuff that nobody wants.

Like we have to fix this.

And the granddaddy is Steve Blank, right?

Steve Blank also.

But here's the interesting thing, just like with the Newton and Leibniz, right?

We kind of arrived at these things independently.

So I was like six months into pretotyping and that stuff that somebody said, hey, have you
heard about the Lean Startup?

I hadn't heard about it, right?

But I thought...

Great, right?

When two people reach the same conclusion, if we invent calculus at the same time, it
means that it works.

That's a very good sign.

And yeah, so that's kind of the genesis.

But also...

personal life of pretotype.

interesting story.

My daughter, right?

You know, she's been my daughter.

likes to, you know, most of the time kids don't do what their parents say, but you know,
she, believes that pretotyping is right.

So she lived in Berkeley at the time with her boyfriend in a two bedroom apartment and she
wanted to move to San Francisco.

And she thought, but in San Francisco, they could not afford a two bedroom apartment.

So she thought, well, can I

Can we live in a one bedroom apartment?

Right.

You know, how do we organize?

So she, she pre-typed it.

She was in Berkeley still.

They lock one room, right.

Put all the furniture that they couldn't use.

And so they pre-typed living in one bedroom for one month, which is kind of silly when
they explained to their friends.

you know, no problem doing this.

And so, yeah, I tried to pre-type a lot of things because not just with new products, as
you people, a lot of time, you know, people buy, you know, they buy

$300,000 sailboat and then they realize, yeah, I don't like sailing, sailing that much.

Just to have an extreme example.

Yeah, I think it's a good example because I think early on, well, maybe it's still today a
bit, people feel this only applies to B2C software, you know, it doesn't apply anywhere

else.

And I have to say most of my, if I look at my client list right now, almost none of the
companies are software, you know, it's doing this in other industries.

And uh I had the opportunity to...

interview one of the founders of Citizen for my book and they kind of prototype the whole
hotel in a way or pre-dotyped, uh you know, using shipping containers to do the room

design and everything.

And he had moved on to doing another, it's like a long stay business traveler friendly
modular unit.

And he was pre-dotyping it and he was not just testing it with travelers, but he was also
testing it with cleaning people.

Because he realized, well,

if it's really expensive to clean, this is still going to fail because even if people want
to pay to stay here, but they can't clean it, it's going to fail for that reason.

So I thought that was very insightful from him.

But I do think the word needs to get out more often that this isn't just a software thing
where you're A-B testing and that's what experimentation is.

I do think that's the trend right now.

If we say experiment, people think, well, that's an A-B test.

I was like, no, no, there's so many more things to be on A-B test you could do.

Hmm

you don't have to just be limited to software.

mean, whatever you're working on, be creative and find a way to test it to see if there's
any demand whatsoever.

That's right.

Yeah.

So you start with the biggest assumptions and then you move on.

Like one example with Google Glass.

So I was there at the time, right.

When it was launched and people say, you know, that's an example.

Why didn't they preto type it?

said, well, actually Google, because, you know, they have hundreds of billions of dollars
for them.

It was just like, for me investing, you know, $20 to, you know, I don't know too.

to try and to read the book if I like it.

So it was really nothing.

It was never mass produced.

And in fact, what they were actually trying to figure out by, remember that the Glass
Explorer program, right?

So, you so you had to pay $1,000, you had to sign up.

It was never mass produced, unlike the Apple Vision Pro, right?

So that one, spent a lot of time building, you know, it was in the stores, they probably,
you know, you know,

tens of hundreds of thousands of them built.

was never like that with Google Glass.

They learn.

Yeah, there is some desirability people like it, but then the desirability was offset, if
you remember by the fact that people didn't like other people looking at them with Google

glasses because they looked a little creepy.

So as you said, you you learn the second order effects, just like cleaning the containers
that in the end are going to harm the success of the product.

So you go from one assumption to the other.

uh And what I've discovered, David, is that the

Desire if you have a lot of desirability people will put up with a lot of missing feature
of Problems, right?

Remember the first iPhone didn't even have cut and paste, know, it was it had a lot of
issue But it was so desirable that people did not

care.

And if you look at the history of a lot of products, their first version is terrible, but
it's so expensive, know, big bulky like the VCVCRs or the, first video recorded, but

they're so desirable, the people, if they want it enough, they will succeed.

So desirability really is a tailwind for success.

And if it's strong enough, it can overcome a lot of other issues.

If it's not strong enough, right, it's going to kill you.

I agree.

It's really tough, um especially these companies that build something and then it's almost
like they frantically look for a customer to sell it to and you either find one before you

run out of money or you don't.

And it's such a stressful place to be.

mean, you have this runway and you're frantically looking for any signal.

I feel as if we're kind of flipping that on its head a bit and saying, no, let's start
with...

you know, the desirability and then back our way into what we build and what we build may
not have to be perfect.

We can be a little embarrassed by what we build.

So I'm really curious, you you've done so much and this is our first time really sitting
down and chatting together.

What's exciting you right now?

I mean, I know I've seen...

You pulled into different things, AI related.

think I may even seen an AI version of you out there somewhere.

There's all kinds of stuff going on.

What gets you excited about some of the hypotheses you're testing out?

Well, I've always been about testing.

So right now I'm shifting from product market fit to human being kind of philosophy fit.

So my next book is really a departure, right?

It's called The Right You, right?

And it's about my, I like to read a lot.

So over the past 20 years, I've read hundreds of books on philosophy, psychology, and I
thought,

You know, I know how, so here's my, here's the progression.

Building products right.

So that was my focus on testing, know, engineering design, know, design excellence, all
those things.

Then I realized, boy, I can build perfect product, but there is no market for them.

So I shifted my focus on.

building the right product.

And now I see a lot of people that have built, have achieved the success with the market
and all those things, but their lives are still miserable.

So I'm tackling, you know, I'm kind of turning 65, actually just a few days ago.

So it's time for me to get serious and start tackling some of the bigger questions.

And I feel that now I'm, you know, I've lived long enough that I'm qualified to do
something a bit more serious.

So that's what excites me.

now because I think, you know, as you probably know, live in the greatest, in one of the
biggest time in terms of affluence and, you know, all the major problems, you know, kind

of at least for us over here are resolved.

And yet still there was a lot of, you know, unhappiness and, know, and, we and boredom and
problems or people arguing and fighting with each other.

So I think now it's time for, you know, Alberto to work on a slightly bigger problem.

But of course I still get really excited.

when I talk about pretotyping, you can tell, or innovation, especially now with AI, think
there's, as you probably know, with AI, mean, building pretotypes with AI is so easy,

right?

One of the pretotype techniques is the YouTube where you have to kind of get a studio,
pretend that your product exists, know, mock it up.

Now you can just ask AI to do it.

So it's great to be, to pretotype.

Of course, there is a lot of risk.

in that too, but that's so that's also exciting, but I want to focus more on You know kind
of going to the next level, you know and work on the human psychology now that we got the

software and the hardware and those things done What about our wet wear still need some
some testing?

Yeah, we definitely have some bugs that we need to address.

Can you give us any insights into it?

how would you begin to test?

Because you've already written a pretty successful book and you've tested your way through
it.

Like, oh what's a big assumption you want to test with, you know, this new book or some of
the piece of it?

so by the way, I'm not here pitching it because I'm still writing, you know, I about a
third, you know, I get distracted because I'm trying to write on as a book, when you write

the book.

anything will serve as a distraction, right?

Kind of to procrastinate, right?

So I have these clients that come back and say, yeah, Alberto, can you help us test this?

We want to innovate.

And I get all excited and say, yeah, I'll do that.

But yes, I'm following the playbook.

So I have my own very hard metrics of skin in the game, right?

So that's one of the key things.

It's easier for people to open their mouth than to open their wallet.

So I think the evidence that you must ask must come with some skin in the game.

So skin in the game can be money, know, people can pre-order the product,

deposit, give you some information like their email or your phone number or in the case of
the book, of a book, the skin in the game is will people read it, right?

Because I bought many books, right, and read 10-20 pages and then I let it go.

I say well okay, $20 spent on the book are not worth, you know, four hours of my time.

So I'm pre-taught tapping, so I wrote the first part of the book.

So let's say 100

90 pages out of 250 pages or so.

So I put in a PDF format, know, in a Kindle format.

And then I picked 20 people initially, friends, even people I met on LinkedIn.

That's, you know, they seem interested.

Hey, would you like to read it?

And then my metric for them was one, do the read to the end.

And I know because, you know, when I talked to them, said, so what do think that one of
the characters called Marcus?

So what do you think, you know, what happened when Marcus jumped in the pond?

And so I know that if they read to that point said, yeah, I thought that scene was a
little bit extreme or it wasn't believable or say, oh, this was great.

So that's the first sign, you know, if they read through, told them, if you don't feel
like reading, don't, because that data is not useful to me.

And then the next one is how much are they bugging me to keep writing the next, the
following, the following part.

So that's how I'm pretotyping the book and kind of getting my data.

was amazing.

I love that.

I have so many unfinished books.

ah There was one a long time ago I started testing.

It was like a business narrative sort of style.

I found it incredibly challenging to do.

It was almost easier to write testing business ideas than try to do that style of book
because I was trying to test a chapter at a time.

It was really hard to keep going.

It really hard to give feedback, to tie all the characters together.

I mean, because then you're functioning like as a proper novelist, I guess, and not just
tinkering with.

It's so easy for me.

I won't say it's easy to write a book, but when you have experience, it's almost it's
almost like, can I write a book like I'm coaching a team?

And what advice would I give them in this situation?

That comes pretty naturally to me.

But writing in a different style.

It's quite challenging to test your way through that style, in my opinion.

Absolutely, yes.

yeah, but what do we do?

Now, having said we talked about this testing desirability and everything, I think there's
something important that both you and I need to tell our students and our clients.

Sometimes ignore everything we say.

Why?

because I don't want to live in the world where the only products and things that are
built are things that are kind of fully tested and vetted and guaranteed to succeed,

right?

So you are not only allowed to get crazy once in a while.

I want to live in a world where I you know, I could tell to you, you know, David, I'm
going to write a book about squirrel watching and...

I don't care if it sells a copy.

I'm just so passionate about squirrel watching, I'm going to do it.

And I think the world needs people like that.

But if you get money, especially if you get money from the VCs, you have, let alone your
friends and family that want to invest in your business, then you really need to make sure

that you're building the right it.

But if you're not putting your other people's money and lives at risk,

I hope you do something crazy without testing desirability, feasibility or viability.

First, I'm just going to do something crazy.

So I don't want to live in a world where there aren't people that take these crazy
chances, like the Wright brothers.

We need those.

Just be very clear.

I'm doing it because I want to make sure I succeed.

And the other one said, I'm doing just because I want this thing to exist and I don't care
if it works or not, or if anybody buys it.

That's a great point.

I like that.

That's a great place, I think, to wrap up our conversation.

So I love that you took time out of your busy day to hang out with me.

I've been looking forward to this chat all week.

uh If people want to reach out to you, what's the best way for them to get a hold of you?

I would say just go on LinkedIn, Alberto Savoia.

know, ideally you say, hey, Alberto, I like your work.

So I didn't know you're trying to spam me and try to sell me something with the next
message.

And but yeah, please reach out.

I also have a lot of YouTube videos, you know, during COVID, you know, I couldn't go, I
couldn't teach in person.

So I started to record a lot of videos because, you know, you can only fit so much
material in the book.

And, you know,

I hope I get some people, if anybody listens to this and you like a little math that
you're not afraid of it, you know, it's like 5 % of the people I talk to, ah I have a lot

of great material for people who can tolerate a little math and I don't put in my book
because I know that it really lowers the desirability, but there is a real science and,

you know, and math and logic and vision, thinking behind what David and I teach, right?

Essentially that's what we're teaching, right?

You know, the base assumption is that most of the ideas fail in the market.

So when you do a test, you do a test to get additional evidence to see if your probability
of success is actually higher than the baseline probability, is, know.

90 % probability of failure.

That's why we test.

And what we're doing can be put into numbers.

I like to say it with numbers, like in the Bayesian framework.

So we're doing it without numbers because people prefer that, but that's my passion.

So look for Alberto Savoia on LinkedIn.

And if you listen to this, because you know me and you do not know David, know, looked up
David on LinkedIn because we're part of the same team.

uh

Yeah, so it's great.

was also looking forward to talking with you.

Actually, I invited myself.

I get a lot of requests to be on podcasts, right?

And I said, Hey, no, should be in your podcast.

After so the pod heard the podcast with Patrick Copeland.

I thought, no, they didn't.

I should also talk.

So I kind of say, Hey, will you invite me?

And I'm glad it came out.

It was an easy yes on my part.

Well, thank you so much, Alberto, for hanging out with me.

uh Like I said, if you're not afraid of math and you want to reach out to Alberto, we'll
also put those links on the page.

Thanks so much for chatting with us today.

Thank you, David, and good luck to everybody out there coming up with new products.