How I Tested That

David J Bland (0:1.124)

Welcome to the podcast, David.

David (0:2.848)

Thanks for having me, David.

David J Bland (0:5.264)

I'm so excited to have you on. We've had all kinds of different entrepreneurs and innovators on the call and talking about very, very different industries. And I was really interested in your industry and we got connected through Jeff Gothelf from a huge fan of, and we had recently on the podcast as well with Josh Seiden, his co-author and partner in crime. And I was really interested in your story about how you're taking what you've learned over the years and applying it to what you're working on, but also kind of the shift from

Hey, we want to move towards experimentation. So maybe you can just give our listeners a bit of background on yourself and how you kind of up doing what you're doing with Lawline.

David (0:43.276)

Yeah, Lawline has been around 25 years from the first iteration and then 17 years from the iteration when I sort of took it and actually turned it into the company it is today. And we are the largest provider of online continuing legal education in the United States. And it's mandatory for about 800,000 attorneys to take courses every year to keep their license. We make it super easy for them to do it online on their...

mobile, you on their iPhone, their Android, and take a lot of really interesting and compelling courses. I love to build product. I am guilty of building products that I care about. Maybe our customers don't care about as much because it's just, that's my artistish, artistry, whatever, however you say that. And so there's been a lot.

of maturing over the years in that sense. But one result of that is having the best technology platform for learning, not just, I would say, in the legal industry, but I think in general for many different LMSs out there. But I'm happy to kind of pause there and then take you through some of the journey of experiments we've done. Just in this past year alone, we had our executive review on Tuesday.

13 experiments so far this year that we've been testing out. So I can go over that. I can also share some of the ones that we've done over the years.

David J Bland (2:10.053)

I would love to dig in a little bit. feels like it's a very resilient company you've built to be in business for that length of time.

David (2:18.670)

You know it's funny because at least in the beginning years I was just like, okay, let's do one more year, we're a startup, let's do one more year, we're a startup. And like 17 years later, I'm like, oh, I guess we're doing okay. So yeah, definitely resilient. And we've worked hard at that. We've worked with coaches specifically in this framework.

It's called scaling up framework where you work with a coach who helps you build systems and processes and communication rhythms and focusing on three-year goals, one-year goals, quarterly goals, and creating that around. That was about eight years ago, and I think that turned us from the spoken hub model where everybody was like, David, what should we do? What should we do to having systems and processes? The best part is the business does not.

rely on me in the slightest. Obviously, I add value and I helped move it forward.

David J Bland (3:39.965)

Yeah, I'm wondering, did this, you mentioned something about building products and your love for building products and especially for things that for you, that you see as problems in the world. I'm interested in knowing the early stage of this sort of journey with Lawline. Is it born out of problems you were experiencing or inefficiencies or maybe give us some backstory there.

David (4:6.104)

So we create our own content, we have our own technology, and that's kind of rare. Usually if you're like an education company, you use a third party platform to do this. this has been almost 20 years, there wasn't really LMSs that existed, especially in our niche. we actually work and create our own content and our own technology. And I happen to think I'm pretty good, have good ideas. And so when I would see what the platform would look like,

It's just like any kind of agile building. You build in short sprints, you see how it looks and you say, well, if you add this, this and this, it can look better. If you had this, this and this. And I just liked making the product better. At some point it became, well, you can do all this and you can add scope creep and you can add 25 different features. But if nobody's using it and it's not leading to more revenue and it's actually taking our time away, is this really productive? And so.

I think over the years, and I'll give you one example, how we've matured, there's a book called Monetizing Innovation. It's a really good book because it talks about the big issue that people have when they're experimenting and building products. And everybody almost has it, where they say, okay, I have an idea. Let me scope out the idea.

let me build the product, let me come up with a marketing pitch for the product. And the last thing to think about is, hmm, what should I charge for this product? And when you do that last, it's already too late because you haven't figured out if you've solved the need. And so the way we try to do more experimentation when it's product focus and it's big enough, we say, okay, what should we price this? Let's survey our market and how we do that. I can give you a specific example. That's a more recent example, but we don't have to jump to that just yet. I'll follow you earlier.

David J Bland (5:57.344)

Okay, yeah, so it sounds like having deep domain expertise, in your field, building things that you feel are, okay, this solves a problem for me, and then hopefully there's other people, enough other people, which sounds like there are, that also have this problem and will pay for this solution. I'm wondering, what are some of the, you mentioned pricing, which we actually had this come up on a previous podcast, which we were talking about, how do you price?

your features or packages and it's not always straightforward. sometimes you kind of stumble into your pricing. There's stuff you can do with surveys, obviously, but it all comes down to, hey, do people actually pay enough versus saying they'll pay enough? So I'm curious with early on, kind of going through trying to define your pricing.

Was there anything there where you could say, hey, I think I can back into a number here because I know this industry so well of what people are spending time on or paying or doing their own research trying to solve a problem. I'm wondering like what your thought process and how you kind of walk through that.

David (7:2.286)

We're in a different kind of area because we're a mandatory based product. And because it's mandatory based product, to some degree it becomes a commodity because there's lots of competition. There's bar associations, there's mom and pop shops, there's large legal tech companies, there's public companies that offer some version of this. And so to some degree, the challenge becomes all the features that we're adding. It's hard to actually price more for them because people are paying for the

content and these additional features may help retain more customers. They may help just differentiate ourselves, but they don't necessarily help you charge more. And I think that's been some of the, yet we continue to grow. And so I would say almost all of our innovation, except for some of the things we've done recently have been about adding more value to our product.

or coming up with a completely different product that we're trying to sell. like examples, we recently experimented and launched our own AI learning assistant. We have 2,000 hours of courses. can now, it's a very specific data set on legal content in the US. It does not look anywhere else.

Instead of having to watch a full 60 minute course, you can actually ask it questions the same you do with ChatGBT, but it specifically points to the point in the video where it gets that source from. So you can click it and actually double check it if you want to. And that was one thing. It's done okay. But because people are here to get their mandatory education and the same way it's hard to figure out what you want to prompt in ChatGBT, it's the same challenge there. And there is no need that

We went to the market for that. That was just what I wanted to do because I'm really focused on just-in-time learning. Complete opposite of that is one of the things that we recently launched and did the whole survey and figuring out the pricing model first is we have a turnkey solution that we built for ourselves. When you create a course, you have to get it accredited in every state. We have a tool that we've done that's all internal. That's really good. We have a team that does that.

David (9:17.818)

over the years of enough firms and companies asking us if we provide that service and us saying no, we decided, well, if everyone's asking for it, maybe we should do that. And we surveyed them ahead of time. We asked what kind of pricing that they'd be interested in. And we launched it earlier this year. it's our fastest growing new business line that we've had. It's a lot of work. It's not just self-service. But that's an example of sort of building that out.

in a different area. But I would say like, you know, there's a joke, you know, from doing this is like, internally, it's like, okay, it's like David's ideas. What does Dave want to do now? Like, what is what's his new idea of the month of the week? And I struggle with that because to some degree, it's like, I think I'm brilliant and it's my company and I want to be if I I'll regret if I haven't tried something I want. But on the flip side, I recognize there's a finite amount of time and resources.

And like we do, we actually changed our core value this year for the first time in a decade. We had a core value that we have five core values and one of them is create. We changed it to experiment because we wanted to, cause it's so hard many times for people. You see, people to experiment, but of course when you experiment, it means you're going to fail. And we've come up with the motto that we want people to experiment and we're for a single.

not a home run. Because we find that home runs are harder to see, they're harder to plan for, and you know, they're too long a cycle. If you plan for a single, you will finally, at some point, get a home run in some degree. But I also like Jeff Bezos mentality. It's kind of related to baseball as well. In baseball, the best you can do in anything is get a home run.

And he said, while 90 % of your experiments will probably fail or be a push, if one in 10 gets you 1,000 times return, right, is it worth failing 90 % of the time? I like to believe it is. It's easier to say that when you're Jeff Bezos, but that's...

David (11:33.678)

what we're really trying to build in the company. So we've had many things. I'll give you a last example. I'm like, guys, because we have a free trial, I want it to be a free trial where you have to beat your credit card in. Because that will convert better. Right now, you don't have do it and only like, I don't know, 30 % convert or whatever it is. I'm like, we'll get less people to sign up. We'll get a higher conversion rate.

didn't work. We went lower and I was convinced it would work, but we tried it. That one you don't have to survey for it because it's not hard to build. yeah, like I said, think we have 13 experiments. I'm looking at the list right now. This year we tried doing a seven figure government contract.

It took us a really long time to put together a proposal. There's a lot of things that we actually don't provide, but we tried it. We took a lot of time and resources to put this RFP together. We didn't get it. And we didn't think we were going to get it, but we said just the process of doing it was worth the experience and the option. And three years ago, we never went to any conferences. And today, we go to nine conferences. But it started with experimenting with one conference and then two conferences.

David J Bland (12:43.195)

Oh,

David (12:47.466)

And then one more thing, just because this is a big one, we do some different discounts in our product. We've been that for a long time. We decided we offer enough value and we have a lot of different things to do. We're going to lower the price discount that we've been doing for a long time, know, specials and things like that. And that was a big deal because

you might lose more customers or things like that. And so far, it hasn't been a home run. It's been a single. And so but you know, I thought it would be better, but I think it will over time.

David J Bland (13:22.399)

Yeah, it sounds like you're coming around to this idea of, I could be wrong, which is hard. It's hard being wrong. It doesn't feel good being wrong. even in my work, I think it's easy for me sometimes because I'm coming into companies that advise. I'm like, oh, yeah, yeah, you might be wrong here, but we'll get through it. Whereas if I apply it to my own stuff, I'm like, oh, this doesn't feel good to be wrong. I should have more empathy for the people I'm coaching, which I do my best to empathize with.

David (13:47.842)

Well, and here's where I struggle. Maybe you have advice is you could be wrong because you didn't give it a good effort, right? You could have done a shitty job in, in, in your first phase. And so I struggle with that. I struggle with where was I wrong or did I just do this bad? And so I don't know how, how you, how you handle situations like that with like, for example, the AI, I think we can do 10 more different. Is it worth continuing to?

David J Bland (14:9.214)

Yeah.

David (14:16.967)

pivot and shift and try different things or is it easier just to move on to the next thing because we don't know if this will ever get there.

David J Bland (14:24.054)

Yeah, I think, well, one, you said something really interesting where you were talking about you're trying to add value, but it might not be direct value in the sense that they're paying for what you're adding, but the retention might be better or there might be word of mouth and they bring in new clients because they're having such a great experience on your platform. I think with that proxy, right, sometimes it's not always clear or there's a lag time.

You know, yeah, you made the platform better, but I don't know what the retention rate is on your mandatory training. So if it's not a couple of days or a couple of weeks, you're waiting, I don't know, potentially what months or a year? I mean, what does that loop look like for you as far as did you get any feedback on whether or not you added value or where do you see those bumps in retention or what's the cadence there?

David (15:15.672)

We have a yearly subscription and generally, even though it's a year subscription, if your deadline is October 31st, you might come to us for two weeks and we might not see you for a year. But our goal and intention is to show you that there's so much value throughout the year, you should be coming back on a more regular basis. And some of our customers, many of our customers do, but not all of them. And because of course the thesis is the more you use it,

the higher retention rate will be and the more value that you'll get. So it's a yearly subscription. pay upfront for the year. And so yeah, that's how it works.

David J Bland (15:55.170)

Yeah, so I was working with a, well, think of like a Costco, right? So I'm working with a company like that or I have in the past. And one of the things we were doing is like, oh, well, people can explore the store. Once they become a member, get more value out of the store, you can measure that as far as like the stuff they put in their cart, you know, when they're out. But then it becomes, well, we don't really know if this worked until a year later when they renew their annual membership. So can we look for leading indicators?

David (16:13.667)

Yeah.

David J Bland (16:24.014)

that they're getting more value out of the store. There's some crazy stats of like the first path you take through a store, like eight out of 10 times when you come back to the store, you're going to follow almost like that same path. And you learn that path by watching other shoppers. So looking through that experience and saying, OK, well, we're running experiments, but we're trying to look for behavior. Is there behavior that we could then look at the renewal rates a year later and say,

here are some leading indicators. We're not 100 % sure, but here's some leading indicators to say they're going to renew. And so you mentioned something similar there, although it's in software, right? Of, well, if they're using the platform more, then potentially that would be a better retention rate or they would have a higher chance they're going to retain. And I'm thinking in that, when you're looking at your patterns, are you seeing like, oh, if they do this, this and this.

David (16:58.061)

Yeah.

David (17:9.432)

Yeah.

David J Bland (17:16.238)

there's a higher chance or if they don't use this part of the platform, there's a lower chance. Like, do you have that sort of process going on internally with your platform?

David (17:25.492)

Not really, to be honest. think the only thing we really look at is how many courses they take and when they take it. And we've built in so many different features that they can use. so, no, we don't. the challenge, where I want to do is, I think when I focus on just-in-time learning, where you don't have to watch it because you need to watch it because you want to watch it.

That is what drives me. And that's where I definitely know there's a need and there's there's opportunity there. And so I'm, sometimes on the belief that I, the flip side of being focused is like, just keep trying different things and eventually it'll lead to an idea. So I don't know. I know that's the complete opposite, but that's, that's what gives me pleasure. But the company is not here just to give me pleasure. The company is here to provide it, to service a need that.

the market has. And I've had many conversations with our friend Jeff on this. you know, it's just like the number one thing that, you know, maybe it works sometimes, but many times it's just, you it's you build something with 15 feature sets that nobody cares about except you.

David J Bland (18:43.164)

Yeah, it's lot of as much as we try to predict, right? There is an element of it's humbling to see what resonates and what doesn't. We can't always predict what's going to be a winner. But you said you had 13 different experiments. So you said talking about pricing, you talked about going to conferences. Maybe let's pause on the conferences for a moment. So I think the tried and true maybe old school traditional method of conference, you like go there, you set up a booth, a sponsor, you try to just promote yourself as much as possible and hope for the best.

David (18:52.716)

Yeah.

David (18:56.653)

Yeah.

David J Bland (19:12.966)

Some of my clients that I coach now, where I coach a lot of B2B companies, so we go to conferences, but we go there with, hey, here's a list of things we want to do, like some discovery, customer discovery, right? We might have brochures with a call to action, or we might do card sorting with people either at a booth or on an iPad, and we're doing interviews. I'm just thinking through your conference experience.

What has worked for you all as far as going to a conference and trying to understand more deeply about the people that are coming to our platform or potential people that you could bring in?

David (19:44.579)

Yeah, and so just to be clear, we have two different sort of targets. One is B2C and one is B2B, which is like a large law firm, a large organization. So B2C is like an individual attorney who finds us and needs it for themselves. They can be at any size firm or they can be solo. And so the conferences are really geared towards the B2B, the big firms, the mid-size firms, the organizations. And so many of the conferences that we go to are based on professional development for these large firms. So it's professional development.

roles who are attending or some form of that. And so, and it's a very long sales cycle in our industry. So it's like, if you go to a conference and you do it, it's about building long-term relationships. And these take years. It really does. And so what we've done is we do at least one to two dinners with a group of maybe 10 to 12 different contacts that we have, either friends or potential clients or existing clients.

We have a podcast that we actually do at our booth. So we interview people at the booth itself, which has been helpful. And then of course, yeah, we have a booth that we showcase and share what we're doing and we have collect leads and then we follow up with them after. So it's pretty basic in terms of the basic things you do there, but like, you know, it adds up. It's real cost. There's travel, there's sponsorship costs, there's all that different things. And it's just a real recognition, a realization that

you have to give a three year window, maybe that's too long, but you have to realize it's not like an online marketing campaign. It just takes a lot of time. And so for some reason, and for that, maybe we're not even in the positive in our ROI for the cost that we spend over the past 12 months for what we've gotten. We have a belief that over the long term, it does pay off.

David J Bland (21:38.783)

Yeah, I like that long-term play. mean, even with some of my clients B2B, I coach a lot of bigger companies and those sales cycles can be years. Or sometimes, you know, I coach somebody at a team in a company, they leave that company and then like two or three years later, they reach out to me and say, hey, remember that thing we did? I want to do it over here now. So I have super long sales cycles sometimes. And I'm thinking through just the complexity of your business, right? So people that show up these conferences, you you might have...

influencers, but they don't have budget authority or decision making authority. And then you might have people that also maybe it's a smaller firm. They have budget and decision making authority. But I imagine once you get those people together, you're sort of like, well, how do I navigate this to get to people who can make a decision in that B2B framing? So I imagine that could be challenging as a small firm trying to figure out, hey, how do I navigate who I'm talking to and whether or not they can actually sign a deal with us?

David (22:37.218)

Yeah, over the years, lot of figuring that out has been challenging, but you figure it out one step at a time. And of course, almost everybody already has a solution in place, right? And so it's how do you determine, are we going to be an add-on to that current solution? Are we going to replace that solution? Those are the two options. Or do they not have a solution at all? And what we found with many of them, they look at us as an add-on versus a replacement.

at least as a first step. And so like, then we have to gear towards that. And it's been successful and who we're competing with is a hundred, almost a hundred year old nonprofit in the industry who's been working with these firms for that long. So it's a hard pitch sometimes, but we're more lean. We focus strictly on online. We have a lot of tools and services that they can't provide and we're cheaper. So we have something there, but again, it's.

David J Bland (23:25.194)

Yeah.

David (23:35.850)

It's always challenging in that space to go after the incumbent who's been there for so long.

David J Bland (23:40.916)

Yeah, switching costs, inertia, you're fighting a lot of different things that working.

David (23:43.534)

The switching costs are real, for sure. And I see it with my own product. I don't know, maybe we have 30 vendors who we outsource different things to. It's internally like HubSpot being one of them. It's like, how do you, it's a challenge.

David J Bland (23:57.838)

I'm curious if you look at your experiments and just how you're testing your way through, I love that you're changing a value from sort creating to experimenting and being open up to the idea of that, we're gonna test and learn and figure this out and do short cycles and just in time. Do you find that you're just kind of splitting your focus between B to B and B to C or do feel like your experiments are leaning one way versus the other? Like give us maybe some insights there.

David (24:21.006)

Yeah, no, it's a great question because I'll give you like, it probably is close to 50-50. B2C is 90 % of our business, but in a way we see the biggest opportunity in B2B. So it is a kind of a push and pull, right? Because for obvious reasons that can grow faster. And so one example like B2C that we did this year was we focused on doing a cart upsell.

for a product that we have. You can buy three tiers of subscription. And so we added in a upsell if you buy the lowest tier. We promoted the tier above that in showing the value. And it's worked out really well. It was a good experiment. It wasn't that hard to do. we did it. But it took looking at data and getting that done. At the same time, for the B2B, I'd say

These experiments are always usually bigger and harder. Like the US government contract was a B2B experiment. Launching this new product, the Sealy as a service was a big experiment. Conference strategy was a big experiment that take time, energy, and focus and longer to see if it works or not. And so that does take a lot more of our energy where you're just doing a...

a shopping cart upgrade. We added video previews on the site, like an experiment, right? But it's not, if it goes wrong, it's not the biggest deal. We added learning paths on the site, which is another experiment. We added, what else did we add on here? We add, again, this is on a backend, we added a new way to track our data. And part of it, some of these things, and I wonder in this, your thoughts, some things maybe you don't call experiments. Some things were just like changing, right?

If we're not, like if like adding the new data, is it an experiment or it's just like, okay, that's something you're doing differently. Cause I guess if it doesn't work, are you going to remove it or you're not? But I think we think of them as experiments. Oh, the latest thing for me, which now it felt like a mountain in the beginning, but now it feels like a mohull, mohull, mohull was launching my own podcast. And I did that just, just in the past couple of months. And it's.

David (26:40.934)

been on the, you know, first, you know, it's like, what technology do I use? What process, what systems, how am I can promote it? How do I edit it? And like, and then how do I communicate with them afterwards? And you have a great system. I've been part of that. But at first, like, I'm to do a podcast. That's no big deal. That's easy. Now that I'm doing it, oh my, oh my God, this is crazy. And for me, the energy that I get and the feedback that I'm getting from that experiment.

That was my goal. Part of it was most, I'd say 50 % was how it forced me to connect with my, like that's the whole point, right? Now I mean, you're connected, I bought your book and that's why you do podcasts. And the other 50 % is hopefully you can get a following, will do it. But what I love about those type of experiments, what felt like insurmountable, know, podcast, now that you've done it, you're like, oh, that was so easy. Anyone can do that. And I think that's...

One of the most fruitful experiments we had in our company was about eight years ago or seven years ago is when we went from just selling a bundle or a course and we sold a recurring subscription. And that was a big experiment that did really well and that was transformative for our business. Another experiment I did instead of doing the discount that I was mentioning before, I said, let's just forget about the discount and give it at the lower price. And they don't, let's just not have a discount. Here's the lower price, that's that.

that failed miserably because people like discounts. They like to know they're getting value. So I was shocked. It was like, okay, great. We'll do it, go back. But I hated selling on a discount. I just hated doing that. So yeah.

David J Bland (28:20.519)

It's interesting with the experiment framing. I've had this come up a couple times in recent podcasts because I use really broadly the term experimentation. But I do think you ask yourself, and I kind of picked this up from Teresa Torres, who I'm a huge fan of as well, who's like, okay, if this passes or fails, does it change our behavior? And there's these things you just do, right? it like...

you're not treating it as an experiment, it's just something we're going to do. But then there are other things where it's just there's so much uncertainty. And that's how I've been framing it recently. Like, where do we have a lot of uncertainty? And we need to kind of just like do these little probing tests, like kind of just in time to see, okay, is there something there or not? And if there isn't, we wind it down or we toggle it off or we're not going to just leave it bloated, know, bloated mess in our product. And I think that's that questioning of, okay, if this passes or fails,

or what happens is that gonna actually shape our behavior and change what we're doing? And then also, are we anchoring the stuff we're testing on sort of the stuff where it's really important to the business, but we also don't have a lot of evidence that it's true. like, know, packaging and pricing or putting out a new product or a new bundle, right? Some of those.

It's not as certain, your expertise isn't going to necessarily just lead you through all that. So that's helped me guide me a little bit through the process. So I'm trying not to treat everything as an experiment, and something's just work, and it's not really worth trying to frame it as an experiment. It's more of, okay, this is just something we have to do.

David (29:40.035)

Yeah.

David (29:46.380)

Yeah, totally. you know, the thing about like, think Google maybe was famous about this or I mean, I Apple maybe was, but I know also Jeff Bezos is shutting something down that doesn't work. if you, think Amazon did this with a couple of things, something with, I don't know if it was the Kindle, some version where they spent a significant amount of money, it didn't work and they just shut it down. And

I think what we have really worked on internally at our company is letting people know it's okay if we shut things down. In fact, that's what we want versus feeling like they've wasted their, that's the challenge. When you do experiments and developers build it and people put energy in it, and then you just shut it down, it didn't work. They just feel like, what am I doing here? And then you start to question that, I shut it down too early? Did I give it enough runway? Did I do the experiment badly? Should I've?

than a smaller experiment. So those are some of the things that you think about.

David J Bland (30:47.605)

Definitely, yeah. And I think time boxing things or maybe have some kind of criteria if you can measure, you we want to hit like this number if possible, but you can't quantify everything. But I think one of things I had with one of my coaching calls earlier this week with another client was very telling. was the CTO at this like billion dollar company I'm helping in it. And she said, we can go faster by shutting things down. And I thought that's a great framing. Like you can actually increase your speed by

David (30:57.934)

Yeah.

David (31:11.950)

Oh

David J Bland (31:16.351)

stop working on the things that no longer are viable. And I had never heard it framed quite like that before. And I'm like, yeah, I'm taking that because sometimes we feel like, we can't shut that down. There's like some cost fallacy. We like invested this much in it and therefore we have to keep it around. But then it becomes this like, it's just something that's gonna break at some point. You have to maintain it. It's like death by a thousand little projects or features that aren't quite creating value, but they're not losing value, but they're there.

They have some usage, but not much. And I find that sometimes we can just speed things up by just cutting our losses with some of this stuff. It was a really interesting framing that I kind of stumbled into this week.

David (31:56.980)

Oh, there's an argument, at least in my business, we could save a lot of time, energy and other things by almost doing none of the experiments that we're doing because a lot of them are, like I said, are about adding more value to our product, about doing other things that probably based on the theory of only 10 % actually turn into something worthwhile, our waste of time and energy based on actual growth.

But I'm not wired that way. And then what are we doing here? And maybe we'd be more profitable. Maybe we'd do other things. It's just, it's not who I am. And I still believe in that 10 % opportunity of really hitting things out there.

David J Bland (32:40.728)

Yeah, but I think if you hit one out of 10, I mean, and even if the base hit, that gives you something. And I'm convinced even the things that don't work, you're going to learn from doing them. If you framed it in a way where you're trying to learn about something that we don't sure if it's true. Yeah. So it's fascinating how you're doing this in what I would consider a heavily regulated industry. Like, what kind of advice would you give to people who say, well, there's no way I could experiment because, I'm in.

David (32:54.388)

For sure. For sure.

David J Bland (33:10.625)

I'm in legal or I'm in compliance or regulatory environments and there's no way I could experiment like what's your message to folks like that that are listening that are in similar industries?

David (33:19.502)

Well, I think right now we're all in some form experimenting with AI, especially in legal. This is, I think the first time that legal from based on sort of some recent data that I've seen is actually ahead of the rest of the industry. Other industries in terms of integrating AI into a lot of the tools from e-discovery to contract management to research because legal is about data and information and AI is really good at helping you sort through that and find.

the tools that are there. So to some degree, in order just to keep up with how fast things are going, if you're not thinking about how to experiment, you're going to fall behind a little bit. But yeah, I wouldn't say we're a regulated industry. We're more accreditation. So I think there's a big difference. regulated, yeah, banking.

I wouldn't even know what to tell somebody how to experiment there because that's above my pay grade. But we are dealing with certain rules that they have to follow in order to get a certificate for their course. So it's a little bit simpler. we can't, right now, unfortunately, it has to be at least 60 minutes to get one credit. And you can't fast forward. It has to be video, has to have a certain amount of slides, and it's a little bit antiquated.

either I can work on changing the rules or I can provide that and then add layers on top of that or next to that or in front of that. And the question becomes, because those are not the actual core, do all those layers that you're adding around actually lead to more revenue, more retention, or are you just kind of going in circles a little bit? And so that's how we go with it. yeah, so I'd say, yeah, if you're at any other...

And as a lawyer, would just say, yeah, the biggest thing is unauthorized practice of law is what a lot of legal tech companies are concerned about. Because I don't know if you know about that rule, but it's like the number one thing that keeps non-attorneys from actually selling legal services. with technology right now, like legal zoom, rocket.

David (35:37.163)

lawyer and other companies like that were helping provide wills and estate planning and other things like that. technology is there at the point where unauthorized practice of law is actually a hindrance to access to justice and access to legal services. There's a statistic only like 10 % of those people who need legal services actually get them. And so there's a really big push right now.

to shift and change unauthorized practice of law to non-attorney-based companies, legal tech companies, to provide legal services. And the challenge, of course, is from bar associations, because it's a monopoly, right? And they're trying to protect their living. But that is a big area that we're seeing change to some degree. And then integrating AI and breaking up the billable hours, the other area.

David (36:35.770)

from a small firms, there's opportunities, small to solo, it's like, building by the hour can only, you can only build so many hours a week, right, as a solo practitioner. And it's a much better process if you can, and I've interviewed a lot of firms who've done this, if you can do value-based pricing based on projects or other things like that versus, okay, this is five hours of work, here's this, and that.

That is another area worth focusing on innovation, but it's not easy to do, especially when you're going to the large law firms. How do you tell a room of millionaires that their business model is broken? And there's a point to that.

David J Bland (37:17.969)

Yeah, so summarizing that, seems as if, yes, you have constraints and some of those you're not able to remove, but that doesn't prevent you from testing around the constraints and making the overall experience better and increasing value. You obviously have some limitations you have to work within to get the accreditation. But, you know, just like you mentioned earlier, hey, we can have an AI or agent.

that will let you get to certain parts of videos that you can go back and maybe review or you can jump around a little easier. And those constraints don't prevent you from testing in that manner and adding more value that way.

David (37:51.042)

Peace out.

David (37:58.542)

Correct, and even with the AI agent, one experiment that we haven't gotten to yet was like, separated into another product, where you can just pay a low monthly fee, cost, and you can have full access to that without everything else, so.

David J Bland (38:18.748)

So what excites you about, you know, some of the big assumptions you want to test like coming up over the next year or two?

David (38:30.926)

Yeah.

David (38:38.924)

Yeah, so we just had our strategy summit a couple of days ago and there's three areas that we're going to be experimenting in 2025. And I'm actually really excited about all of them. And they're based on the assumption that the legal landscape is changing so quickly. We need to one, help prepare attorneys for the future of law. And we need to partner with them in where they're at now. And so in order to do that, we feel that on demand education

needs to be supplemented through more two-way based communication. Let's say also through coaching and through more written based communication. Because right now we're all video based and generally even it's a lot of webcasts, it's one to many. And so we see as an example, imagine faculty hours, teacher faculty hours and being able to have

conversations and ask questions to the faculty versus just taking their courses. Or imagine lawyers who are in transition in their careers are able to work with a coach for several months or a coaching group for that process. Or imagine with AI on the course, have 75 hours of content, video-based content in AI. Imagine we can, from that transcript, from those courses, from working with the faculty, create for each course three to five

specific very focused articles and written.

detailed things that attorneys can look at and use versus having to watch the whole course, the times that by all these courses, those are more options than availability at value. The question that we have and the experiment that we haven't figured out yet is what is for adding more value to the product and what is for adding more increase in adding a new product that is a different price point. That's always the challenge that are there, but those are some of the things that we haven't fully fleshed out yet, but I'm excited to start thinking about. I'm open to any advice.

David (40:40.272)

you have.

David J Bland (40:42.252)

Well, think separating it out and breaking it down into smaller chunks is always helpful. And I think it's somewhat challenging to do that. But if you think around, I always use those themes like desirable, viable, feasible. So like what kind of risk around those areas do you have around your customer? So B2B customers, B2C customers, what kind of risk do you have around pricing and value? What's the risk versus just adding value versus spending that off in a separate product and charging a separate price for it?

And then there's also feasibility, which usually isn't the biggest risk. But you know, I mean, you're relatively small group. it's, you know, how do we prioritize and do this internally and execute on this really well? So I like those themes. They tend to work for me. And, you know, sometimes I realize, you know what, our risk isn't in building this. We can build kind of anything as long as we keep it reined in. It's more of.

David (41:20.045)

Yeah.

David J Bland (41:32.172)

Do people have the need and will they pay enough if they have the need? And that's usually where I'm finding a lot of the risk and while the experiments gravitate to is just really deeply understanding, will they switch and use this and pay enough for it? And that seems to be where the majority of the risks just ends up being as what I'm seeing.

David (41:50.030)

And your is your approach that is survey customer feedback questions to get to those answers before they build it.

David J Bland (41:59.786)

If we can, I usually, but I always caveat it with what they say in an interview or what they put in a survey is really light evidence. It's not necessarily how they're going to behave when you have the thing there. And so what can we do to sort of like find our next best test to get a little better evidence from them? So it might not be building the whole thing right away, but it might be, you know,

David (42:9.868)

Yeah.

David J Bland (42:18.969)

B2B, I see a lot of co-creation. Like, hey, could you be a part of our customer advisory council or something and we give you a preview and you give us feedback on it or we do some sort of preference and prioritization where, hey, here's what we're thinking. Can we prioritize these with you and get a better understanding of what you think is value here or not. With B2C, it's more of, okay, what if we had a clickable prototype version of this or maybe something that's kind of...

David (42:24.268)

Yeah, yeah.

David J Bland (42:45.240)

Wizard of Oz where we have to do it behind the scenes, but we're delivering value to them. We haven't built it all out yet. And just kind of like step through it methodically that way. I find it gives us at least more directional evidence of whether or not we decide to go all in and build some like build some kind of MVP or even something that's scalable. It gives us a little better evidence because wow, so many times people say they're going to behave a certain way and then.

David (42:50.860)

Yeah.

David J Bland (43:9.422)

Whatever reason, cognitive dissonance, there's all kinds of like behavioral science reasons why they don't, but it's really deflating when you think you have something and you release it and it's the crickets because the people that said they would act a certain way and bite something don't end up doing it.

David (43:13.592)

for sure.

David (43:27.330)

for sure.

David J Bland (43:29.090)

Yeah, so hopefully that helps a little bit, but I think you're really excited. It's really interesting areas to explore. I love the fact that you're really looking across the AI landscape and saying how it's going to impact legal and how it's already impacting legal. I think it's such a fascinating space going into 2025 and beyond, and I just want to really thank you for sitting down and kind of walking through some of the stuff you've done, how you're thinking about things, especially in your space. You know, it's a really interesting space where it's mandated, but you have these.

David (43:31.596)

Yeah, no,

David J Bland (43:57.979)

really established competitors have been around for a long time and you're trying to find a way to test through it, but you seem to be, you have this really interesting mindset to it and you seem to be very resilient. And so it just shows through the success you've had in the company.

David (44:8.174)

Yeah.

David (44:11.790)

And we have that startup mentality that we fight against with like, okay, maybe it's easier not to be just to slow things down a little bit. But yeah, I'm excited. I bought your book. I'm excited to kind of leaf through it. Any advice? I know it's not meant to be read like in one sitting and then you're supposed to go to any advice for me or anyone else listening. How it's like, you know, I mentioned these experiments that I have coming up. How should I approach like the different examples?

I don't know, 80, whatever is 80 or so that you say to do experiments. Is that something that would be useful for the examples that I said to you?

David J Bland (44:48.708)

Yeah, I think what I would recommend is definitely not a front to back book. So I think looking at the beginning of the book, there's a section on assumptions mapping, which I actually learned from Jeff and Josh back at NEO, which is just a two by two where we kind of prioritize risks. It'd be really interesting with your team to say, hey, do we all agree on what's really risky here? Like the most important things that we don't have evidence on with and pertaining to your strategy. And then I would probably start with the discovery experiments.

they're kind of categorized in discovery versus validation. And so these open-ended experiments of how do we get a little better evidence beyond surveys and interviews? So there's subcategories there. I think for B2B, the preference and prioritization section would be really valuable to you. And then also looking at things that are tagged viability, because that's a lot about value and pricing. yeah, I think.

David (45:20.269)

Okay.

David (45:32.267)

Okay.

David J Bland (45:41.372)

I would just jump around a bit. would look at the two by two and then see are there experiments that match the kind of risk that you're talking about.

David (45:46.702)

When you say two by two, you mean like the urgent important type? Two by two? OK. Got it.

David J Bland (45:51.016)

Yeah, it's more like importance and evidence is the way I use it. So it's more like how important is this to the success of this thing that we're potentially thinking of building? And then do we have any evidence that supports what would have to be true? And I imagine some of it like you already have evidence and it's not a giant leap, but some of it there might be things like, hey, I actually don't know if they would pay this price point or if it was a different product, would they sign up for it? So I think.

David (46:3.682)

Got it.

David (46:15.224)

Got it.

David J Bland (46:18.118)

Just, I mean, usually that two by two doesn't take very long to do, but it would help anchor your experimentation. And so you're focused on stuff that, you know, can make or break the new product or some of initiatives you're thinking about.

David (46:29.070)

Awesome, thank you for that and it will keep you posted how it goes.

David J Bland (46:32.536)

Yeah, please do. So for those that have listened and are really interested in reaching out to you, what's the best way of them to get in contact with you?

David (46:41.400)

Truthfully is LinkedIn. I do a post almost every day on LinkedIn because that's, just like, it's almost like I journal through LinkedIn about what I do. so you can, and I usually reply to those messages very quickly. So I would just go to David Schneurman on LinkedIn and then we can connect and go from there.

David J Bland (47:3.518)

We'll also include the link to David's LinkedIn on the detail page for you all, so you can also get to it that way. Thank you so much for hanging out with us and going in behind the scenes. yeah, good luck on your experimentation in 2025.

David (47:5.985)

Yeah, am I LinkedIn plus?

Perfect. Thank you. Thanks, David.

David (47:16.497)

Thank you, I'll keep you posted. Bye.