Speedscale ‘SpeedChat’ Episode 1: Discussing software product management, ‘dogfooding’ and scaling quickly for product/market fit without breaking things.
Show Links
Listen or download the podcast on your favorite player here:
Watch the Youtube version here:
Full Transcript
Jason: All right. Well, welcome to speed chat. The Speedscale show, where we talk about all issues related to software quality, reliability, resiliency and making production work better by testing better and not impacting production. Anyway today’s show we have I’m joined by Leonid Igolnik and Matthew LeRay — Matt LeRay. He’s the co-founder of Speedscale and Leonid is an investor and a fellow advisor for Speedscale.
We have a very interesting topic on tap today. So, take it away.
Matt: Okay. I think the topic of conversation for today is generally the importance of dogfooding in your product, right, as you’re building the product. And I I’ll kind of expand on that a little bit is how, how you can use dogfooding in lieu of some of the heavier product management disciplines, right.
As things as you get bigger. So so anyway Leonid you know, missing from Jason’s introduction is Leonid is the former head of engineering for SignalFX as well as I think Taleo and did it stand at CA APM bunch of different places? So Leonid, I don’t know if you wanna introduce yourself or talk about the dogfooding topic.
Leonid: Yeah, no, I, it does matter, my background, I used to be a software engineer once and then switched to leadership, ran engineering for a number of products at Taleo when we were still convincing the world that SaaS is the thing. And by the way, SaaS is kind of interesting cause makes dogfooding easy in some ways. Most recently around engineering for a company called SignalFX, all the way through their acquisition by Splunk. And depending on the market, I think dogfooding is a great tool in various ways, but also could be a fairly dangerous tool. If you don’t use it right. And I’ll elaborate on that in a second, I’ll give you a few examples. So let’s use Taleo where we were building and selling HR software.
Obviously, every manager at some point has to touch HR software, but this is a piece of software that you’d get to touch infrequently. And most of the engineers that are working on the software, I have a harder time relating to the user and the buyer because they don’t use this thing every day.
Let’s compare and contrast to something like SignalFX. Which was a monitoring system for modern architectures, for modern cloud applications where we were monitoring SignalFX with an instance, a dedicated instance of signalFX and all of us on the engineering team on the product management team on the sales team, pre-sales team were using SignalFX, multiple times a day.
Now it has tremendous benefits for a startup. Number one, you get to use your own product and kind of get to understand how awkward, how easy certain things are. It’s great for recruiting, right? It’s very rare in very, very few circumstances software engineers get to work on the software they actually get to work on.
To use everyday rather, right. If you’re working on a payment processing system for Visa, you may encounter it as an end user, but you probably not doing chargeback processing as part of your daily life. So there’s a class of software where a dogfooding is great for recruiting. We also talked about dogfooding as the way of understanding.
How your software is being used and what’s awkward. What’s easy. I think it’s great. The important thing in that type of dogfooding to, to understand is not everybody is like you, especially if you’re selling into companies that don’t look like you’re not organized, like you may not be developing software like you. You have to remember that your sample set of one and dogfooding is a great complement to product management.
Like. Responsible and methodical product management, but , I would never consider that an adequate replacement just for that reason. Right. Not everybody is like you, and it’s very easy to forget that not everybody does software like you do. So maybe start there, Matt, what do you think?
Matt: Yeah, so kind of rapping off that so the best in my opinion, the best product managers are people who represent the the target personas. So like in the case of speed scale, we make tooling that is useful for SREs, as well as feature developers, people who build new software. And so it’s been super helpful for us in particular, but like, as you said, Leonid, that’s because we are the people that use this thing.
We’re the people that actually have the pain and so what it allows us to do, dogfooding it allows us to make certain shortcuts in the design process, around the software specifically around ergonomics and, I like using the fancy word ergonomics to describe basically — what is the user interface for it?
So, as an example, one of the things we were able to do is, we have a command line tool called a SpeedCTL, right. Which is very common in the Kubernetes world to have this sort of CLI interface. And we started there because we knew that people were going to integrate Speedscale into like a GitOps workflow.
And so we were able to, for the first year, the company developed very little user interface because we knew that people would put up with, you know, not having the greatest UI at least initially. And we knew that because we were the people that were going to get use out of it. Right. Yeah. Please.
Leonid: No, that’s awesome. And I agree with you Matt like this dogfooding certainly helps prioritization. like you just described. So, the question really is for those who are not fortunate enough to be able to use their own product and their daily life, how the hell do you dog food? Right? What do you do?
And I want to go back to some of my earliest experiences as a software engineer, the first product I built for which I was compensated commercially was a billing software for an ISP. For which I was working. And I remember distinctly in, I guess intuitively I realized that that’s the thing to do because I didn’t think anybody taught me that –I would go and for half a day, spend a time sitting behind the computer itself.
People that were using my software within the ISP, like processing bills and invoices, and just watch them use that software. And I think it’s a very important things that a lot of startups that don’t have that position of using their own software don’t leverage well, because they either too shy or think it’s not a productive use of time is not observing the users, ideally in their natural habitat using their own software.
And while product managers I think are very good and professional product managers. Absolutely very good at representing the persona of the users. I think it’s also important to equip the individuals on the engineering team with that understanding of the persona. So more articulate and perhaps more data fact-based arguments about pros and cons of a particular feature design can ensue.
Because they’re based on concrete observations rather than opinions. And I don’t know if you have any other advice for folks that don’t have the product that they get to use every day. There’s some calories to be spent here.
Matt: Yeah. So now you get into kind of the the professional product management, because I think that professional product management begins where opinions if you like an example when I’ve encountered a lot of startup founders who are very opinionated about things they don’t know about.
Right. And I try to avoid this, this particular tactic for myself is it’s okay to be super, super opinionated when you are the persona. But like you said, eventually you’re going to branch out and you’re going to get into where you need to have empathy for the users. And that’s where I think you start to get into more of a professional product management process, which, you know, you’re going to go through.
You’re going to go through the research phases. You’re going to go through, you’re going to run surveys. You’re probably going to run experiments. I know at Speedscale, probably 75% of the code that we wrote, we ended up not using because they were just experiments that failed.
Right. And then until we found that 25% of that was really core. But I’m interested , Leonid in what you know, I think that this kind of gets into like the next stage of startup scaling. Right. I’m curious what you guys did at SignalFX and in other successful product management organizations.
So, I think any product development and you talked about being opinionated, I will challenge and maybe disagree with you. I think having a point of view. And opinion on the market is important. And if you’re lucky enough to have the correct one that the market agrees with, you’re going to be incredibly successful.
You know, if you’re not lucky then things may go different. Now, there are ways of informing that point of view, and we just talked about methodically observing and understanding the needs of the user. Right. And I think it’s also important to realize that as the company grows as the number of people making tens, if not hundreds of small decisions every day grow, that empathy to the user is one of those things that can fundamentally improve your ability to build a product that somebody wants to buy.
Leonid: Right. And the question becomes, how, how do you achieve that, at scale? Right. And when I say scale, typically those things tend to break when the engineering team approaches 40 to 60 people. And this is also typically a stage of the company where a big round happens, a bunch of money being poured into the company.
There’s a blank check. Hire as many engineers as you can, you know, we’ve got to capture the market and the question becomes, how the hell do you scale all of this? Like, how do you double an engineering team from 60 to 120 without losing that empathy to the market. And again, this is where that empathy has to become professionalized and disciplined, right?
Bigger companies have access to resources. To conduct professional research. When, for example, Matt, when you and I were colleagues, one of the projects that we worked on that was fairly successful turned around the major multi hundred million dollar business spent $750,000, I think on research, like no startup can afford that, but the techniques that were used by us at that level given what was at stake are absolutely applicable to even the smallest startups.
And it’s really about not understanding what a user will do. Or wants to buy or what features you should be building, but a deep and, insightful understanding of durable problems they need to solve. And understanding that by observing them and asking them how they solve those problems, not how would they solve the problems in the future?
I think you know, asking people about what they would, they use it particular capability of the software is as good as asking them whether they’re going to stick to their new year’s resolutions. And we all know how that waltz works. People don’t change behaviors. Right. And that’s another important, very important thing to remember.
Now, the most interesting thing is. Not just, this data and those observations are not just useful for designing the right product, but they’re also useful for downstream activities, such as messaging and sales. And Matt, I know you have a unique perspective on how those insights, not just inform what’s in the product, but also how do you tell the story of the product you want to share some of that?
Matt: Yeah. So my opinion, when you’re early stage we look forward to having the problems you’re talking about at Speedscale, where we are here, we’re trying to hire every single engineer. You know, going into the massive funding round, we hope to have those problems…
Leonid: As an investor. I’m looking forward to that problem as well!
Matt: That’d be lovely. But so early on one of the biggest issues around like you’re talking about user interviews, right? Users, they tell you what their problem is, but if you ask them to tell you , how to solve it, they’re, they’re not going to tell you the right things.
So what we did at Speedscale, I think it served us well early on is we said we have certain product principles and trends in the industry that we believe are catching that give us an asymmetrical advantage to anybody else who’s working in the space.
Leonid: So that’s that opinion, right?
Matt: Exactly. Yeah. And it’s targeted, as you said, at a durable problem, right. This is a real problem. We know, cause we’ve, we’ve done enough research there. So we, wrote them down. Then we said, these are the durable problems we’re solving. And then here’s what we think are the major trends.
And if you look at our website, you’ll see, we believe that when you design a system with a cloud data warehouse as the backend, you’re going to build something fundamentally different, right? Even if it’s not a professional cloud data warehouse just you’re using like Amazon services or whatever, you’re going to design things differently.
And the whole product approach is going to look different. So we believe that that’s sort of a driver. Right. Not just cloud technologies, but more like cloud data management technologies. A second thing that we believe. So, so part of that is part of not just what you believe is also what you don’t believe.
And so we like to talk about how we have certain contrarian viewpoints. So one of those contrarian viewpoints is that developers don’t want to — don’t want to write tests. They just don’t want to do it. And I think that the existing theory is that, well, they have to though. And we say no, what if, what if they didn’t have to?
What if you built a product so that don’t have to write these tests any more to, in order to achieve validation? Another sort of contrarian viewpoint, I guess that we have is that , the new techniques like canaries are incredibly important. We do them. They’re awesome.
You should do them. But they’re not going to solve the problem for every type of enterprise. You know, that that’s sort of a contrarian viewpoint because everybody’s running towards that, but it’s just not our experience. Right. So what, but having those principles written down and understanding that they’re, they’re sort of implicit in the product design allows us to decide what not to do.
What to take out of the engineering design , what to take out of the product management design. And I think as you, when you’re a big company, you don’t have to do as much of that because you have more resources. At least I think.
Leonid: I, I don’t know, I’ve worked on products, all kinds of things.
You’d never have enough resources to cover your entire roadmap. So I agree with you though. I think it is as important to use both. Dogfooding and those methodical user interviews to decide what to put in the product. But a lot of startups fail by not realizing that it is as important to decide what you’re not going to do.
What you’re not going to be? It’s very tempting to go chase a deal with a bell and whistle. Right. But and there’s, there’s always a balance that there’s no black and white, but it is important to inform that decision based on facts and numbers. And the only place I know where to get them is. Go talk to the people that will be using this software.
One other thing that I see a lot of startups made a mistake on as in a typical enterprise software there is the buyer and there’s the user. And while they may look alike, those are two distinct personas was different priorities and different facts of life, right? The head of operations or the head of engineering.
Likely the signer on the check for your product, but it’s not necessarily the end user. And if you want to be in my experience, if you want to be successful as a product, you have to understand the user, not the buyer. What do you think Matt? Well, what, how have you seen that experience? Have you seen that delineation between what the buyer wants and what the user really needs in your interviews?
Matt: Yes. I have seen it at a large company and I’m very grateful that that problem exists. Because I think when at large companies it’s extremely difficult to fight. So one of our advantages of startup founders or startup engineers, Is that we we can target the user without paying attention to the buyer.
I think it’s an old — I don’t remember which musician said it, but they said, you know, I had 10 years to work on my first album and that’s why it was great, but I only had two years to work on the next one because I had to make money. And that’s why it wasn’t as great, you know, they took, they were kind of describing the sophomore slump and sort of albums.
And I think the same thing applies in engineering is that you have sort of a blank canvas to go and tackle these different issues, right? When you’re an early startup. And so what you don’t have to pay attention to is being profitable, right? We’re worrying about business cases as much right? Now, the downside of it is you can get enough rope to hang yourself with, and then your startup fails.
But. If you’re smart about it, you can really build something that is tailored right at the next problem that’s coming. The one that the big company can’t think about the one that they’re too busy, in satisfying the buyer, you know, in my own experience, you know $10 million deals, right? Sounds wonderful.
And they are but at the same time, they are they can be, if not managed properly, they can be crippling to a product and engineering organization because you end up just doing what sales tells you. And if you fall in that trap, then you might as well just just be order taking and running spreadsheets instead of being in the creative process of software engineering.
So
Leonid: So we started on dogfooding. We grabbed, we went to all around user research and kind of understanding how, what to put in what to put out into your products. I guess we have to summarize like dogfooding is great, but I think remembering that you’re a sample set of one is incredibly important, remembering that there are users and there are buyers, and they’re not always the same cohort.
As well as understanding how to extract the durable problems while having an opinion like it seems Matt, both you and I which is unfortunate that it would have been more fun to have controversies in a violent agreement around those areas. What else do you think folks need to take out of this conversation if they’re thinking about dogfooding and a kind of product design?
Matt: Well, I add one thing to it is Product market fit. Everybody talks about it. Nobody really knows when you have it. I don’t know when we have it necessarily, but one thing that strikes me is that on the other day . So one of the things that Speedscale does is we record traffic from live environments and we allow developers to reuse it.
And I found myself working with a big restaurant chain, and I couldn’t figure out what was going on. I couldn’t tell if it was something we were doing, something they were doing. And they actually used speed scale to solve that problem by bringing traffic from their environment, right onto one of the developer desktops to analyze, to figure out what’s happening and run basically equivalent of like unit tests against the real production environment.
Without actually having to touch production. That’s the one thing I’ll say is that you can, dogfooding in a way, if you are the right persona, you’ve satisfied all the other things that we’ve talked about, it is an indicator, one early indicator of product market fit.
You’ve got something that is a, that if you can dogfood and it really is useful, you still don’t know if people pay for it, but you know, you’ve got a real product on your hands, so. I think we’re running up against time here.
Leonid: So I guess on that note, we’ll toss it back to Jason.
Jason: Yeah. Fascinating discussion guys. I really appreciate it. I liked the I liked this transition from dogfooding to empathy, to product market fit and back. So appreciate the time and and maybe we’ll see you on a future episode.
Matt: Thanks.
Leonid: All right
Jason: so annnnd scene..
Many businesses struggle to discover problems with their cloud services before they impact customers. For developers, writing tests is manual and time-intensive. Speedscale allows you to stress test your cloud services with real-world scenarios. Get confidence in your releases without testing slowing you down. If you would like more information, schedule a demo today!