In the recent episode of ‘Beyond the Code, Host Kovid Batra welcomes Marian Kamenistak, engineering leadership mentor & coach, founder of Augment Lab, and a former VP of Product Engineering at Mews. The core of their discussion revolves around boosting engineering efficiency with data and empathy.
The episode initiated with a fun, fireside chat, allowing audiences a glimpse into Marian's personality beyond the professional realm. Moving forward to the main section, Marian delves into the obstacles he faced in his engineering journey and details the strategies for overcoming them. He highlights the significance of maintaining a data-driven mindset as well as underscoring his three core principles: transparency, trust, and prevention. He also elaborates on two distinctive terms - 'Roadmap contribution' and 'Epic Cycle Time’.Wrapping up, Marian imparts valuable insights for engineering leaders, emphasizing key considerations to drive efficiency within their teams.
Kovid Batra: Hi everyone! This is Kovid, Kovid with a K, your host at ‘Beyond the Code’ podcast series by Typo. Today with us, we have an interesting, amazing guest from a really amazing place called Prague. He has 15 years of engineering leadership experience. He has been a former Vice President at Mews. We would love to learn and hear your thoughts today, Marian. Welcome to the show.
Marian Kamenistak: Thank you, Kovid. Thank you for having me. I’m looking forward to run this show.
Kovid Batra: All right, Marian. So, before we get started with the main section of how to use engineering metrics and how to have a data-driven mindset, I have a quick fireside chat to know you more, okay?
Marian Kamenistak: Okay.
Kovid Batra: And you have to be, like very comfortable. Share your real thing, okay? We would love to know, our audience would love to know you. Are you ready for that?
Marian Kamenistak: I hope I’m ready!
Kovid Batra: Oh, no, it’s going to be light.
All right. So, we’ll start with the fireside chat. The first question, which is your best memory from childhood about coding?
Marian Kamenistak: Oh, coding! Yeah. The thing is, like I started coding at age 11 approximately, and it was not really like, it was the oldest PCs, if you know what I mean? So, by that time, if I wanted to play games, I had to, you know, uh, basically run it from the audio cassette. That’s the first thing.
Kovid Batra: Yeah, yeah.
Marian Kamenistak: And the second thing of course is that, you know the graphics was not really exceptional, but I was lucky to have some sort of coding book where I went through the tutorial step-by-step. That was the first like, you know, enlightening of saying like, you, this might be the right thing to do and to play with around. Yeah.
Kovid Batra: Great, great. I think starting to code that early would have been amazing, man! I think nobody at that point of time probably would have known much about it. So, that’s really interesting.
Marian Kamenistak: Yeah. And the thing is, like the computer was not really ours in my family. It was, you know, my father’s gift. He’s somewhat moved it from his office… To home sometimes from time to time. And, I was benefiting from that experience, right? And then I was, like really asking him hard way to bring the computer every single time after he’s back from work, right? So, which was really comfortable in the end.
Kovid Batra: So, now you know how legends are born, right?
Marian Kamenistak: Yeah!
Kovid Batra: Great! Moving on to the next fun question. Now, this is my favorite. Which one’s your favorite Netflix series and why?
Marian Kamenistak: Oh! There’s a couple of them. To be open, I’m not really watching Netflix or, you know, Disney anymore cause I’m really strict when it comes to my schedule. Nevertheless, if I do have some time, I, last time I remember I was watching, I think it’s called ‘The Sandman’. The Sandman is really like, you know, something that I was really astonished to see cause, you know, the story is amazing. Plus, you know, the graphics, that’s something which really caught my eye. But nevertheless, like, you know, the story about, you know, the dreams and the way how to think of some, you know, parallel universe, that’s something what was really mind-blowing in my case.
Kovid Batra: Thanks for sharing that. I think we have got a few more to watch on Netflix now.
Marian Kamenistak: But, I’m sure there’s so many better series than this, but, you know, I’m already out of the current time. Yeah.
Kovid Batra: Perfect. Last question. How do you unwind your day?
Marian Kamenistak: Oh! You know, when I’m usually finishing my day, I’m getting pretty tired, to be open and honest, because most of the time it’s mostly advisory to specific companies or mentoring. And, there is, by the way, one thing I found out, which is I can’t handle more than, I would say 3 to maximum 4 mentoring sessions a day, because it’s as intense as handling interviews and, openly speaking, I’m just moving to my men’s cage, if you know what I mean? And, cry on my shoulders to preserve my energy. But openly speaking, you know, uh, I’m living in a cottage house, so, what I love doing is really to go and walk, you know, around my garden. Plus, I’m a family person. So, you know what it means, you know, having the family and taking care of the kids, and so on and so forth. Nevertheless, as I was saying, I desperately need all the time and each and every day to preserve some of my just private time in my cave for myself with no internet, nothing, just, just, you know, taking a rest and taking a deep breath, right. Yeah.
Kovid Batra: Yeah. Thanks for sharing all this. It was really, really nice knowing you more.
Great! I think now it’s time that we move on to the main section where we talk about how to maintain engineering efficiency with a data-driven and empathetic mindset. So, this is, I think very close to home to you also because you have been a leadership coach and you have been posting a lot about such topics. So today, I have a very, I would say interesting, as well as deep questioning here from you to actually see how you lead engineering teams while you work with them. How do you teach them that data-driven mindset and empathetic mindset to actually improve the efficiency? Yeah, right. I think we’re all set.
Marian Kamenistak: Yeah! We are good. And we’re looking for the topic cause I’m a data-driven person, to be open. So, looking forward for the topics we have ahead of us.
Kovid Batra: Sure, yeah. So, I think I’ll start with your experiences first. You have 15+ years of coaching, being VP, I mean, diverse experience as an engineering leader. You would have seen a lot of challenges, and you would have overcome them. So, let’s talk about something from your experience at Mews probably. Anything that you would love to share with us? Any specific challenge? How did you overcome it?
Marian Kamenistak: Oh, okay. So, putting a little bit more perspective into my last full-time gig, it was the story of Mews where I was working as a VP of Product Engineering. And, now I can say the story was pretty funny and interesting. Nevertheless, when I started at Mews, it was not that funny anymore because it was the COVID time with the C, not with the K, of course. So, there was plenty of the layoffs and you know, the let’s say the atmosphere was not that great, I have to say. By the time I was running around 8 to 10 teams, different product engineering teams, and after almost 3 years, I ended up with more than 30 teams in total, right? So, the experience was that after a couple of years from, you know, running basically, resurrecting, that’s, I would say, the right word, the teams from sort of a poor efficiency into something meaningful, into something that the teams are really considered as an ammunition that can overcome and run over the competitors basically. And, that was what we were after. And, in the end we ended up being successful in Series C investment, which was around 200K, right? Pardon me. 200 million, right? A bit different number here. So, that was really the story of Mews. Of course it was not only my job. It was the job of, you know, other stakeholders, participants, other peers. I wouldn’t be able to basically resurrect and grow the team just on my own. That being said, the important portion of that goes to, for example, product management, and to other departments in the company.
Plus, we had to change the culture and prepare the teams for the goals, right? So, that was one of the things. So, you know, roughly speaking, when it comes to the challenges, it was really, you know, making sure that all the teams are aligned, cause, there was so many different initiatives that we were running. Plus, you know, the focus was not there. I would dare saying that the teams were not really efficient in the end, right? Although we were busy, you know how it goes.
Kovid Batra: Right.
Marian Kamenistak: The productivity was not really there. And I was told that, you know, when I entered the company, I was told by the Chief Product Officer that “engineering is a black box.” Basically, that was the message. And, so turning that into something that works, into something that’s predictable, that’s transparent, and, I would say the keyword here is ‘trust’ to turn the teams to something that is trusted.
That was really, you know, one of the stories that we were after and successfully, we made it work this way. Yeah.
Kovid Batra: Okay. Did you do anything about solving the “black box” particularly? Like, when it was told to you that it’s going to be a black box. And, then of course, when you are leading a team, you just don’t want it to be a black box. You want to know a little in-and-out. Of course, things run on trust. That’s totally there. And I totally believe that’s the core of every good team or a successful team. But still, were you comfortable with that thought or did you do anything about it?
Marian Kamenistak: Well, of course there were some sort of initiatives that, you know, I made live. Openly speaking, I started my first 3 months doing some sort of an internal audit, meaning like, you know, making sure that I understand, you know, the foundation, I understand the product, I understand, you know, when it comes to the teams, to their productivity and, you know, all the different aspects of that, including leadership, of course. And, the second thing is, you know, after the audit, it was pretty clear to me what are the main, you know, most valuable items or initiatives that I should be focused on.
To be more specific, I started observing the teams, the way that I attended their different ceremonies. I was working closely with the product engineering site. One of the issues was, you know, again, openly speaking, the cooperation between engineering and the product as such. And the second issue we were facing was transparency, meaning we wanted to make sure that, engineering, product are both transparent enough in terms of knowing where we are when it comes to sort to, to ongoing initiatives, where we are when it comes to the backlog, and what awaits us. So basically, the purpose of that was to lower down the communication gap. And, once again, the transparency was, is one of my principles that I’m sticking to quite well. That being said, I moved from, you know, spreading the gossips and the hypothesis about the gut feelings to the system that really worked through data. And seeing how things work for us, by, inspecting certain things in reports, basically. I know it doesn’t sound very attractive. But, leading more than a dozen of different engineering teams, you don’t have a much of a choice, because that would go to micromanaging the teams, which is not the right path to go, especially in case you want to scale the teams, right? And give them the ownership. Yeah.
So basically, coming back to your question, I would say ‘transparency’ is the keyword we are looking for. Yeah.
Kovid Batra: Hmm. That totally makes sense.
Great! Thanks for sharing this. Moving on to my next question, which is kind of related to this, when you talk about transparency and not micromanaging right? I have seen different teams, different type of leaders, like ones who are purely data-driven, right? Then, there are teams where leaders are purely, like intuition-based. There are 40 developers working, but they are working on purely the intuition. What’s the productivity? What’s the output? How things are moving? So, people are very intuition-driven. And, I’m not saying that it’s wrong or right, but every situation demands a different balance of both. That’s what I believe. What’s your take on that? Like, you must have seen teams in the similar manner as I have seen. Some are data-driven, some are purely intuition-driven, some have a balance of both, right? So, what’s your take on that? I just wanted to hear it from you.
Marian Kamenistak: Yeah. Well, of course, it all depends. It depends on, you know, what’s the current stage of the company, whether that’s an early startup, the scale-up, or I would say, up to a corporate, you know. The second thing is whether, you know, the company as such is working on projects or products, if you know what I mean, right? That’s a, that’s a huge difference. The other differentiator might be whether the company is a stage of, you know, sales-led vs product-led growth…
Kovid Batra: Yeah.
Marian Kamenistak: And, you know, strategy; that’s by the way, really important because when you are sales-led, then you need some sort of a lead who’s, you know, giving you the direction and who’s sort of almost micromanaging things, which might be the right thing to do at the given time. Nevertheless, if you want to scale, then you should be slightly starting to move towards the product-led organization, right? What I was going to say, there are different aspects that, that might influence our decision process, how much I’m going to lean towards, you know, the gut feeling and empathy vs data. Nevertheless, my message would be, if you are running more than 8 different teams, or I would say 6-8 different teams, then it might be the right time to verify your hypothesis and help the teams with the productivity through the data-driven approach, right? And that was really my path. And, again, I stick firmly with my principles, transparency, and providing this sort of, you know, transparent reports with valid indicators, of course, is, in my case, the right way to go. And it always, sorry to use this word, it, it always saved my ass, right? So, that’s, that’s how I see it, right?
On the other hand, when we have, you know, gut-feeling teams, if we may put it this way, what I see often, there is a big proportion of inertia and a big proportion of complaints and sort of inability to make the right decisions. To be very specific, what I mean is that, for example, if an agile coach tells me, you know, “We have an issue with a specific, let’s say, with velocity, with estimate or with, uh, the quality of the product or with the deployment frequency, or with the confidence when it comes to the sprints completion.”, then, it’s not only to have this gut feeling, but also to look into the data, because since the moment I’m showing some sort of trends based on the figures, then people start to listen to my arguments, as opposed to say, “Okay, that’s yet another hypothesis, and we heard already hundreds of similar hypothesis. So, thank you for sharing your opinion and let’s move on to the next topic.” Right? So, that’s not something that is really helpful. And, the way, for example, how I’m treating data is that, for example, I’m very specific when it comes to fixing or improving the teams. What I mean to say, for example, I want to, I’m really tying improvements to specific measures and here I may say, for example, when it comes to the sprint completion to me, the healthy consideration of the team is I want to see that the teams deliver, minimum 80% of the sprints, right? When it comes to the roadmap contribution, I expect that the teams deliver or spend minimum 60%, for example, generally speaking on the product roadmap, right? So there are different, let’s say thresholds or metrics that I’m using just for the sake of making sure that the team is considered highly-performing and healthy.
And here I want to make some, some sort of a side note, which is often I see that, you know, each and every manager or, you know, owner of the company wants to have highly-performing teams. You can have these sort of teams, but when you are throwing junk into the teams, then you get the junk out of the team, right? So, it’s not only about having the highly-performing teams, it’s also really important to know what I’m asking these teams to accomplish, what’s under their plate, what are the main initiatives and whether these initiatives are the most valuable things that these teams can do as opposed to doing something that goes to the shelf and not being touched by the customer, right? So, that’s something that I see being neglected sometimes, yeah, a bit often than I would wish.
Kovid Batra: I think I personally have felt this that this seems very right with scaling. You have to have this kind of a mindset. You have to bring in that transparency. You have to be little data-driven, in fact, not little, little more data-driven than you think you should be. But, even though it seems right, and most of the engineering leaders I feel are very intelligent, they are well-aware of the situation and they know that this is probably the right thing to do. But why do you think there is a restriction or there is an apprehension towards not adopting such frameworks? I might be wrong here. This might be an assumption, but I talked to a lot of them, but there is some level of apprehension I feel in the engineering leaders usually that they don’t move towards this. What do you think could be the reason here?
Marian Kamenistak: Well, you know, I had a luxury to implement or to help with, with the advisory, implementing, you know, this sort of data-driven approach checks when it comes to the teams in a couple of different companies. Most of the time, this sort of, you know, pushback or rejections comes from the fact that you know, we don’t want to treat our people as numbers, if you know what I mean? So usually that’s the number one thing that I’m hearing. And the second thing is that, you know, data is just part of the song, you know, part of the story, which is absolutely right. And, here the important distinction is, we were talking previously, you know about extremes, which is, you know, either you have the gut feeling based teams or the data-driven teams, of course, the combination of both is the, you know, that’s the biggest “wow factor” that you can obtain. Speaking very openly, you know, I treat these numbers not as goals, but rather as aspirations or indicators. That’s the right word in my opinion, because you know, to be very specific, you might be having a person with the high amount of pull requests or opposites to that. And you might be saying like, you know, “This is the right developer.” Opposite to that, you might be a person, you might have a person with the very low to zero number of pull requests. But, it doesn’t mean that that person is low-performing because instead of you know, individually contributing to the code, he, he might be doing mentoring and code-pairing and doing some more programming, which has a much higher aspect…
Kovid Batra: Absolutely.
Marian Kamenistak: …you know, impact compared to the metrics that we are using. So, there has to be always some reasoning why and some connotation. So, yeah, the thing is that, you know, we shouldn’t be using these numbers, some definitive metrics whether the team is highly-performing or not. There is always some sort of context into that that has to be added there. Yeah.
Kovid Batra: Right, right. Perfect! I think well-answered there. I get your point.
Cool. Let’s move on to something more interesting here.
Marian Kamenistak: Okay.
Kovid Batra: There are different, different metrics, which each team has to look at different scenarios, right? Like when, when you talk about, like cycle time, how many lines of code somebody has written or how many commits somebody has done. There are different metrics that you can look at. I mean, there are 20s of them.
Marian Kamenistak: Yeah.
Kovid Batra: Prominent framework is around DORA and SPACE, but let’s say if we are choosing to work on something in a team, there are a few metrics that one should look at, right? So, how do you choose those metrics? How do you choose which ones to really look at?
Marian Kamenistak: Okay. Okay. Yeah. Exactly as you are saying, Kovid, the basket from which we can choose is pretty broad.
Kovid Batra: Yeah.
Marian Kamenistak: Here, we speak about dozens to almost a 100 of different indicators that we might find useful. You know, before I start diving into specifics, let’s uncover some of the principles that I’m trying to stick to before, before advising specific indicators to which to lean to, right? So, to me, the number one thing is that, you know, as I was saying, these numbers are just indicators. So, it shouldn’t be tied to some sort of any performance reviews or efficiency of the teams and these sort of things, because it’s dangerous. The people will game it anyway, right? If you put it like this, right? So, it shouldn’t be tied to any bonuses or compensation metrics or whatever, that’s a way to help basically.
The second thing that, and I’m pretty radical here, is that I don’t want to see the data being used for the purpose of measuring the efficiency of an individual as a developer, right? So, you know to me, the lowest granularity that I’m looking at is the team itself, right? You know, since the moment you are starting to look into individuals, it’s a dangerous game. Of course, we might be having some underperformer in the team. That usually, that’s what I’m hearing as an excuse why to have it. But in that case, I would rather see the team to deal with that situation on their own, as opposed to teaching them to wait until a manager comes and makes the resolution. So, to me the notion of, you know, self-healing teams is something that is really important. Of course, I might be having a look into specific data about some individual, but I would still be very strict in making sure that it is the team itself that makes that ultimate decision, without using the data, right? Because to be honest, people already in the team know who’s sort of performing, not performing very well and so on and so forth, right? So, that’s the second thing.
So, again, looking at the efficiency or productivity metrics from the perspective of the whole department, the divisions, tribes, chapters, however we call it, up to specific teams, that’s absolutely fine. But, I would be strictly against using it for individuals, on the individual-level, right? The one thing that I learned is that we shouldn’t be diving too deep into, you know, how the sprints work, what’s on the plate, what’s the velocity of the teams, and so on and so forth, because usually it’s, you know, either it’s counter- intuitive, but it’s not as important as, for example, how big of a focus the teams can have and handle, or how much of importance the initiatives that we are throwing to the teams are, right? So, basically it’s more about making sure that the discovery process works right, first and foremost, and only then we can have a look into, you know, the efficiency of the teams. So usually the story is like, you know, a company’s inviting me to do some internal audit, and saying openly, “My teams are not working very well. I don’t have trust in my teams. Please have a look.” Usually, you know, doing the deep dive in the teams, the teams themselves, usually they work pretty well. And of course, we can fine-tune some of the things and increase their efficiency by, I would say 10-15%, right? Might be to 30%, which is quite ambitious already. But, the gem lies somewhere else. And it’s about the discovery, what we are throwing into the teams, what’s on their plate.
And the second thing which I learned is to create synergies. Being more specific, it’s more about synergies between the, for example, an Engineering Manager of a team and their Product Manager. If you create a synergy and the fit and the chemistry between the two, then all of a sudden, your productivity might get increased by 300%. So, that’s yet another message, not looking too much into the data and creating the synergy between and the chemistry. That’s what’s really, like boosting the efficiency like hell. Yeah.
So, being more specific, usually what I would advise is first and foremost, making sure that we are working on the right things, meaning, seeing what’s the, for example, the adoption rate. That’s how I call it. That being said, I want to see that most of the releases and the features that we offer are well-accepted by the final customer, right? So, what’s their satisfaction? And I want to see that, for example, we are working that I would say about 65% adoption rate. That being said, of course, we can’t ask for a 100% because that means we don’t allow any mistakes and no psychological safety. And basically we end up copying the work of the competitors, which is a nightmare to do, right?
Kovid Batra: Right.
Marian Kamenistak: So, that is one thing. The second thing is, I want to make sure that my teams are working or have enough capacity to work on the things that matter the most, meaning the roadmap. So, I call it a ‘roadmap contribution’. If we have, for example, product engineering teams working on the product, then I expect to have this number around 60-80%, right? That being said, most of the time I want to see my teams working on the product roadmap, right? We might have some technical roadmap, or usually people are forgetting about the off-roadmap items, which is business as usual. And usually you would be surprised if you start measuring the roadmap contribution, you might find out that some of your teams are spending 55% off-roadmap, right? Which is sort of alarming, right? So, you know where your focus should be. And that’s already a pretty good indicator to start with, right?
Kovid Batra: Right.
Marian Kamenistak: So, the ‘roadmap contribution’ is number one thing. The second thing is, usually what I see is that if you’re on sprints, then we want to fix, for example, the trust towards the teams. That being said, you know, if you’re on sprints, as I was saying, I want to see that we deliver at least 80%, of the sprint, as I was saying before. That’d be, you know, with the number of the tickets in the sprint itself, right? So that’s basically, you know, the amount of the planned tickets vs the amount of the tickets being delivered in the very same sprint with no, move over to the next sprint and again and again, the carryovers, right? So, that’s yet another thing. And, basically this way, the teams are getting trust, and build the trust on their own. And, moving this number from, let’s say 50% to 80% is already a big achievement, right? Again, we don’t want to have a 100% because it’s estimates, it’s not like some sort of commitments, right?
The other thing that I might be saying is sort of exceptional that I’m measuring is, I call it ‘Epic Cycle Time’. So, what I mean here is that usually, you know, Jira and all the tools, they are great at measuring the cycle time on the level of the tasks and these sort of minor things, which are not really important. I want to see that, the epics, the, the milestones and the increments as they are being deployed to the product are deployed on a regular basis, and the deployment frequency is roughly around a month. In my definition or in my view, in my book, the definition of the epic is that the epic should deliver a tangible value to the customer, right? So, it shouldn’t be some sort of some simple user story. It should be something really tangible as an increment. And, I really want to see the teams sticking to deliver on average one epic per month, right? So, we are talking basically about the art of vertical slicing here, to be honest, right? And the reason why it’s important is that usually you don’t want to see the teams that go dark for 3-4 months, then they move to, you know, they go back from, from the dark ages, after a couple of months and they want, they want to, you know, get celebrated because they achieved something very big, but the product manager says it was not what they meant. The stakeholder is sort of angry at seeing, what they produced. Then, there’s a bunch of, uh, endless bunch of bugs for another 3 weeks. We all experienced, you know, this situation and that’s something we want to avoid. We don’t speak about anything unique here. It’s nothing more than the feedback loop and the frequency or, you know, the… the rapidity of the feedback loop. So, we are in the principle of agility. Agility is nothing more than, you know, the speed of the feedback loop, basically. You can forget these hundreds of thousands of books about agility. It’s nothing more than that, right? And this allows the teams to steer and to adjust, you know, stick back to the roadmap basically, right?
So, you know, and of course, we can speak about the DORA metrics, the deployment frequency, change failure rate, and so on and so forth, the usual or the SPACE metrics, that’s already fine. I know I’m speaking for too long. Nevertheless, there is one tiny metric or indicator that I want to add here.
Kovid Batra: Sure.
Marian Kamenistak: And, I was really surprised how greatly it helps. And that’s the, you know, satisfaction, or employee NPS score. Basically, where you are looking at the productivity and, you know, there’s quite a lot of tools that offer you this sort of functionality, where you can measure, you know, what’s the satisfaction of the teams, what’s their cooperation, what’s the mood, what’s the wellness score, what’s the relationship score, relationship when it comes to their peers, to their team or to their manager. And the reason why I’m talking about it is that I was literally surprised how much it has helped me to save some of the teams, as opposed to see the teams rotting and not handling certain situations well.
If I may put a specific story here, is that I might see that in one specific team or specific department, the communication goes down quite rapidly or the relationship score goes down quite rapidly. So, that gives me a hint that, you know, I’m coming to an Engineering Manager and ask him, you know, “If you run the next 1-on-1s, then please ask your teams about this specific thing, and come back to me with, what’s going on.” And most of the time people say how things are, if the culture is good enough and with no intrigues and they are open to share their views. So, you already have certain hints what’s happening in the team and what your action might be, as opposed to not knowing anything about it, people being silent, your team being disassembled while not being aware of that, and you end up by in a situation that you cannot basically save the team, if you know what I mean?
Kovid Batra: Yeah.
Marian Kamenistak: And you need to rebuild, which ends up, you know, running yet another investment for half a year for usually rebuilding that team, right? A couple of months. And you can, when we speak about the budget, we all know that the biggest cost in software companies are developers, right? So, speaking of the cost, that’s a that’s a huge impact here. So basically what I was going to say or point to is that again, that’s the principle of ‘prevention’ here that helps me quite significantly to make sure that we are still on the same spot and, keeping the tendency high when it comes to the performance of the teams.
Yeah, so hopefully that was a pretty deep list of what we can talk about.
Kovid Batra: No, I think that that is much needed. I mean, I can’t say even for a single point that you mentioned that I, I differ in the opinion. In fact, I’m more now concerned and I want to go back to the same question that it seems so important, it looks very important to follow these things, be it your investment distribution, be it tracking your Epic Cycle Time, which was a very interesting piece by the way. Like, people track the PR cycle time or small things. Tracking the time to delivery of the value is something that one should track. Of course, you can break it and then track individual pieces to optimize different pieces. But yeah, it is very important to track that. So, each and every point seems so interesting and so important for an Engineering Leader, or an Engineering Manager, like imbibe in the team from the very beginning itself. I still find that adoption of such things is very hard right now. Like, people struggle to adopt these things. Any, any piece of advice on that? Because right now, when you are talking to teams, you would have also felt that. What exactly do you think you do that people start looking at it as an important thing to do, as a responsibility, or as one important thing one in engineering leadership should be doing?
Marian Kamenistak: Yeah, that’s a really good topic to uncover cause if we don’t run the adoption of these, you know, engineering efficiency metrics, right? Then, then, you know, we are not in a, in a pretty situation. Usually what I’m saying, what I’m advising, you know, the companies, how to adopt these sort of things is that, you know, the numbers is just a third of the story.
Kovid Batra: Yeah.
Marian Kamenistak: Two-thirds are how we treat these numbers and to create the ecosystem around these numbers, right? If you just, you know, throw the reports on the teams and the numbers, it doesn’t provide much of a value, right? So speaking of the adoption here, Kovid, is you know, again, we can start from the way how, you know, to motivate people and, explaining them the reasoning behind it, right? So, what’s our motivation? Why we want to introduce this sort of efficiency indicators and so on and so forth. And basically to articulate and over communicate the message, what’s the reason why? That being said, you know, I’m always, I’m constantly talking about the principle, we don’t want to, you know look at you as a number. That is one number one thing, and the, and the number one worry we have.
Kovid Batra: Yeah.
Marian Kamenistak: And the principles are we want to be transparent and we want to be preventive, right? So, these are the two utmost principles that I’m looking for, and I’m telling them and explaining that to the teams and to people via using stories. What happened to me when I had no luxury of looking into specific data, right?
Kovid Batra: Right.
Marian Kamenistak: So, that is number one thing. The second thing is that, of course, these things shouldn’t be just run top-down. It should be run, from the very beginning with some sort of a tighter team around and making sure that you choose the right people and you invite them to run these conversations bottom-up, right? So, you can start with specific Engineering Managers, Engineering Directors, Tribe Leaders, Chapter Leaders, whatever you have. And there are people who are actually interested in being better leaders and they are usually the people who raise their hands up and sign up for that, right? So, having these people on board from early-on stage, that’s really vital because if you just throw it to them top-down, it doesn’t help very well, right? So, these are the two, I would say most important things and of course, there has to be some, I would say, activation and adoption period. The activation is, you know, how you start running it. And the second thing, the adoption itself is, it’s not just about, you know, “Here are the reports, and uh, please deal with it.” It’s mostly about, “Let’s run some sort of a continuous workshop about what these numbers mean, what are the situations where they can help you actually identify certain, you know, inefficiency, potential factors, right?
And, also giving them, the ability to provide some sort of a feedback. And say, for example, “This specific indicator doesn’t make sense.” or, “We don’t measure it well.” Because, for example, being very specific, the deployment frequency might be measured in, you know, different ways, or the cycle time might be defined in a different way depending on what stages in the cycle we are considering or not.
So, these are all the different things that we might be able to consider, you know. A good example of that is, again, talking about the cycle time, I wouldn’t punish the teams in the B2B sector, for example, of course, because their cycle time might be too long and up to the deployment. So, there is always a question whether, you know, to include the deployment phase into the cycle time or not, because if you are deploying just to, you know, in the B2B world, once per half a year, then obviously, you know, this sort of metric doesn’t make sense for you, right? So that’s, that’s one, just one of the examples, right?
And, talking about the adoption itself, I would say the most important thing that I found out also was to make sure that you provide some guidelines in a written way into your knowledge base about what specific numbersmean, why we are measuring those, including some How-to guidelines, meaning like, “What are the thresholds?” And if you are in red numbers, what might be the right things to start in to look at, you know. Your focus might be totally broken or you are running too many initiatives in parallel at the same time, or you are having too many ad-hocs, you know, disturbing your sprint. So, let’s start measuring these, or you know, these sort of How-to manuals. They’re really great because from that moment on people can point their finger on to what’s the definition of that measure, without again, trying to be smart and, every person has a different, you know, explanation of how these things are measured.
So again, having specific formulas, the motivation, what’s next or what’s right for us and some sort of How-to manuals, that’s really helpful, right? And the last thing to say, part of these manuals or these guidelines, there should be some sort of the, as I was saying, the ecosystem around. That being said, for example, I want to make sure that an Engineering Manager as a first-level manager-person is responsible for these specific metrics and should be looking on these metrics, let’s say on bi-weekly basis. The Director or the VP-person should be looking on another metrics on department-level, I would say once per month or something similar to that. And, making these sort of measures part of your daily routine, that’s really important. And, ensuring that we are talking about these numbers on our dailies, on our 1-on-1s, just checking whether the health check is fine or not. And eventually trying to, basically distill some situation and you know, moving the teams back on track when some of the indicators might be not showing the best performance ever in the team. Right.
So, these are just a couple of things that, I’m really paying attention to when it comes to the adoption itself and, as opposed just to throwing the numbers on the teams and, “Please deal with that”, it never works. It never works. Yeah.
Kovid Batra: Yeah.
Marian Kamenistak: So I hope that makes sense.
Kovid Batra: Perfect! I think I couldn’t have asked for more. One of the most interesting sessions for me personally. To learn from you and see how things being done. Ggreat, man! I think I would love to have you one more time on the show talking about specific use cases where we can spread more knowledge around these topics to our audience. I think that would be really interesting.
Marian Kamenistak: Yeah, yeah. Thank you. Thank you, Kovid. And, it was a pleasure to have you. So, enjoy. Bye!
Implement DORA metrics, gain dev visibility & drive success in your teams.
Try Live Demo
