**Person A**: The first question is about your background, and it directly starts with some basics like how long have you been working on platforms?

**Person B**: And in our current position or in general? Yeah.

**Person A**: In general. In platform engineering, not necessarily just the one platform you're currently working on.

**Person C**: I mean from before the term platform engineering was a thing. I would probably say I started in that direction around 2009 or 2010. Yeah.

**Person B**: And I think for me that would be later. I probably have been working for 5 years. Yeah,...

**Person A**: And how long has the platform that you currently work on existed so far?

**Person C**: What can we say here? Two years roughly depending on what we took from—yeah. Yeah,...

**Person B**: I think maybe we should go from the reorg because before that none of us were here but we were in infrastructure, right? And then there was reorg where the platforms got created. That was maybe half a year before you joined, Person C, or something. Yep.

**Person C**: That's probably right. Yeah, I—yeah.

**Person A**: Which year was that?

**Person C**: So that's around, I guess, beginning of 2022. So what's that? 

**Person A**: Three, three and a half... two, two and a half. All right. And you already mentioned the reorg. So, at this point, are you part of a designated platform team? What's the size of that platform team? And what other roles do you have on your team besides the typical platform engineers I guess. Mhm.

**Person B**: So for my team, this is of course also where it gets tricky. My team is building on top of Person C's team. So we do stateful services, they do containers. And I would say, Person C, for your team, you are basically all platform engineers, right? You could say that then there's management on top. But while in my team we have at least two engineers I would call database administrators fully. And then the remainder—we are, I guess, 11 in my team, I guess. Yeah. Two of which are database administrators, two of which are management and the rest is—you could say yeah that's a consultant. Yeah.

**Person A**: Mhm. You said two are management, right?

**Person B**: And currently I count myself as management but I'm also an engineer. So depends on how you want to cut it.

**Person A**: Do you have a specific name for that management role or is it just management?

**Person C**: We have it right now. That's the only thing that I think that diverges. We have—the constellation at our company is that we have a lead engineer and a competence lead. So lead engineer, technical stuff; competence lead, people stuff. And...
The funny part here is that it's been designed so that in infrastructure at least in Hosting department, which we are a part of, Person B and I are actually what you would call lead engineers and product managers. So we are both in what—we don't have a product manager; we are the product managers.

**Person A**: Very good. Very interesting. Good. Do you have any special name of you two teams in the organization like what's your team called?

**Person B**: My team is called the Data service platform team. We should have been called something like Stateful Services team or something like that. But we are where we are.

**Person C**: And we're called the Kubernetes platform team.

**Person A**: So you definitely have also the platform in your name specifically.

**Person B**: Yes.

**Person A**: Very good. A bit of a more tricky one, more open one. How is the platform actually defined in your organization?

**Person B**: That's not tricky. So, we are organized into digital products. So, we have an input domain model where in our cases our teams exist as products and then a product offers business capabilities. So in my case, my product is the Data service platform and the business capability is Managed RabbitMQ for a specific department, Managed MS SQL for a specific department, and Managed Valkey for a specific department. So those are the business capabilities, and it's phrased like that because these are capabilities then that the rest of the organization can then pull on. So if they need to do MS SQL stuff on-prem, they can use that capability that is offered up. Yeah, I think for you, Person C, I guess that is your capability, right?

**Person C**: So that's Container Hosting as a Service, I would say, and then due to reorgs probably we have other capabilities in our team as well with artifacts and such forth but I don't think it's worth mentioning here. It's part of the evolution that it will divert to another team later. We found ourselves in a position to do stuff like observability on ourselves to be able to enable ourselves, which then spans out into new initiatives and new teams. We have had a couple of those. Yes.

**Person A**: All right. So the purpose of the platform is the provisioning of those services and capabilities. Right?

**Person C**: So the containers—that is what we are supposed to deliver, and that's what our main practice is.

**Person A**: Who are the customers of your platform?

**Person B**: Yeah, it's internal developers—that's the easy answer at least, and then you could go into stakeholders. I mean there are delivery directors for a specific department that are interested in the existence of the platforms, right? Because it reduces the operational and cognitive load of the teams in their organization because now they don't have to maintain a database. So they are—depending on the layer you talk about customers, right—they would not be the customers of the specific database but they are definitely customers of the existence of the platform.

**Person A**: Good. Let's dive a bit deeper into your platform and how you actually work with that platform. Could you describe the last feature or capability that you released to your users?

**Person B**: So I could look at it two ways for my team. One is that we've built a Managed ValKey service—so the Redis fork. The other way I could view it is that we've also launched what's called—so we have a developer portal that's so many other—and we've just enabled end users to be able to view all the assets in that development portal. So, I mean one is the actual capability; the other one is quality of service of those.

**Person C**: So we managed the EKS clusters for them. We developed the platform to be as agnostic as it could be, and now it's running on AKS, might be next year. And the parallel here would also be that I think that the most recent release that's more feature-closed conceivable—that is also developer portal capabilities, just I think we did the same day officially anyways. Yeah. Yes.

**Person A**: Coming back to those capabilities, let's look at the portal one for now. Why did you exactly choose to deliver exactly that capability?

**Person B**: Let's see. I'm trying to be careful not to just come up with my own answer for it and the actual reasoning, right? Yeah.

**Person C**: I can start if—so our service on EKS came from a demand that we are actually replacing a grassroots product—containers as a service—that didn't manage to deliver. The team didn't scale, and it wasn't systematically organized. So that's why we created a new platform. New people were hired and a big team was set for it. And our service on EKS is essentially the last part of replacing that old service. So we need to make sure that the old users are transferred to the new platform. That's what we're doing now. And that job will finish, I guess, in—and then at the same time we're also getting in new customers all the time. Yeah, that's the main reason for that specific thing. When in regards to our developer portal that is self-service to reduce our KTLO and to enhance the developer experience essentially. Yeah, business-wise I don't think we have a good case for proving the value, which we prefer to do, but we all agree that it's there. That is why we have done it.

**Person B**: And to add to that, there's the platform grouping we are in—it's called again "internal"—so we are a group of four platforms that work together, and we together have put on these things on the internal developer portal. We have the strategy of providing cloud-like services because our main focus has actually been on-prem. So it has been enabling these services to run at the manufacturing sites. We want the colleagues that use our services to have a cloud-like experience. It should be as easy for them to use the services as if they were using public cloud. Part of that is to have a good development portal and APIs and stuff. But this is the next thing we're then looking at.

**Person A**: Very good. Can you walk me through the process of working on this capability, especially from the whole decision to work on it to the delivery?

**Person B**: With this cloud-like thing, we need this. Okay, let's go and build it. So, as Person C said, it's not as data-driven as could have been. We need that thing. It's easier for me to talk about the managed Key-Value service because that actually came through our ordinary process.

**Person A**: I mean it's interesting both to see the different approaches, but let's focus on the Key-Value then.

**Person B**: So, we have a process where each year a number of investments—there's a process where end users that need capabilities can pitch the business cases for the fact that they need that. That gets combined with larger strategic initiatives that we have with all that, so it's down again, and then the various product teams are then presented with this year's goals in form of objectives and potential key results we can deliver towards. For my team, there was a request for delivering a managed in-memory cache, and it was deliberately phrased like that. So there was not a specific request for Valley or Redis as it was back—it was the business capability. And my team then looked at what was currently at the manufacturing sites, which determined that something like Redis was in great use, but teams were running. So this aligned with that need that had been communicated—that someone managed an in-memory cache.

**Person B**: So we were in conversation and identified some stakeholders and were in conversation with those teams and then decided, okay, we're going to manage Valkey, and then it was identifying those stakeholders and then working together with Kubernetes platform team and other teams that could help us with how to actually build all the automation to run all of that, and then ended up delivering it. While we did that, we had close collaboration with some of the teams that were going to be using this to understand exactly what they needed or what they didn't need for this. Yeah.

**Person A**: Let's explore that a bit further. You already said you have OKRs especially in the beginning. How do you know that a new capability is successful? Do you have any specific metrics that you measure there?

**Person B**: Yeah. We are still onboarding the users as it is right now. The main metric we have right now is the fact that they are running it themselves, and they know that they are spending time doing that. So while we haven't been good at actually measuring it, we just know that it's at least work that we take on—if you look at the number of hours spent in the entire company in running this, we don't currently have any measures to make sure that we are not spending four times the amount of time operating this as compared to what they did. So we're not tracking that right now. But we're just tracking the fact that the number of existing self-managed instances is going down. That's the metric that we are tracking, and then we're assuming that that is a proxy for greater efficiency because we assume that we are better at running that than the end users, and we could do it more efficiently. It's literally right now I'm taking—because we just did the similar thing for another service, and for that we counted the number of clusters that existed, and then our key result was literally just decreasing that number. And we're doing the same—for the reason I'm using the other one as the example is because we did this last—I guess we are still in 2024—at the end of this year because we were late, it was literally just one. I wanted to go down to zero. Next year we'll then find a new number and then start driving that number down.

**Person D**: What was the proxy metric again? The fact that you're running it instead of them?

**Person B**: Yeah, but we're just using the assumption that it's better for everyone that we do it than the teams that are doing something else while they are also operating the thing.

**Person D**: Quick question. You said that—do the users come up with the business case earlier or is that combined with you?

**Person B**: Ideally, as it has actually been this year, it's combined with us. We have a lot of conversations across, and we try to identify cases. I know from back then there was a wish to create these platform teams, and in order to fund that there was a discussion with the teams that are currently self-running these things, and then it was basically what services would you like us to do in return for this investment we're getting to build these platform teams. And again, to be as flexible as possible, they were stated as capabilities. So again, it was literally in-memory cache—that's the thing we need. And then of course subtext would be really nice if it was Redis compatible, but actually the business capability we were asked to do was that. Yes, we had similar things this year where there are some services that we know some engineers need in order to build a thing they want to do end of next year. So we've been in conversation with them about the concrete technologies that could potentially do that, but because we are not actually testing the final implementation of that as we are building that business case, it's phrased as we need this capability. And if that gets funding, that is the request we then get—that we need you to invest in building out these platforms. And here are some examples of things you could do. And then it's up to us that we have the funding to actually go and verify that. Person C, you need to say something because I can see your face. Unless you need more clarification on all that, just you should probably spell it out.

**Person C**: It's your wellness. I'm just trying to give space here because I might fill up a lot now. So let me take you to the AKS or EKS example—the AWS example. So I have had a mentor from my previous many years at another place who learned me a trick called CD3. And I cannot help mentioning this...Because the big problem I had when I was hired here was that I did not understand the value created by the choices made beforehand. I did not see the red line. So what we started to do was to do CD3 analysis on the opportunities we had. And we've done so since I would say September last year. Cost of delay of course divided by duration—that is CD3. I can't remember his last name but CD§ is the name if you want to read about it on the internet. And essentially it's about what's the cost of delay compared to the time you implement it. It's pretty simple when it gets down to the basics. But point being that I need a way to prioritize what of the many features we had to do was most important for what would produce the most value for our users, and I needed a way to capture all the ideas that came in. So we started using CD3, and when it came to the AWS example, we were forced to do AWS because we have customers on the old platform. But we did the analysis and it topped out at, I think, number three. Let me just check. At number three. Maybe number five. I think we implemented one above. And so it actually turns out that the value it creates is at number four, number five here. The first one we implemented and the next three have dependencies, and they depend on, for example, other teams wanting to do this, having time to do this. So we have not implemented those. So we actually end up following the CD3 analysis. And the reason that it made sense is that when we did the analysis, we looked at adoption. That was our delivery lead's main way of measuring our success, I think, last year was looking at adoption. So we analyzed how many potential customers we have out there. There's 250, 270 teams, and we came up with a number that was 50 teams.
I recently discovered that that is actually 110 teams that's already running containers but not running on our platform which means the value case is even bigger for us delivering this when we can deliver it because we can't deliver it to all teams. So CD3 analysis has been a way of prioritizing the things we do. It doesn't fit the bill for everything but it gives you an indication of the big bets. Let's call it that. Afraid to coin something here. Yes I know what you're thinking guys. But it gives an indication of where we can create the most value and a good example here is that the number two and three if I remember correctly is something that came from the support team in a specific department. They came with an idea for us last year and that turned out to be extremely valuable when we did the CD3 analysis because we can reduce interruptions by helping them and that is only with observability and dashboards. And that actually tops that one which is super interesting. So we've been doing that on the side as a not official—we don't just follow portfolio chaos. We do this as something we decide to do in the team because it makes sense because the engineers and the support team says they need this not because portfolio OKRs were given to us. So that is a game essentially right—you figure out what do you have the bandwidth for, how can you distribute this between how many tasks you can juggle at once in a team with this many people—weighted shortest job first.

**Person D**: Just a quick question. Is that essentially WSJF, weighted shortest job first, or are you using that in the evaluation to order? Just a quick question. 

**Person C**: Is in the area of it. But it's really about—the beautiful thing about CD3 is that you talk about the money. You talk about the missed opportunity. That is the beauty of it. So you talk about assumptions, you put the people who actually know something about this, not the managers in the room, and they say, "I think we will save this much money by doing this." And if you have facts, you can get them from finance or wherever you can get them. You put them in there and you share them. You make it transparent to everyone. You share the assumption. You do a calculation. "This is what we think; it's about a week." You divide with the duration. You have the CD3 score. And then you place—which one can we do fastest, giving us the most value essentially, right?

**Person B**: It's true.

**Person C**: But it's all based on the conversation around money because everybody understands money. That makes it very concrete and it removes a lot of biases which is just—I get the chills now. I can talk a lot about that. But it's a way to try to navigate it. That said, we will stick to that method until someone shows me something that's better and more based on evidence. Because it is all assumptions and then we try to measure as we go. But the matter of the fact is that we're not good at completing something, going back, measuring did we actually deliver, and that is an ongoing debate. We're not good enough for that. For example, the adoption, we challenged the delivery lead and we said, "Okay, adoption. What is that? Is it that just a new user has joined, but does that mean the user is active? How long does the user need to be active before you count it as adoption?" And it gets tricky, right? It's pretty complicated.
So the way we look at it now is more like how much support do we need to do and how many people are continuously adding to the platform and how many incidents do we have to compare. And then we reach out all we can so it's very hard—I think we need psychologists to help us pinpoint how to measure this correctly and not just think that we can make a metric with this based on engineering.

**Person A**: All right. Yeah. Yeah. I can understand your passion for it. So looking at—that's what you said earlier with onboarding one customer, one team, and you want your first number to go from one to zero, and also, Person C, your onboarding sounds then also quite complicated if you're interested in so many measures.

**Person D**: But before we get too much further, how much time do you both have? Because we're still in basically the first section of what we wanted to cover. I just want to make sure if you have to go at the top of the hour. What's up? Okay. Okay.

**Person B**: I think Person C has something.

**Person C**: I have something at 2, but I can come back at half past 2 if need be.

**Person A**: I'll just speed up a bit and stop you if needed.

**Person A**: Okay, perfect. So, coming back at my long intro to the question, basically the idea is do you communicate releases of new capabilities and platform changes to your users in general?

**Person C**: We try to take everything we can.

**Person A**: And what channels do you use for that?

**Person C**: We try to be creative. So we have internal conferences. We do talks whenever needed. In our support channel, it's just a Teams channel. Whenever there is an option for asking people if they want a meeting, we do that for any questions. There are newsletters. We do posts in the internal Teams channels that are not about support but in general. We do roadmap sessions. We just had a local tradition event where we served some local drinks and we show we're going to do what we're going to do tomorrow and release new features as Christmas gifts, stuff like that. And then we use t-shirts and stickers and then we try—this is the good thing about our position as a front representative of infrastructure—that we talk to so many people. So whenever we can put a sticker on it, we do it and we get involved and we promote ourselves as an enablement team essentially because we know a lot about how things are put together across the organization. So enablement becomes something we can do and something we can brand ourselves with.

**Person A**: All right, that's already one of the other questions which is like do you provide support—if you understand yourself as enablement and you already said you have a couple of channels to also provide support, right?

**Person C**: I would add that the most important thing from day one with the Kubernetes platform has been the users, listening to the, recognizing the users, and having that human communication up and running. And we're not just talking about giving them what they need. We're talking about connecting. And that's the part they don't learn you at school. If you want to be successful, you have to communicate with people and not just about zeros and ones. So that is very, very much important for us to meet the engineers and help them with their problem. Not saying that's someone else's problem always.

**Person B**: Yeah. And it actually—but yeah, Person C and I can talk for hours—but I can actually see there's an interesting difference between what you're doing in Kubernetes platform that are general compute engineers and what we are doing with providing stateful services because I have to be slightly more strategic about some things. I have to be quite hard about the number of versions of SQL server we run. I have to have a strategy for we are going to be delivering this service because this is needed. Over here I provide a very specific service to some engineers because I also have to be a bit more deliberate about some things. So there are actually some interesting areas in there. With regards to the communication thing, I just wanted to highlight the thing with the roadmap sessions deliberately in the product family we are in, this group of four platforms, every quarter we have a roadmap show. Where we invite everyone and the lead engineer from each team are doing a walkthrough of what we've done for the last quarter, what the plans are for next quarter, and whether we have specific callouts for colleagues. And then besides that, we are doing a bunch of other things but is the thing we know that we do on a regular cadence that everyone in our wide area are doing. Whenever you do platforms you have this show you do every quarter.

**Person C**: The roadmap show—it's under reconsideration, right? Because the number of people that meet up to the roadmap sessions are changing so we're not really sure the actual value of it. For us it's very important but there's not many that shows up actually which is interesting.

**Person A**: All good. Let's get back to that in a minute. We still have two questions for your platform in general. You already talked about the portal. Is there any other form in which a user can access the capabilities of your platforms?

**Person B**: I guess—I mean, do you have, Person C, all the IGO stuff?

**Person C**: Depending on what you can—because you can't really do much with—that's more like you can restart a pod essentially if you wanted to. It's IaC all the way but we have the API. They can use the API directly that has—is—what we have enabled, right?

**Person B**: We have an API but at this point we probably wouldn't approve access to it because they wouldn't get much use out of it but it's a goal for—that's future-facing stuff. Yes. And then—okay, so no, actually we've just closed our ServiceNow access but in reality what the portal does right now when you request new services is that it posts on Teams on your behalf. We do also document that if you want to you can also just reach us directly on Teams. Then there's a structure to the message you should post to request services from us. For general support also, that's also via Teams.

**Person C**: I would say our API is actually implemented based on an opportunity as well. That proved to be valuable for our users. So they can use that directly for their pipeline work especially—that is mostly where it's intended to be used. I don't know if the fact that we set up access to observabilities will count for this. It's an indirect capability because it's not ours, so to speak. Yeah.

**Person B**: And they use your platform using GitOps also. So when they actually go and deploy something today, they commit to their own repository and then your automation picks it up. So depending on definitions, that is also a way to use the platform.

**Person A**: We already had a bit about it but what other capabilities is your platform providing besides the runtime and the storage in that sense.

**Person D**: It sounded like you had observability. You have key value, observability, SQL, messaging.

**Person B**: Then we have RabbitMQ messaging.

**Person D**: Yeah, messaging. Yeah. And Person C, do you provide build pipeline or—

**Person C**: That's just for the point—we are not into the pipelines. We would love to be in the pipelines but it's a mix of many technologies and it's a very, very big organization so it's baby steps in that direction creating an abstraction. Person B and I have been talking about that since day one I think. Yeah. Of course, if we take our side hustles, we have observability because we have driven Grafana into the company. All the things, I mustn't mention this. We provide them with a setup for the ELK stack, for example, also. We don't serve the stack—we have Artifactory because we need container registries, which means that due to the many initiatives at the company, we also have started to offer other offerings from Artifactory. So this is JFrog Artifactory, so we've started to offer PyPI packages as repositories and we'll be offering a lot more of those next year. Not because it's one of our capabilities but because there's a big need for it, and with that comes vulnerability scanning and reporting of that. So we are diving into that even though we don't own the capability but the team that has the capability doesn't have the bandwidth for it. And same goes with the security part. So we are taking the initiatives for that on our own behalf because there's a need essentially. Anything else? There's compliance stuff there as well with security scanning with Kubernetes that we will be supplying. Yeah. Alerting—no, PagerDuty is not our field.
Yeah, I think that's it. And then the case I talked about for—we are creating some dashboards too and this is super interesting—a guy came from support and told us, "Hey, we would love to have some dashboards about how the resources are doing in this specific department. Let's look at that." And there are many sections in that department. This is just about one part of it. So we talked with the team, said we can provide dashboards, and then we talked with the team who creates the applications for that, and they run our platform. So for me that is a very important capability and I also dare to call it kind of a service, but making sure that there are dashboards that they can understand—the support, which are the most important people, can understand it when there's incidents and the application team can understand when there's whatever is going on. But it's super important to get a common language and start bridging the gap and increasing the value. So depending on how you want to see it, look at that. And then there's the whole enablement function which if you want to call that a capability or service where we just link people together. And how—I'm going to stop it.

**Person A**: No, no, it's great. If we would have more time, I would definitely enjoy this talk to continue forever. Good. We already talked about how you set priorities in your case, especially Person C with the CD3 framework. Person B, do you have any similar frameworks in place in how you decide the priorities of your roadmap or work?

**Person B**: Not at this point because our things are pretty simple. We don't have competing demands from end users at this point. So I haven't had the need to have anything besides doing the normal OKR things. So we have few enough things that we can sit down with the team, sit down with stakeholders, talk about what you need—you need this thing, okay, then we'll build that over the next quarter. If we had more things to do than we could fit into a year or a quarter or whatever we would do something but we haven't had the need. 

**Person A**: Who are the stakeholders that would typically be in such a conversation?

**Person B**: So for me it's okay—so the area we are in, could zoom out a little bit and squint a bit—you could call us infrastructure. You shouldn't, but you could. Then we have a bunch of colleagues that are working in the, you could say, more classical value streams. So we have lead engineers in both areas. So my stakeholders currently are lead engineers for those value streams in the other part of the organization, the principal engineers that go across all of this to try and talk about the forward phrasing, the future stuff. Then I always try to find engineers in the individual teams that are actually using these things. And then of course I have my own delivery lead who is ultimately responsible for the deliveries we do in our area. I think that's currently my main stakeholders. And then support of course also but that's more your area actually because I have not—

**Person C**: Yeah, I think it's hard because we get people from all over because of the things—for example, there's what there is—is this FinOps, they're called FinTechOps, I can't remember—they came to us last year because they had a hunch about licenses that we could help them with, so they're a stakeholder and they had a very big case in the CD3 score for us. Then we have the is people working on the product, right, and they're engineers some of them—and then we have—but in general I would say engineers come with an idea, they come with an ask, and then we set up a meeting. If we can get them to give us an issue in GitHub that is the preferred—it's happened once—the rest of the times we just patch them in a chat somewhere, we set up a meeting, and then we create the CD3 work on the side with them and then we include who needs to be there.
We try to avoid getting too far ahead of ourselves including PMs and less because things get problematic when you do that. And it can be hard to harvest the value. Sometimes they're there. It depends. And you really need the engineers to speak up which is quite important. So I would say it's a bit of everything. And when we do new location, it is everyone who has a stake in it and will call you to ask why is this not working? When do you expect to be ready? The time schedules for creating a new location are insane. I've never seen a map so crazy when it comes to timing.

**Person A**: For example, the platform user—what you're currently working on and what's the progress of that besides the quarterly roadmap shows? 

**Person C**: We have syncs normally when we work intensely on the feature. So that's where we touch point with them. Normally it is just small chats where they get notified when it's depending on the scope, right, which if it's big things like AWS we do post continuously of the status and then the syncs are probably the most important ones because posts can go and die whereas the sync they are there in person and we can get some feedback which has proved valuable but often it has a life cycle so sometimes you start up with a sync and there's a lot of interest and then the sync really dies—the number of people is not relevant anymore so it's kind of a tool you need to know when to use and then you need to do something else afterwards. We haven't figured that one out yet.

**Person A**: But you don't have any public roadmap that people could just look up.

**Person C**: This roadmap is in Git but we do not put months or dates on it. That is sacrilege in my mind to do that unless there is a real deadline—at this date the state will cut off this connection for whatever reason. That is a real deadline. Everything else is up to discussion and I will not put dates on it because it's about delivering. That's why you have the CD3, because the CD3 tells you that this is the most value you can produce. So it doesn't make sense to do anything else unless you can prove there are dependencies that increase the value.

**Person B**: Yeah. For our things, it depends on the service we do, but normally we have a specific status page for the different errors we have. If you're doing a migration, for instance, the people involved in the migration will have a Confluence page for all the objectives and key results. For teams, it's public within the group, so anyone can go in. It's not a normal thing for an engineer to do, but if you wanted to, you would be able to go in and look at that burndown of clusters that we migrated. Weekly average depends.

**Person A**: And how often do you then revisit those priorities, especially if you have those things and all of that?

**Person B**: If things are burning, then we'll do it more often. I think the sync we have with our leads and PE, that's every three weeks—bi-weekly now, I think it's three weeks. Yeah.

**Person A**: It depends? All right. One in the box for that. Do you actually collect feedback from your platform users?

**Person B**: So we've tried a couple of things. So we do do surveys every now and then but the results are very mixed because you never know how you hit people. There is also an annual State of Developer Productivity, I think it's called, where you can ask to have some questions included in that questionnaire. So we try to poll the entire group once a year. But at that level it's very hard to get granular results, right? They won't tell you that they're happy about the SQL Server. And even if they do, then that would just be one engineer. So the thing I try to do is to get feedback from the lead engineers by as much as possible meeting them in person. So a lot of the teams that we actually deliver services to are located in [City], which is the other end of [Country]. I'm situated in [City]. So I try to make a habit out of going to [City] at least once a month and then whenever I'm there to set up in-person meetings because that is where I can actually gauge how satisfied they are. Because in general when things are going we don't hear anything. Yeah.

**Person A**: Yeah, the issue with infrastructure, right? All right.

**Person B**: Yes, exactly.

**Person A**: When was the last time you collected some feedback?

**Person B**: I mean that thing of going in person, I literally did it last week. So we went and visited everyone. Yes. Yeah.

**Person A**: Very good. And what did you do with the collected feedback from last week? In general, is that going back into your prioritization or...

**Person B**: So I did two things. So I brought one of my senior engineers along just to have one more person in the room. And then the dissemination of information afterwards is then pretty informal. But I'm then bringing it up in meetings and things to my senior engineer that now we could bring up in conversations that so they are actually saying this or when we asked them this was not a problem. I also use that feedback literally to calibrate my roadmap for next year.  There was a service I now have moved to—we'll maybe look at it at the end of next year—but the main thing they're actually requesting is this thing, and I moved that to start of next year. But nothing like formal. We're not documenting it, writing it down or anything at this point.

**Person A**: All right. That was already the feedback part. Looking at the platform success, we already also have a lot from in between the lines. The first one there is always a bit tricky, which is what is the goal of your platform? But in this term a bit more like does it tie into business goals or something like that? What's the business side of your platform?

**Person C**: To think on with about regards to success. The way we right now comprehend success is that we look at the testimonials, we look at the reactions we get from the people we help essentially. Sometimes we give them a survey and that—we get back—that one thing that we focus on is their satisfaction with the team because as long as we have good communications they can handle that something is not functioning correctly because they know they will get the help. We are building friendships essentially, right, and that is fundamental to get the trust that you need to be able to execute because we have to do hard things sometimes. 

**Person B**: So, we actually have a to-do for next year as a team to sit down and do the whole product strategy and vision. And then the team itself would figure out what the value would be. But from a business point of view, it's actually all the way back to that domain model and the products and the applications. So in that it literally says that—I can't remember it verbatim anymore—but that the Data service platform is delivering stateful services to reduce cognitive load. And then for the individual business capabilities it would say things like this is in order for us to have a SQL offering. So it's very focused on that—to reduce the risk that other colleagues that are more business facing—that they have to do that non-differentiating heavy lifting of things.

**Person A**: And you already talked a bit about the impact of your platform. Do you track the impact of your platform in general onto the whole organization?

**Person B**: So for things like the developer portal thing, for instance, we're collecting metrics and things looking at whether people are pressing buttons but that is only showing whether people are using our services, not whether it's delivering any value. I know we have high-level KPIs about how much more value is being delivered of our entire digital organization, but that is mostly looking at the amount of things that we are doing, the number of things we are delivering. There's one—to my knowledge—made any direct correlation between that and the platforms. They're looking at before and after the reorg, can see that we're delivering a lot of things. Yeah, so it's not being specifically tracked in a way that I as an engineer would say was valid. Because of course people are counting number of developer hours and trying to say that we have freed up this many hours by having asked someone about how much time do you use doing that? Okay, you don't have to do that anymore. Multiply that with the number of engineers—we've saved a number of thousands of hours. But, to me it's a measure, but I don't have a strong correlation. A lot of that is being used.

**Person A**: Metrics. What do you do with those metrics then? Is that just informal or do you actually...

**Person B**: So usually there will be things like bonus targets where the purpose is to save this number of developer hours to basically have funding for headcount in some other teams. So if a team is tasked with improving developer productivity, in the end their goal is to save any number of developer hours. So it's being used like that. It is also being used to steer and verify that we're heading in the right direction but it's at a higher level. It's another thing where as a lead engineer is counting the number of hours I save and use that directly just here. For me the success metrics is more down to those conversations we talked about before and whether or not people are happy basically. Yeah.

**Person A**: All the last bit which I always call the half last bit because that's a bit tricky. But you actually already talked a lot about it between the lines. So it's investment and the whole funding model. How does your team actually get funded?

**Person B**: So we have the number of people on the team and then there's a rough number for how many of those are dedicated for keeping the lights on and how many are allocated for slash improvement investment work. So of course it's not actual headcount rate but it's like a rough estimate. So for a team like mine I think currently my KTLO allocation is about 70% or something like that. So 70% of my team's time is towards reducing toil, keeping things updated, right-sizing things, cost savings, the usual KTLO work. And then that portfolio process we talked about before where there's bottom-up information gathering and then we get those high-level goals. Those are the ones that the development headcounts are used for. Again, it's fluent, so anyone on the team can actually be working on it. So, I roughly know that around 30% of my time could go towards building these new capabilities that I've been asked to do. Together with that comes investment. So, I have—for the different business cases that we pitch that comes down again to me—there's also money and potentially headcount. So my input was together with the lead engineers and principal engineers in manufacturing. If we were to build this capability, we will roughly need this amount of money to do this.
And that money is then allocated to me when it comes back into—and then in the portfolio there's a suggestion for a measure. So for me it's: here you get this amount of money, you should grow your platform adoption with 20%. And then it's deliberately vague because it's up to me to figure out what my baseline is, how I measure the 20%, and how I show that this thing made my platform grow. And that is then a discussion I have with the usually delivery lead that is responsible for that key result. So they're the one in the upper management who is in charge of that key result and I can then go to him and then have the discussion about: so you say 20%. How do we go about measuring that? Yeah. Yeah, so that's roughly how it works.

**Person A**: Do you have executive sponsors like C-level support or whatever? Especially the project sounds a bit like that would come from...

**Person B**: Yes. So the portfolio things—for each of those key results, there's an owner of that key result. You could say that the delivery lead level of the organization are the ones that are the sponsors of this. So as a person, his responsibility is to make sure that this thing gets delivered. And then it's up to the teams that align with those to help that make that happen. That is the person who—I mean it's not his money but he's made in charge of that investment, right. So for development work—that I was missing—the other part of my funding is my KTLO budget. So those are two separate budgets. So I have a whole separate budget for keep the lights on. So that's licensing, strictly speaking, some headcounts, consultants, things like that. That is owned by my area's delivery lead. So we have a delivery lead for that part of infrastructure I'm in and then it's not C-level but you could say there's a C-level sponsor for the portfolio investment that we're doing. So management has in this year 30%, my local delivery lead has the responsibility for the 70% of that investment. But yeah. Yeah. So for the KTLO work that is a delivery lead whose responsibility is to run all those delivery for all those platforms.

**Person A**: Why exactly do these sponsors support the platform? If you know that, what value do they get from the platform?

**Person B**: So I mean he's mostly motivated because that's the strategy. His task is to keep things running. For the portfolio work, all of that is linked back up into this year's investment strategy. So there's a tiered list of priorities, priority one, two, three, and then there's innovation investment. And as a group there's an effort to push for more digitalization. So it's like the bottom-up process identified the need for doing that and then that's the motivation.

**Person A**: All right, perfect. You're actually the first one that can talk and give good answers to that—or even answers to that—in the first interview I did here. So that's amazing to hear.

**Person B**: Yeah. Awesome.

**Person A**: And with that we are through with our script.