Hey Joe you nailed it with your failed data project post. That is 90% of what I get called in to do with companies and pretty well you have my playbook covered!!!
I’m gonna add a couple of complications to make it more fun:
1. You usually have someone there who is left over from the previous data project and they have both good knowledge (which they may not share immediately) and a whole heap of historical bias why certain things won’t work
2. The application stack where you source the data from is constantly changing - the customer wants a time invariant view of the data but they have no real record of what happened on the past. They only know the data is wrong when they see it because you forgot to include the “missing data from 2020 that got deleted” - oh frabjous day, he chortled in his glee
Great post and episode!! Loved the format, it had a "don't teach me what to think, teach me how to think" vibe :)
Dark matter systems do seem like a great place to start data initiatives because there is an existing Job to Be Done and a suboptimal solution. Feels like it has been easier to show "value" in this type of projects.
The conversation on buying tools reminds me of the guitar trap metaphor where many people buy guitars to learn how to play, but then it ends up standing in the back of the room. With a "buy solution first" mentality it is easy for the solution the end up in the back of the room as well 😅 (I just can't remember who wrote this analogy, if anyone does please remind me 😄)
Makes me wonder about my scenario where the project didn’t “fail” as it produced something that the immediate stakeholders get value from, but because it’s unmaintainable and cannot be easily extended, very poor coding and modeling practices, it’s a failure in that it can’t serve the next use case that is of major value and importance.
I think the strangle architecture works in this case as well, since I’ll need to build this thing greenfield both to get understanding of the data to properly model it, and do it in such a way that we can easily plug new use cases into it.
Great article! A few potential additions to the list: If you inherit a legacy stack, you can speed up the requirement-gathering process by finding some logs or monitoring tools already running. Once, I inherited a legacy SAP BW installation into my stack, and looking into what was being used really helped me stop many things from day one and focus the inherited team’s attention (with all the biases Martin mentioned) on things that mattered. In terms of technology I focused on tools which can deliver rapid positive user experience, so I just started a few OLAP cubes which were accessible even from excel which were the most understood channel by my users.
Hey Joe you nailed it with your failed data project post. That is 90% of what I get called in to do with companies and pretty well you have my playbook covered!!!
I’m gonna add a couple of complications to make it more fun:
1. You usually have someone there who is left over from the previous data project and they have both good knowledge (which they may not share immediately) and a whole heap of historical bias why certain things won’t work
2. The application stack where you source the data from is constantly changing - the customer wants a time invariant view of the data but they have no real record of what happened on the past. They only know the data is wrong when they see it because you forgot to include the “missing data from 2020 that got deleted” - oh frabjous day, he chortled in his glee
Great post and episode!! Loved the format, it had a "don't teach me what to think, teach me how to think" vibe :)
Dark matter systems do seem like a great place to start data initiatives because there is an existing Job to Be Done and a suboptimal solution. Feels like it has been easier to show "value" in this type of projects.
The conversation on buying tools reminds me of the guitar trap metaphor where many people buy guitars to learn how to play, but then it ends up standing in the back of the room. With a "buy solution first" mentality it is easy for the solution the end up in the back of the room as well 😅 (I just can't remember who wrote this analogy, if anyone does please remind me 😄)
Dark matter = instant classic.
Makes me wonder about my scenario where the project didn’t “fail” as it produced something that the immediate stakeholders get value from, but because it’s unmaintainable and cannot be easily extended, very poor coding and modeling practices, it’s a failure in that it can’t serve the next use case that is of major value and importance.
I think the strangle architecture works in this case as well, since I’ll need to build this thing greenfield both to get understanding of the data to properly model it, and do it in such a way that we can easily plug new use cases into it.
Zombie projects are worse than dead ones!
Great article! A few potential additions to the list: If you inherit a legacy stack, you can speed up the requirement-gathering process by finding some logs or monitoring tools already running. Once, I inherited a legacy SAP BW installation into my stack, and looking into what was being used really helped me stop many things from day one and focus the inherited team’s attention (with all the biases Martin mentioned) on things that mattered. In terms of technology I focused on tools which can deliver rapid positive user experience, so I just started a few OLAP cubes which were accessible even from excel which were the most understood channel by my users.