12 Comments
User's avatar
Bianca J. Schulz's avatar

I also think there is a lot of foundational work to do. If not even all people in a team understand what the meaning of all data is for the business, how are AI agents supposed to understand it.

In my opinion it starts with the organization. The right people need to come closer together and work on the right things at the right granularity, and getting that right is pretty hard! It doesn't fit into a single instruction manual.

And what about the data that is scattered everywhere, structured, unstructured, in varying quality, and then not everything is properly connected yet. So I think the work is not going to get less.

Someone who writes about systems here on Substack and comes from the automotive industry told me recently that we will have to work on establishing these connections for the next 30 to 40 years in his opinion.

I mean, there are not just data warehouses, there are also machines, hardware, all kinds of other devices that also produce data. And then, as you say, the humans with all their human characteristics who somehow has to manage to build a good system out of all of this, which is really hard and challenging.

John Y Miller's avatar

Agree with your observation - that there will be more work to do and we’ll need more direct collaboration internally!

Kalim Saliba's avatar

This really resonated. The hard parts have become even more important to get right because the promise of increased delivery velocity means you can build a lot of things on the wrong fundamentals, which then makes the harder parts even harder.

I believe a focus on the fundamentals increasingly needs to look across functions because the shape of R&D and GTM motions needs to dramatically shift to really make 1 = 10.

Corrine's avatar

If your org struggling with "sunsetting" human process workflows "no longer" originated by humans because "no longer" useful , can see how deploying "autonomous" AI Agents would be self-defeating. Continue to be baffled by orgs counting widgets that do "not" net things.

Can "high" self-agency when no one is looking really be operationalized? Very difficult with humans and now facing same challenges with AI 🤔.

Maarten Vlemmings's avatar

Wish I could’ve been there; would’ve been good to hear the real stories from the veterans. And I always appreciate your honesty, Joe.

I’m glad the naive ‘let’s stick our head in the sand and wait till it’s blown over’ attitude seems largely gone and we can get constructive together. I love how Claude helps me to quickly build up the scaffolding for a new data platform. But it does so using a big library of architectural content that I built up over the years. And even then, I discovered it gets creative and suddenly starts to denormalize my upstream pipelines when I told it the structure of data should be 1:1 with the source. Interesting. We move a lot faster, but I’ve learned that you need to start doing your QA well on time, coz it’ll grow too big to oversee very quickly.

John Y Miller's avatar

Loved hearing it presented live… and especially the agreement on your conclusion… “You’ve been told you don’t have time for fundamentals. The data says you don’t have time to skip them.”!!

Joe Reis's avatar

Thanks and great to meet up with you this week

Maury Carollo's avatar

Going to mention the same.. "You’ve been told you don’t have time for fundamentals. The data says you don’t have time to skip them." Joe is great for these insightful quote nuggets, and injecting quotes that need to be repeated as they are relevant today as they were in the past. It seems each generations of leadership, VP and C level, repeat the same mistakes from the past. Can't skip the foundational core principles; it is context for AI. "Show me the incentive and I’ll show you the outcome" - CM. As we try to understand data requirements from a 2 sentence user story for a system that nobody owns... hot potato... ;)

Matt Martin's avatar

POC purgatory is a real thing. You actually need to put points on the board, e.g. ship meaningful products to production, to move forward as an org.

If all you want to do is POC’s, go be a lab rat in some research firm

Marton Horvath's avatar

Great article, thank you for sharing it with us. My main question: how can we build fundamentals fast? If it takes ages, nobody will start working on it because the reward cycle is just too long.

Joe Reis's avatar

Fundamentals are fundamentals for a reason…

Harshvardhan Pandey's avatar

Absolutely, and as you alluded to in the conference and in your post, getting the data modeling right is the most significant yet underacknowledged challenge of the AI era. This was true ten years ago when ML pipelines took off, and it's going to be true now.

On a lighter yet honest note, one of the key takeaways for me was that 'meta-metadata' is the metadata of the AI era. Thanks so much for being so warm at Undercurrent!