Wartime Data Teams
Joe's Nerdy Rants #73 - Weekend reads, podcasts, and other stuff
The amount of anxiety I’m seeing is sky high. When I talk with people around the globe, there’s a general sense of doom, whether it's concerns about layoffs or the world's future. I’ll try to keep this rant even-keeled, as there’s enough news to keep you panicked if you choose to do that to yourself.
While reading "This is Wartime” from the wonderful Not Another CEO newsletter, the author David Politis advises CEOs: “...do not delude yourself: this is wartime. Stop watching the news, assume the worst and start taking action now.” He lists what a CEO should do, such as contingency planning, cutting costs, and securing capital. I’ll argue these are things you should also do in Peacetime, but the essence of his arguments is rock solid.
I first read about the Wartime CEO in Ben Horowitz’s “The Hard Thing About Hard Things” when the book was published in 2014. As Horowitz describes, “In peacetime, leaders must maximize and broaden the current opportunity. As a result, peacetime leaders employ techniques to encourage broad-based creativity and contribution across a diverse set of possible objectives. In wartime, by contrast, the company typically has a single bullet in the chamber and must, at all costs, hit the target. The company’s survival in wartime depends upon strict adherence and alignment to the mission.”
As a side note, if you’re a founder or a leader, I highly recommend reading The Hard Thing About Hard Things. It’s full of great stories from the trenches about the rollercoaster of starting and leading a company. The main point is business is fucking hard and your main objective is to push through, no matter what. Most business books are fluff, written by people with more luck than smarts. Not this book. Horowitz is smart, lucky, tenacious, and wise enough to know the difference.
Anyway, back to Wartime. Since the audience of my newsletter is mostly data people, I started wondering how data teams need to behave in wartime. I’m not talking about actual war, although I know engineers who’ve kept working while being attacked in war. I’m talking about the current state of many businesses whose prospects are shrinking and are left with a single proverbial bullet in the chamber. And understand my advice will vary. If you’re at a startup, your experience and ability to execute drastically differ from working at MegaCorp, Inc.
Here’s one business war I experienced. A week before the Great Financial Crisis (GFC) hit, I took a job as a supply chain forecaster and planner. My job focused on forecasting (duh) and helping implement a new supply chain planning and forecasting system. Good forecasting depends on neat historical time series data that contains enough patterns and trends to extrapolate the past to the future. The GFC decapitated nearly every lever of a functioning economy—cash, investment, credit, etc.
My company was an e-commerce company, and consumer demand dropped off a cliff. Business looked very dire. Our mission - the single bullet - was to keep cash flowing so we could survive. A big part of e-commerce cash flow is inventory and outstanding orders. That’s most of the cost, and the priority was cost management. We had to cut or eliminate orders from some vendors and negotiate new payment terms with others. I was responsible for identifying areas to cut, and had to be the bearer of bad news on many occasions. This was heartbreaking because many of these vendors were small businesses, and I knew my decisions would likely put them out of business. But in wartime, that’s how it goes. Everyone in the company realized the gravity of the situation and acted accordingly with the singular mission of surviving. We were transparent about the situation and the impacts if we couldn’t keep cash flowing. Surprisingly, we didn’t have to do layoffs. Eventually, things turned around. Consumer demand picked up.
The scar tissue remains with me to this day. Things in business can turn on a dime. Black swans are more common than you think. Once in a generation crises happen every few years. The big lesson: don’t take things for granted and avoid complacency. Understand how you can succeed, and be paranoid about how it can all come crashing down.
COVID and the aftermath were like two wartime events in one. At the time, I was running my data engineering consultancy with Matt Housley, and we were taking on many new clients. The National Emergency happened in March 2020, and business fell off a cliff seemingly overnight. Most of our clients laid off their data teams. Thankfully, Matt and I had enough cash reserves to weather what we expected to be several months of nothing. Then, in one of the wildest head fakes I’ve seen, those data teams were hired back a few weeks later. We were suddenly busier than ever. Interest rates dropped to zero (sometimes negative), and many data teams purchased a mall’s worth of Modern Data Stack tools and toys.
A few years later, interest rates rose, and those same data teams, now bloated from excess exuberance, were being looked at with new glasses. They are not rose-colored ones, but they are more like creepy aviator glasses. Overall, the tech sector saw a quasi-mass extinction event, with engineers getting laid off from Big Tech and small companies. Data teams were getting cut or eliminated. That trend continues today, and many data teams I meet with are battle-hardened. For these data teams, it’s been wartime for the last couple of years.
With new tariffs and a trade war underway, the economy is under even more pressure. You're probably scrambling for new options if you’re in the physical goods space. One friend who imports furniture from China got a surprise bill of an extra USD 40,000 per shipping container. Several shipping containers were on the water, costing them a few hundred thousand out of pocket. This is a family-run furniture operation, not some mega corporation. That’s a lot of unexpected money to spend.
Similar things are happening everywhere. Businesses that weren’t doing the massive layoffs seen in Big Tech are now considering or doing layoffs. Some companies are closing. Add a volatile stock and bond market and a rapidly reshifting geopolitical environment to the list, and things will get…interesting.
So, what should a Wartime Data Team do? Here are some things I’d focus on. It’s not a complete list, but you’ll understand how I approach this.
Have a singular focus on whatever the company needs to do to survive. Hopefully, your company’s leadership is communicating this mission to you. If not, that’s probably a red flag in itself. Do what it takes to use data to help achieve this mission: cut costs, sell more, innovate quickly, etc.
Deliver quick and visible wins. Don’t hide away in the shadows. You have the advantage of knowing the data better than anyone. Find ways to leverage your domain expertise to help the business. Be an integral part of the business.
Be scrappy and resourceful. You’re not getting new tools or tech unless it helps you be more productive. Make do with what you have and use those tools to their max. Leverage AI as much as possible if it helps improve efficiency and output.
Collapse boundaries and siloes. Everyone needs to wear multiple hats. Work shoulder to shoulder with product, engineering, and leadership.
Move fast. Slow deliberation is practical if it means you can move faster. But don’t get stuck in long, drawn-out planning cycles. Your goal is survival, not making another committee. Have a bias for action.
What’s funny is these are things I also do in Peacetime. I’m a big fan of staying lean and mean. Of course, most companies and data teams are not like this.
Like all things, this will pass. Eventually, everything will be full of joy, including unicorns and rainbows. Just stay focused and ruthlessly deliver wins to keep the business afloat.
Please listen to the audio above or on Spotify (or your podcast platform of choice).
Have a wonderful weekend,
Joe
Cool Weekend Reads
Here are some cool articles I read this week. I hope you enjoy them.
Data Engineering: Now with 30% More Bullshit
There Are No New Ideas in AI… Only New Datasets
Guiding an LLM for Robust Java ByteBuffer Code
Vibe Coding is not an excuse for low-quality work
Tech hiring: is this an inflection point?
Principles for coding securely with LLMs
This is Wartime - by David Politis - Not Another CEO
Johnson & Johnson Pivots Its AI Strategy
Upcoming Events
With AI reshaping how businesses leverage data, ensuring high-quality and reliable data is critical for success.
Data teams must navigate complex pipelines while minimizing risk and maintaining reliability at scale. Making your data AI-ready isn’t just about having the right technology—it also requires strong processes, governance, and clear ownership.
Join us on April 24 at 9:00 AM Pacific Time (PT) for a quarterly Data + AI Quality Day live event on The AI-Ready Data Blueprint: Ensuring Trust with Data + AI Observability.
Agenda
Session 1 | AI-Readiness: Architecting for Scalable, Reliable Data - 9:00 AM PT To successfully leverage AI, data leaders must ensure their data is trusted, scalable, and AI-ready. This session will cover key strategies—data quality and governance to observability and scalability—to help you build a future-proof data foundation for AI success.
Session 2 | Getting AI-Ready Through Data, Systems, Code, and Models - 9:30 AM PT
Scaling trusted, AI-ready data isn’t easy—are your data practices set up for success? Join our panel to hear industry leaders share real-world strategies for data + AI observability, governance, and reliability to power AI with confidence.
Session 3 | How to Get AI-Ready with Data + AI Observability on Snowflake - 10:15 AM PT
AI-ready data isn’t just about technology—it requires quality, trust, and data + AI observability across your entire ecosystem. Join us to explore how Snowflake and Monte Carlo help teams improve data reliability and governance to accelerate AI success.
Register Today! https://hubs.li/Q03hKbC10
About Data + AI Quality Day
Join data leaders from engineering, analytics, governance, and AI as they share proven strategies for making data AI-ready at scale. Explore emerging trends in data quality, actionable practices for data + AI observability, and the foundations of reliable, responsible AI. Learn how to build trust in your data to drive more accurate models, smarter decisions, and stronger business outcomes.
It’s been two years of Deep Dish Data—two years of real talk, great guests, and honest conversations about the state of data.
On April 29, we’ll go live from Denver to celebrate with a conversation on where data engineering stands today and where it’s headed next.
My semi-regular co-host, Mark Balkenende, and my unhinged twin, Carly Taylor, join me to discuss what we’ve learned from the field, what teams are struggling with, and what’s on the horizon.
We’re also diving into fresh findings from Matillion’s Data Integration and AI Readiness Report—and let’s say some of the results might surprise you.
🎙️ Deep Dish Data: State of Data Engineering
📅 April 29 | 8 AM PT
📍 Streaming live on ON24
Hope to see you there. This one’s going to be good.
Check out Matillion’s AI Readiness Report
SLC Low Key Happy Hour - May 9. Register here
I’m excited to be going to London in May. Join me for the European data streaming event of the year with Confluent!
I’ll discuss the implications of AI-generated content and a digital ecosystem drowning in bad data and software, and how to avoid this!
Join me at #Current25 London and use my discount code L-PRM-JOEREIS for 50% off!
See you there! https://cnfl.io/438XnTV
Snowflake and/or Databricks - June TBA
Iceland - Global Data Summit, June 23-24. Register here
Australia (Sydney, Melbourne) - Data Eng Bytes, July 24-30. Register here
UK - Big Data London, September 24-25. Register here
Helsinki Data Week - October TBA
More to be announced soon…
Podcasts
Freestyle Fridays - Wartime Data Teams (Spotify)
Tim Berglund - The Art of Developer Relations, Hardware Hacking, and More (Spotify)
Freestyle Fridays - Coding is Fun (Again) (Spotify)
Juhani Vanhatapio - From the Arctic Circle to AI (Spotify)
Freestyle Fridays - Shifting Left AND Right. The Data Engineering Lifecycle in 2025. - (Spotify)
Mark Freeman - Shifting Left in Data, Startup Rocket Ships, and More (Spotify)
Vaibhav Gupta - BAML and AI-First Tools (Spotify)
Freestyle Fridays - Figuring Out Your Next Move (Spotify)
Willis Nana - Navigating Data Engineering Leadership, YouTube, and More (Spotify)
Salma Bakouk - Data Observability, the Balance of Running a Startup, and More (Spotify)
Freestyle Fridays - Public Speaking Tips w/ Jordan Morrow (Spotify)
Simon Späti - The Art of Writing about Data Engineering (Spotify)
Todd Beauchene - The Early Days at Snowflake, Modern Data Platforms, and More (Spotify)
There are way more episodes over at the Joe Reis Show, available on Spotify, Apple Podcasts, or wherever you get your podcasts. Also available on YouTube.
Thanks! If you want to support this newsletter
The Data Engineering Professional Certificate is one of the most popular courses on Coursera! Learn practical data engineering with lots of challenging hands-on examples. Shoutout to the fantastic people at Deeplearning.ai and AWS, who helped make this a reality over the last year. Enroll here.
Practical Data Modeling. Great discussions about data modeling with data practitioners. This is also where early drafts of my new data modeling book will be published.
Fundamentals of Data Engineering by Matt Housley and I, available at Amazon, O’Reilly, and wherever you get your books.
The Data Therapy Session calendar is posted here. It’s an incredible group where you can share your experiences with data - good and bad - in a judgment-free place with other data professionals. If you’re interested in regularly attending, add it to your calendar.
My other show is The Joe Reis Show (Spotify and wherever you get your podcasts). I interview guests on it, and it’s unscripted, always fun, and free of shilling.
Would you like me to speak at your event? Please submit a speaking request if you want me to give a workshop or talk at your event.
If you’d like to sponsor my newsletter or podcast, please get in touch with me.
Could you be sure to leave a lovely review if you like the content?
Thanks!
Joe Reis
it's pretty incredible to have to ponder this. And yet, your are right, Joe...