AI Underpants Gnomes, C-Suite Edition

Joe's Nerdy Rants #26 - Weekend reads and other stuff

I am currently in Paris, spending the weekend meeting up with friends, seeing my favorite DJs, and enjoying the French winter vibes before I return to the Bat Cave for an extended period of book writing, course creation, and very few meetings (by default, I avoid meetings, and they will become even more sparse).

Some might wonder, “Didn’t Joe say he’s stopping international travel?” Yeah, I am stopping for a couple of months. But as I told someone today, international trips are normal for me as trips to any domestic spot, sans the jet lag. When you fly a lot, your perspective of time and space shifts. Old habits die hard. C’est la vie.

Merci,

Joe Reis

AI Underpants Gnomes, C-Suite Edition

This week, I attended the CDO and CAIO Summit in Boston. I caught up with some old friends and made some new ones. Lots of heavy hitters in attendance, with CXOs (X = data, AI, data and AI, data and AI and stuff, etc) from many prominent organizations in attendance. Despite the title power in the room, I couldn’t help but notice a couple of threads.

First, the job of a CXO has a ton of potential but looks pretty frustrating¹. My impression is the CXO position is still immature, in flux, and misunderstood. After a brief honeymoon period, you’re on your own, often against adversarial executives jockeying for control of data and technology initiatives. The other complication is that the board of directors and CEO who hired you likely don’t understand the complexities of your proposed data initiatives. As a result, CXOs are often short on budgets and timelines and attend endless “alignment meetings” with high expectations to “deliver value” in record time. By the way, that also describes the data field in general. We’re always only another project away from adding business value. But deliver it now, because the business is tired of waiting.

Second, AI is the hotness capturing every corporate board and CEO’s attention. Not having an “AI Strategy” is akin to not having a website. “WTF…where’s the AI?” screams a CEO who might not even understand what the A and the I stand for. Hence, the zeitgeist is “every company is an AI company.”

What does this have to do with Underpants Gnomes? Back in the day, South Park did an episode called “Gnomes,” a very short clip hysterically summarizing the entire Dot Com boom - “Collect Underpants…?…Profit.” I wrote about AI Underpants Gnomes last year. Little did I know that right after the article was published, ChatGPT would kick the AI hype cycle into warp speed. These events are definitely correlated ;)

The AI Underpants Gnomes strike again. Every company must be an AI company because of FOMO and stuff. If you listen to reasons why, you’ll hear things like “We need to unlock the power of AI to make our workforce more efficient” and “AI will unlock new opportunities for our business.” These are very empty claims along the lines of “AI…?…Profit.” Yet highly paid executives say these platitudes with a straight face.

As my homie, Karl Ivo, wrote in my LinkedIn post, “Anecdotally, recently it seems CFOs and other C-folks are way more excited about implementing Gen AI than actual CIOs and CDOs.” I see this, too. It’s like the people who understand data know it’s difficult and want to get the basics right, and the ones pushing AI are doing it for…other reasons.

It’s wild. But it gets crazier. Look at what organizations hope to use AI for - improve workforce efficiencies and unlock new business opportunities. Then, consider the root causes of these, which are often the result of politics and human misjudgment (inefficiencies) and domain expertise mixed with luck (opportunities). There’s a notion that AI will compensate for human deficiencies and “make the world a better place,” essentially saving us from ourselves. I’m very bullish on the potential of AI, but I disagree that our AI overlords will save us from ourselves. These models are trained on data from…humans². As much as we want to optimize our models to outperform our defects, the circular reasoning creates some awkward contortions. But now we’re getting into the realm of science fiction, when the reality is most companies can still barely do basic dashboards and reporting. The chasm between AI overlords running an organization and the Excel-hell in most organizations is very wide.

So, will every company become an AI company? To paraphrase my good friend Sol Rashidi, focus on your data strategy before your AI strategy. But these days, companies often want to jump straight into AI without any strategy. Having been through other ML/AI hype cycles, I’ve seen most ML and AI projects crash and burn because insufficient time is spent understanding WHY such an initiative is mission-critical to the organization. These projects are doomed from the start for many reasons, including skipping the building of a solid data foundation (data quality, governance, data engineering, etc). ML/AI projects wallow in POC purgatory. Because results are slow, investments are pulled back. For the few organizations putting in the hard work upfront, success is far more likely.

Will the AI Underpants Gnomes fill in the middle question mark of “AI..?…Profit”?Probably not. The current wave of GenerativeAI has a ton of promise to scale in areas where humans won’t scale, but I don’t think this is yet the panacea for human-created problems in organizations³. The old problems that plagued data initiatives - support, alignment, budget, and willpower - will continue to get in the way.

Listen to the audio clip above on this topic, which is also my 5-Minute Friday on Spotify.

---

Cool Weekend Reads

Here are some cool things I read this week. Enjoy!

Tech, AI & Data

Excuse me, but the industries AI is disrupting are not lucrative (The Intrinsic Perspective)

“Call it the supply paradox of AI: the easier it is to train an AI to do something, the less economically valuable that thing is. After all, the huge supply of the thing is how the AI got so good in the first place.”

I find this a handy read if you’ve been wondering about the economics of things that Generative AI will create. The argument is Generative AI will drive the cost to zero of whatever it consumes.

A further argument - which I’ll dive into very soon - is whether the notion of LLMs as an analytical interface makes sense. I understand both sides of this argument, but the bigger question is, what is this worth to companies? Is it nice to have, or truly game-changing? Stay tuned…

Weak-to-strong generalization (OpenAI)

“We study a simple analogy: can small models supervise large models? We show that we can use a GPT-2-level model to elicit most of GPT-4’s capabilities—close to GPT-3.5-level performance—generalizing correctly even to hard problems where the small model failed. This opens up a new research direction that allows us to directly tackle a central challenge of aligning future superhuman models while making iterative empirical progress today.”

This exciting research will hopefully unlock some very cool things with GPT. I am curious about reverse feedback loops, namely big models influencing weak models toward “certain behavior.”

Tech predictions for 2024 and beyond (Werner Vogels)

Werner Vogels has been the CTO of AWS for ages and is someone I admire. He’s frank and has no BS, yet he is thoughtful and innovative. I find his prediction on education particularly interesting (read the article to learn more), as education is an intense focus for me in 2024 and beyond.

Business & Startups

Europe’s technology startups are doing just fine (The Economist)

This year, I spent a lot of time meeting with European companies of all sizes. The feature and bug of Europe is things move a bit more slowly than in America. In Europe, I observe a business culture that’s generally more thoughtful and methodical than American companies.

Revisiting The Death of a Venture Fund (Investing 101)

A big story that flew under the radar was the closure of OpenView, a prominent VC firm. This is odd for several reasons, which this article dives into.

The diminishing half-life of knowledge (Redowan’s Reflections)

Ever feel like you master something, only to lose your edge and see your skills get rusty? This is a real thing. Skills and knowledge have a half-life.

New Content, Events, and Upcoming Stuff

Monday Morning Data Chat

Coming up…

Mike Ferguson from Big Data LDN

This will be the final show for 2023. We’re coming back with a packed roster in 2024!

In case you missed it…

Tristan Handy - Data Engineering Ecosystems, Moats, Semantic Layers (Spotify, YouTube)

Sol Rashidi - Getting Business Value From Data, the CXO Playbook (Spotify, YouTube)

Sarah Nagy - Automating Analytics w/ Generative AI (Spotify, YouTube)

Dave McComb - Knowledge Graphs, Semantics, and More (Spotify, YouTube)

The Joe Reis Show

Coming up…

Will Gaviria Roja, Sol Rashidi, Steve Nouri, and more…

This week…

5 Minute Friday - AI Underpants Gnomes, C-Suite Edition (Spotify)

Eleanor Thompson - Partnerships Deep Dive (Spotify)

Ben Rogojan - Ben Rogojan, aka Seattle Data Guy - Origin Stories, Content Creation, and More (Spotify)

In case you missed it…

Karin Wolok - All Things DevRel and More - (Spotify)

5 Minute Friday - “This Was All Predictable” (Spotify)

Peggy Tsai - Setting CDOs Up For Success (Spotify)

Bill Inmon - History Lessons of the Data Industry. This is a real treat and a very rare conversation with the godfather himself (Spotify) - PINNED HERE.

Events

2024

Data Day Texas (Austin), end of January - Register here

Data Modeling Zone (Arizona), end of February - Register here

Skiers in Data (Switzerland), March 1-3 - Register here

Saudi Arabia - March 4-7, TBA

London - May, TBA

Malaga, Spain - May, TBA

Berlin, Germany - May, TBA

Morocco - May, TBA

Vancouver, BC - June, TBA

South Africa - TBA

Dubai - TBA

Australia - TBA

Asia - TBA

Thanks! If you want to help out…

Thanks for supporting my content. If you aren’t a subscriber, please consider subscribing to this Substack.

You can also find me here:

Monday Morning Data Chat (YouTube / Spotify and wherever you get your podcasts). Matt Housely and I interview the top people in the field. Live and unscripted. Zero shilling tolerated.

The Joe Reis Show (Spotify and wherever you get your podcasts). My other show. I interview guests, and it’s unscripted with no shilling.

Fundamentals of Data Engineering (Amazon, O’Reilly, and wherever you get your books)

Be sure to leave a lovely review if you like the content.

Thanks! - Joe Reis

1. I have never worked at a big company. Barring any unforeseen life changes or major offers, I don’t plan to work at a big company. I simply think I’ll be too bored with the slow pace and frustrated with the politics.

2. The transitive property still applies, even in the world of AI overlords.

3. Look at OpenAI’s Game of Thrones drama for an example.