Hope you all had a great Thanksgiving (assuming you celebrate it). Keeping my rant short this week, cuz the holiday weekend and book writing.
If you’re at reInvent next week in Vegas, hit me up.
Thanks,
Joe Reis
"This Was All Predictable"
“This was all predictable” are four words that trigger me beyond belief (I’ve got an extensive background in forecasting, which nowadays I think is mostly as credible as phrenology or astrology). Pundits with crystal balls were among the most annoying souvenirs of the OpenAI debacle. Not gonna name names, but I saw quite a few pundits and influencers continuously making confident play-by-play predictions about “what’s next,” only to be wrong. They continued anyway. Overconfidence and persistence are worthy in some situations, but not when you look dumb. But hey, they got “engagement," which seems to be all that matters these days.
If you predicted the exact outcome of the events of OpenAI last week, please make this public. Could you prove you knew every tiny twist and turn as it happened, second by second? Otherwise (at least to me), you’re a charlatan and bullshit artist.
“I don’t know” are often the best three words you can say in situations of major uncertainty. Because you often don’t know. Simple as that.
Listen to the audio clip above on this topic, which is also my 5-Minute Friday on Spotify.
Cool Weekend Reads
Here are some cool things I read this week. Enjoy!
Tech, AI & Data
(From October 2023)
I’ve been brooding over this article from Ben Evans for a bit. He’s got some excellent points I keep coming back to.
Regarding LLMs - “I think it brings us to two new problems - a science problem and a product problem. You can ask anything and the system will try to answer, but it might be wrong; and, even if it answers correctly, an answer might not be the right way to achieve your aim. That might be the bigger problem.”
In the broader context of what I wrote last week about LLMs and data management (among other things), the science problem is a huge deal. If you use an LLM to generate a data pipeline or solve a data governance question, who the hell knows if the output is consistent? It’s like dealing with an employee (at best, an intern) who does mundane work and oscillates between decent and poor performance on an hourly basis. Then you’ve got the underlying data, which is often a mess in most organizations. That said, there’s enough interest and progress in LLMs that I feel mildly comfortable believing that LLMs will help to scale certain types of automated data management solutions.
Margaret Mead, Technocracy, and the origins of AI's ideological divide (Res Obscura)
“Technocracy, it said, called for “a nation eventually to be controlled by engineers.” The goal: automating industry so that the average person would need to work only a dozen or so hours a week.
“Nobody can bring this about but engineers,” Scott was quoted as saying. “So far they have been very successful in doing away with material waste and duplication of energy in the various industries in which they are employed. But they have not thought of American industry as an industry in itself. The time has come when that must be done. We are nearing a crisis.”
Techno-optimism is nothing new. Good read about the Technocracy movement in the 20th century, which is relevant to today’s debates about AI.
AI: The Coming Revolution (Coatue)
Interesting prognostications from Coatue, a leading VC. I also like to read VC prognostications because you get an idea of what the investment community (i.e., herd) is paying attention to.
The Changing “Guarantees” Given by Python's Global Interpreter Lock (Stefan Marr)
“However, since the Python community is taking steps that may lead to the removal of the GIL, the changes in recent Python versions to give much stronger atomicity “guarantees” are likely a step in the wrong direction for the correctness of concurrent Python code. I mean this in the sense of people to accidentally rely on these implementation details, leading to hard-to-find concurrency bugs when running on a no-GIL Python.”
This is a detailed look at the impacts and tradeoffs of the “GILectomy” coming soon in Python.
Business & Startups
Confessions of a Middle-Class Founder (NYMag Intelligencer)
“Every founder tells themselves a story about why they’re heading to the gold rush, but the executive coach I would eventually hire says there are really only two. Do you want to be rich, generating wealth in service of some further end? Or do you want to be king, with money a mere byproduct of trying to make the world the way you feel it should be?
At the time, I told myself I wanted the freedom of being rich. I thought I’d be able to recognize a winning hand fast or fold. Now, several years later, I’m still waiting for the river card that determines my fate. You could call me a middle-class founder: proprietor of a business you may or may not have heard of, tenuously wealthy on paper, by no means a failure but not yet a success, chugging along in the twilight of an era that minted more giants and more waste than any other in history, with no exit in sight.”
Welcome the reality of running a business. I’m no stranger to startups, and I’ve got little desire to do it again (I could be convinced, given the right opportunity). Even if you’re a startup/scale-up, many things can happen beyond the regular scope of running a business, such as a funding environment that changes overnight, VCs constantly shifting their focus and demands, etc. I’d rather focus on running an actual business that does weird things like serving customers who pay money and making a profit.
A Coder Considers the Waning Days of the Craft (The New Yorker)
I love to code because it’s a pure form of problem-solving - your solution either works or doesn’t. What happens when we outsource coding to AI? This is a really good read about one programmer’s opinion on what this means.
Companies Hesitate on Specialized Industry Clouds (WSJ)
Having partnered with all major clouds and big vendors, I’ve seen this firsthand. Engineering teams want to engineer custom solutions, especially where they provide a competitive advantage. Specialized industry clouds provide an almost good enough solution, but then you have to shoe-horn into their way of doing things. For now, it’s generally easier to use the off-the-shelf building blocks (servers/serverless, object storage, etc) and roll your solution.
New Content, Events, and Upcoming Stuff
Monday Morning Data Chat
Coming up…
Sarah Nagy, Tristan Handy, Mike Ferguson, and more…
In case you missed it…
Dave McComb - Knowledge Graphs, Semantics, and More (Spotify, YouTube)
EU AI Act w/ Kai Zenner (Spotify, YouTube)
Apache Hudi Deep Dive w/ Nadine Farah (Spotify, YouTube)
Why is Data Security So Hard? w/ Yoav Cohen (Spotify, YouTube)
The Joe Reis Show
Coming up…
Karin Wolok, Matt Harrison, Ben Rogojan, and more…
This week…
5 Minute Friday - “This Was All Predictable” (Spotify)
In case you missed it…
Peggy Tsai - Setting CDOs Up For Success (Spotify)
5 Minute Friday - Things I Didn’t Expect, AI & Data Management, and More (Spotify)
5 Minute Friday - The Biden AI Executive Order w/ Katharine Jarmul (Spotify)
Bill Inmon - History Lessons of the Data Industry. This is a real treat and a very rare conversation with the godfather himself (Spotify) - PINNED HERE.
Events
November
Las Vegas - ReInvent, 11/28 to 11/30
December
No speaking events. Taking the month off :)
2024
dbt + Joe Reis Roadshow (Dallas) - TBA
Data Day Texas (Austin) - register here
Data Modeling Zone (Arizona) - register here
Skiers in Data (Switzerland) - March, TBA
London - May, TBA
Malaga, Spain - May, TBA
Berlin, Germany - May, TBA
Morocco - May, TBA
Vancouver, BC - June, TBA
South Africa - TBA
Dubai - TBA
Australia - TBA
Asia - TBA
Thanks! If you want to help out…
Thanks for supporting my content. If you aren’t a subscriber, please consider subscribing to this Substack.
You can also find me here:
Monday Morning Data Chat (YouTube / Spotify and wherever you get your podcasts). Matt Housely and I interview the top people in the field. Live and unscripted. Zero shilling tolerated.
The Joe Reis Show (Spotify and wherever you get your podcasts). My other show. I interview guests, and it’s unscripted with no shilling.
Fundamentals of Data Engineering (Amazon, O’Reilly, and wherever you get your books)
Be sure to leave a nice review if you like the content.
Thanks! - Joe Reis
It was all predictable that you would write this - lol :)
(I feel the same way about people who confuse hindsight bias with foresight)
Great stuff Joe. Keep it coming. I love the concept of an industry specific cloud. The industry specific cloud knows, the workflow, the skill of the users, the file types, and has industry specific AI agents that will get the context right more often than not.