Note - this article covers my perspectives on data engineering as a discipline. I might write a separate article covering my thoughts on the tech and tooling landscape. It's also quite a bit of work, and since I'm in the middle of writing a book, so I'll only write this if there's sufficient interest. If this interests you, give a reaction or drop a comment reply. Thanks.
I would add security and compliance to the Enterprisey list of data things. I agree that this and your other listed boring aspects of data will be a trend in 2024.
You said “FinOps shouldn’t just be about cost control and leveraging data to increase revenues.” So what should it be about? 😂 I’ve been getting into trading and looking to build out some retail trader FinOps tools, but no real perspective on the landscape. Any insights you have are always welcome. Especially if it can help me find my next job 💪🏻
Might have been a small yet confusing typo on my part. Should read something like finops is not only about cost control, but also revenue. I’ll correct. Thanks
I thoroughly enjoyed your insights in this post, especially your emphasis on the continuing abstraction of data engineering tools. It resonates deeply with our goals at DLT to enhance efficiency and streamline workflows. Just as you've highlighted the importance of reducing low-level infrastructural work, we've taken a similar approach in addressing schema evolution in our work. For those interested, we delve into this solution in detail in our own blog post on automating data engineers, which you can find here:
Note - this article covers my perspectives on data engineering as a discipline. I might write a separate article covering my thoughts on the tech and tooling landscape. It's also quite a bit of work, and since I'm in the middle of writing a book, so I'll only write this if there's sufficient interest. If this interests you, give a reaction or drop a comment reply. Thanks.
I would add security and compliance to the Enterprisey list of data things. I agree that this and your other listed boring aspects of data will be a trend in 2024.
Oh, definitely security and compliance!
You said “FinOps shouldn’t just be about cost control and leveraging data to increase revenues.” So what should it be about? 😂 I’ve been getting into trading and looking to build out some retail trader FinOps tools, but no real perspective on the landscape. Any insights you have are always welcome. Especially if it can help me find my next job 💪🏻
O'Reilly has a really good book called Cloud FinOps.
Might have been a small yet confusing typo on my part. Should read something like finops is not only about cost control, but also revenue. I’ll correct. Thanks
I thoroughly enjoyed your insights in this post, especially your emphasis on the continuing abstraction of data engineering tools. It resonates deeply with our goals at DLT to enhance efficiency and streamline workflows. Just as you've highlighted the importance of reducing low-level infrastructural work, we've taken a similar approach in addressing schema evolution in our work. For those interested, we delve into this solution in detail in our own blog post on automating data engineers, which you can find here:
https://dlthub.com/blog/automating-data-engineers
Keep up the great work, and I look forward to more of your insightful observations! 🙌
Aman Gupta,
DLT Team