The Elder Scripts

Latest — Mar 17, 2026

How to hire a data engineer in 2026

I hope you're here because the previous post convinced you that you definitely need data engineers in your organization. Without another witty but ultimately useless intro, let's move right to the practical advice. Why the Old Playbook Is History If you've tried to hire

More issues

Why you need a data engineer in 2026

Since the words "Big Data" started appearing in presentation slides in 1999, we've heard how important it is to treat your company data with respect and that sort of fearful admiration that little children have for bulldozers and garbage trucks. And just like waste disposal infrastructure,

Baking the context cake

If you ask 10 executives which roles they need to run an AI adoption project in their company, 9 of them won't mention data engineers. The AI development community spent immense effort in the past few years working on models and orchestration methods, but only recently we started

Why LLM API Calls Break in Production

When you build an integration with an LLM provider and test it locally, everything usually looks fine. You send a request, you get a response, and you move on. The problems start when you deploy the same code to a cloud environment. You begin to see errors such as “connection

Grounding LLMs in the Logic of Planning

What makes a plan good? Have you ever thought about this? We humans never stop planning: from stacking toys as children to shipping products at work. A good plan feels intuitive when you see it: it's structured, realistic and flexible enough to adapt when things change. In other

Self-correction in LLM calls: a review

In the fast-evolving world of large language models, building reliable pipelines often feels like wrestling with a brilliant but unpredictable collaborator. From my own experiments shared on X (document generation workflows with structured outputs) I've repeatedly slammed into frustrating roadblocks. For an example: You are prompting an LLM

Improving Structured Outputs from LLMs: A Two-Step Pattern

Large language models can reason about complex input. They can also follow formatting instructions. But when asked to do both at once, they often fail. For example, turning messy data into valid structured output usually leads to errors or oversimplification. This post explains a two-step method that improves reliability by

Hiring manager interview. User manual.

The first interview in the hiring process for software developers is often a conversation with the hiring manager. Candidates tend not to take it seriously because it is usually a free-form conversation, without the scary live-coding task or deep technical questions. It is a mistake, though, and people should not

Explained: data governance vs data management

In this post I will define the concept of data governance from first principles and explain its place in the larger area of data management. Modern companies require an ever-increasing amount of data to stay competitive. It is not uncommon for data lakes to reach the size measured in petabytes.

Vital non-technical skills for an engineer

Recently I have been writing about the responsibilities of a data engineer and sharing my advice on how to succeed in this job. In the first post, I listed the technical skills that data engineers have [https://theelderscripts.com/data-engineering-skills/]. Last week I wrote about the notion of a 10x