On ownership and emotional resilience

Is AI taking these away?

This week, I've been thinking about how my ways of working as a data engineer have changed. So much has changed for me over the past few years. AI tools have become more capable, and I’ve become more engaged in data communities.

A lot of my progression has come from being inspired by others online who share technical insights and their approach to work. Data engineering is at its core about solving problems, but it's also about the mindset of ownership, resilience, and strong decision making that helps solve bigger problems.

Here are some of my reflections on these skills.

TL;DR

• Trustworthy data engineers take ownership of their work

• Emotional resilience is built by doing the hard work

• Over reliance on AI risks undermining these skills

Ownership

A key trait of trustworthy data engineers is their ability to take ownership of what they build. It’s not enough to just get a pipeline up and running through the environments. It needs to remain performant and reliable over time. This is where a checkbox task translates to a mindset of ownership.

In practice it means asking myself:

• Will this still work in six months?

• Will someone else understand it easily by reading the README?

• Can we recover quickly if it breaks?

• Who is the main point of contact if things go bad?

Having processes in place to fix issues before they escalate starts by having clear and maintainable documentation. These are some of the proactive maintenance actions to ensure ownership even after handing work to a support team once development is complete.

Thinking of data as a product helps reinforce this mindset. If a pipeline runs but produces unreliable data, it’s not a successful product. Success comes from focusing on usability, reliability, and maintainability so that the data produced remains valuable.

Anyone maintaining the product should be able to take steps to rectify issues or reach out to someone who can help. This will also impact my reputation as I am more likely to be seen as someone who is reliable and can deliver value, not just someone with good technical skills.

Emotional resilience

Ownership isn’t just about technical decisions. It also requires emotional resilience. Debugging pipeline failures can be frustrating, but this is where I am putting in the reps. I’ve been reflecting on how using AI more is taking this away.

I’ve spent a lot of time optimising my IDE to move fast and iterate quicker and I recently started using AI code completions in my IDE. This was saving me some time from having to copy and paste into ChatGPT, but now I’ve noticed I'm relying more on AI suggestions rather than thinking about things thoroughly.

Maybe I need to enforce a rule to only turn to AI after making some attempts to solve the issue myself. I don't want to lose out on the resilience that comes from working through tough problems. If I start skipping the debugging process and jump straight to AI generated suggestions, I fear that I am weakening the very skill set that makes me effective.

This can then impact how I approach development. I enjoy working on greenfield projects as it is both an opportunity and a challenge. I get to use my experience to do things better, thinking about what is important from the beginning for scalability and maintainability. Greenfield development requires independent thinking. If I become too dependent on AI to generate solutions, I risk losing the ability to reason through foundational design decisions. AI might help fill in gaps, but it doesn’t understand business context the way I would.

Without strong decision-making, a blank slate can quickly turn into a tangled mess of short-term solutions that create long-term problems. That’s why the ability to think critically, take ownership, and troubleshoot problems is so important as a data engineer, and I want to be wary of things that might be detrimental to developing these skills.

Thanks for reading,
Elias