by Data pipeline interviews test whether you can move data from a source system to something a dashboard or a model can use, without quietly corrupting it along the way. Interviewers care less about which orchestration tool you name and more about how you handle the failure modes: a source that changes its schema without warning, a job that runs twice, data that arrives 3 hours late.
Most of these interviews are for data engineer, analytics engineer, or ETL developer roles at companies running their own warehouse or lake. Expect a mix of design questions, tool-specific questions about whatever the team already runs (often Airflow and a cloud warehouse), and at least 1 question about a pipeline that broke.
Data pipeline role at a glance
| Item | Details |
|---|---|
| Typical employers | Companies with a data warehouse or lake: SaaS companies, retailers, financial services, any employer with an analytics or data science team |
| Median pay | $139,500 median annual pay for database architects in May 2025, the closest BLS category (BLS doesn't track "data pipeline engineer" pay separately) |
| Job outlook | Database architect employment projected to grow 9% from 2025 to 2035, much faster than average (BLS) |
| Education | Bachelor's degree, usually computer science, data engineering, or a related field |
| Key tools | Airflow or Dagster for orchestration, Kafka or a cloud pub/sub for streaming, dbt for transformation, a warehouse such as Snowflake, BigQuery, or Redshift, Python and SQL |
| Interview format | Recruiter screen, then a technical round covering pipeline design and a specific tool, sometimes a take-home or live SQL and Python exercise |
How the interview usually works
- Recruiter screen. Confirms which tools you've used and at what data volume.
- Technical or design round. You design a pipeline for a given scenario or walk through one you've built.
- Live coding or take-home. Often a SQL transformation exercise or a small Python script that reads, cleans, and loads a dataset.
- Behavioral round. How you've handled failures, disagreements, and changing requirements.
- Team or manager round. Fit and expectations for on-call or pipeline ownership.
General and background questions
1. What's your experience building and running data pipelines in production?
Why they ask: They want a real system with real data volume, not a class project.
How to answer: Name the scale, the tools, and how long you've owned it in production.
Sample answer: I've built and owned pipelines at a mid-size e-commerce company for about 3 years, moving roughly 40 million events a day from our application database and a third-party ad platform into Snowflake. I use Airflow for orchestration and dbt for the transformation layer, and I've been the on-call owner for that pipeline set since I joined the team.
2. Walk us through a pipeline you built end to end, from source to consumption.
Why they ask: This shows whether you can reason through a full system, not just 1 piece of it.
How to answer: Cover the source, the transformation steps, the destination, and who consumes the output.
Sample answer: We pulled order and inventory data from a Postgres application database using change data capture through Debezium, landed the raw events in S3, then used Airflow to trigger a dbt run that modeled the data into fact and dimension tables in Snowflake. The finance team queried those tables directly for daily revenue reporting, and a separate job fed a subset into a forecasting model. The whole pipeline ran on a 15-minute schedule, with a separate daily job for full historical reconciliation.
3. What's the difference between a batch pipeline and a streaming pipeline, and how do you decide which one to build?
Why they ask: This checks whether you'd default to streaming because it sounds impressive, instead of matching the design to the actual requirement.
How to answer: Define both plainly, then give the real factor that drives your choice.
Sample answer: Batch processes data on a schedule, hourly or daily; streaming processes each event close to when it happens. I default to batch unless there's a specific reason not to, since it's simpler to build, test, and debug. We moved 1 pipeline from a daily batch to Kafka streaming only when the fraud team needed to flag a suspicious transaction within seconds instead of finding it in the next day's report. For most reporting use cases, a 15-minute or hourly batch is plenty and a lot less to maintain.
Technical questions
4. What's the difference between ETL and ELT, and when would you pick one over the other?
Why they ask: This is foundational vocabulary, and how you answer shows whether you understand the reasoning, not just the acronyms.
How to answer: Define both, then name a concrete case for each.
Sample answer: ETL transforms data before loading it into the destination; ELT loads raw data first and transforms it inside the warehouse using something like dbt. I default to ELT now, since modern warehouses like Snowflake and BigQuery have the compute to handle transformation cheaply, and keeping raw data around means I can rebuild a model if the transformation logic had a bug, without re-extracting from the source. I still use ETL-style pre-processing when a source system charges per API call and I need to filter or aggregate before it ever leaves the source.
5. How do you design a DAG in Airflow so that a failure in 1 task doesn't corrupt downstream data?
Why they ask: This checks whether you understand task isolation and dependency design, not just how to write a DAG file.
How to answer: Describe task boundaries, retries, and how downstream tasks depend on success signals.
Sample answer: I break a DAG into small tasks with clear boundaries, extract, load, then transform, rather than 1 giant task, so a failure in the load step doesn't leave a downstream transform reading half-loaded data. Each task writes to a staging location first, and the next task only reads from staging after Airflow marks the prior task successful. I also set retries with exponential backoff on tasks that call external APIs, since a lot of "failures" are just a source system timing out once.
6. What does it mean for a pipeline to be idempotent, and how do you build that in?
Why they ask: Pipelines get rerun, retried, and backfilled constantly, and a non-idempotent pipeline creates duplicate or wrong data every time that happens.
How to answer: Define idempotency precisely, then describe a real technique you use.
Sample answer: An idempotent pipeline produces the same result no matter how many times you run it for the same input window. I build this in by having each load use a delete-then-insert or merge pattern keyed on a business key and a date partition, instead of a plain append. When an Airflow task retries after a partial failure, it overwrites the same partition rather than adding a second copy of the same day's data. I learned this the hard way after a retry once double-counted a full day of revenue before anyone caught it.
7. How do you handle backfills without doubling data or overloading a source system?
Why they ask: Backfills are common after a bug fix or a new requirement, and doing them carelessly either corrupts data or takes down a source system.
How to answer: Describe your process for scoping and running a backfill safely.
Sample answer: I run backfills through the same idempotent load logic as the regular pipeline, just triggered for a specific date range, so reprocessing a partition overwrites rather than duplicates it. For a source system with rate limits, I throttle the backfill to run outside business hours and cap concurrent requests, since a backfill pulling 6 months of history at full speed once got our API key rate-limited and took down the regular hourly job for 2 hours.
8. How do you decide between Kafka and a simpler queue or scheduled batch job for a given use case?
Why they ask: Kafka adds real operational cost, and this checks whether you'd reach for it by default or only when the use case actually needs it.
How to answer: Name the specific requirement that justifies Kafka's complexity.
Sample answer: I reach for Kafka when multiple independent consumers need the same event stream, or when the volume and latency requirements rule out batch, high volume, near real time, more than 1 consumer. For a single consumer with no low-latency requirement, a scheduled job reading from a database or an S3 bucket is simpler to run and debug. We used Kafka for clickstream events feeding both a real-time personalization service and a separate analytics pipeline from the same stream, which would have meant 2 separate polling jobs hitting the source otherwise.
9. Walk through how you'd design a Kafka consumer that can't tolerate duplicate processing.
Why they ask: Kafka gives at-least-once delivery by default, so exactly-once behavior has to be designed in deliberately.
How to answer: Name the actual mechanism: idempotent producers, consumer offsets, and a downstream dedup key.
Sample answer: Since Kafka 3.0, idempotent producers are on by default, which prevents duplicate writes from producer retries, but a consumer can still process the same message twice after a rebalance or a crash before committing an offset. I handle that by having the consumer write to a destination with a unique key, like an order ID plus event type, so a duplicate write is an overwrite rather than a second row. For strict cases, I commit the offset only after the downstream write succeeds, and I've paired that with a database unique constraint as a backstop.
10. What data quality checks do you build into a pipeline, and where do you put them?
Why they ask: Bad data that reaches a dashboard erodes trust fast, and this checks whether checks are a real part of your design or an afterthought.
How to answer: Name specific checks and where in the pipeline they run.
Sample answer: I run row-count checks and null checks on required fields right after extraction, before any transformation runs on top of bad data. After the transform step, I add checks for referential integrity, every order should have a valid customer ID, and a freshness check that fails the pipeline if the newest record is older than expected. I use dbt tests for most of this, since failures show up in the same run logs the team already checks, instead of a separate monitoring tool nobody looks at.
11. How do you handle a schema change from an upstream source without breaking downstream consumers?
Why they ask: Upstream teams change schemas without warning more often than anyone would like, and this checks your process for absorbing that safely.
How to answer: Describe schema detection, a compatibility approach, and how you communicate the change.
Sample answer: I use schema-on-read for the raw landing layer, so a new column from the source doesn't break ingestion, then handle mapping explicitly in the transformation layer where I control it. For a breaking change, a column removed or a type changed, I set up a check that fails the pipeline loudly instead of silently dropping or miscasting data. When a product team once renamed a field without telling us, the pipeline failed within the hour instead of quietly loading nulls into a revenue column for 3 days.
12. How do you monitor a pipeline so you find out about a failure before a stakeholder does?
Why they ask: A pipeline that fails silently until someone notices a wrong number is a common and preventable failure.
How to answer: Name the actual alerting setup and what triggers it.
Sample answer: I set Airflow task failure alerts to a Slack channel the team actually watches, plus a separate freshness check that runs on a schedule and pages if a table hasn't updated within its expected window, since a task can succeed on Airflow's terms while still producing wrong output. After a pipeline once failed on a Friday night and nobody noticed until Monday's stand-up, I added the freshness check specifically to catch that kind of silent failure over a weekend.
13. How do you partition data in a warehouse or data lake, and how does that choice affect query cost?
Why they ask: Partitioning strategy has a direct, visible effect on both query speed and cloud spend.
How to answer: Name the partition key you'd choose and why, tied to how the data actually gets queried.
Sample answer: I partition by the column most queries filter on, usually event date, so a query for a specific day only scans that day's partition instead of the entire table. On a 2-year table with about 500 million rows, switching from an unpartitioned table to date partitioning in BigQuery cut a common dashboard query's cost from scanning the full table to scanning a single day, which dropped that query's cost by more than 90%.
14. How do you deduplicate records that arrive more than once from an upstream source?
Why they ask: Duplicate records are common with retries, replays, and at-least-once delivery, and unhandled duplicates quietly inflate metrics.
How to answer: Name a specific deduplication key and technique.
Sample answer: I deduplicate on a natural key plus an event timestamp, keeping the latest record per key using a window function in the transformation layer, rather than trusting the source to send each event exactly once. For a webhook-based integration that retried on any non-200 response, we were getting the same event up to 3 times during a source outage. Deduplicating on the source's event ID rather than our own ingestion timestamp fixed it completely.
15. Walk through how you'd design a pipeline to handle late-arriving data.
Why they ask: Real-world data doesn't always arrive in order, and a pipeline that assumes it will produces quietly incomplete results.
How to answer: Describe a specific technique: a reprocessing window or a watermark.
Sample answer: I keep a rolling reprocessing window, typically the last 3 days, that reruns even after the initial load, so data that arrives a day or 2 late still gets picked up and merged in through the same idempotent load logic. For a stricter case in a streaming pipeline, I've used a watermark that holds a window open for a defined lateness tolerance before finalizing it, then routes anything later than that into a separate late-arrival table for manual review instead of silently dropping it.
Behavioral questions
16. Tell us about a pipeline failure that caused bad data to reach a dashboard or report. What did you change?
Why they ask: This checks whether you own the failure and fix the underlying gap, not just patch the 1 incident.
How to answer: Name the specific cause and the concrete change you made afterward.
Sample answer: A currency conversion job used a stale exchange rate for 2 days because the rate provider's API had silently started returning the previous day's rate during an outage on their end. Revenue numbers in 1 international market were off by about 4% before finance caught it. I added a check that compares the fetched rate against the prior day's rate and fails the job if the difference exceeds a set threshold, instead of trusting whatever the API returned.
17. Describe a time you disagreed with a stakeholder about a data pipeline's design or priority.
Why they ask: Engineers and stakeholders often see urgency and risk differently, and this checks how you handle that without just capitulating or digging in.
How to answer: Describe the disagreement and how it actually got resolved.
Sample answer: A product manager wanted a new pipeline shipped in a week without data quality checks, arguing they could be added later. I pushed back because the pipeline fed a customer-facing usage report, and I'd seen what happens when bad data reaches customers directly. We compromised on shipping with just 2 checks, a row-count check and a null check on the primary key, in the first week, with the rest added in the following sprint. That caught a real bug in week 2 that would have shown wrong usage numbers to customers.
18. Tell us about a time you had to redesign a pipeline that had outgrown its original design.
Why they ask: Pipelines that worked at a smaller scale often break down as volume grows, and this checks whether you can recognize and fix that.
How to answer: Name what stopped working and the specific redesign.
Sample answer: A pipeline built as a single large Python script had run fine for a year at low volume, but once event volume tripled it started missing its schedule window and occasionally crashed mid-run with no way to resume partway through. I rebuilt it as an Airflow DAG with small, idempotent tasks that could retry individually, and moved the heaviest transformation step into the warehouse using dbt instead of pulling data into Python memory. The rebuilt version ran in about a quarter of the time and could resume from any failed task instead of restarting from scratch.
Situational questions
19. A critical pipeline breaks at 2 a.m. and a report is due at 8 a.m. What do you do?
Why they ask: This checks triage under time pressure: fix the immediate problem versus diagnosing the root cause first.
How to answer: Prioritize getting a correct report out, then explain the follow-up.
Sample answer: First I'd check whether the failure is upstream (source outage) or in our own pipeline, since that changes what's fixable in the next few hours. If it's a fixable pipeline bug, I'd patch and rerun the specific failed task rather than the whole DAG, to save time. If I can't fix it before 8 a.m., I'd tell the report's stakeholders directly that the data will be late or partial, with a real time estimate, rather than let them find out when the report doesn't show up. I'd do the root cause analysis after the immediate fire is out, not during it.
20. A stakeholder asks you to skip data quality checks to ship a pipeline faster. How do you handle it?
Why they ask: This tests whether you can hold a technical standard under deadline pressure without just refusing outright.
How to answer: Offer a real, scoped-down compromise instead of an absolute yes or no.
Sample answer: I'd ask what the deadline is actually protecting and offer the minimum checks I'm not willing to ship without, usually a row-count check and a check on whatever field the business decision depends on most. On 1 project, that meant shipping with 2 checks instead of the full 8 I'd planned, with the rest added within the following week. That kept the deadline while still catching the kind of error that would have been expensive to walk back publicly.
Questions to ask the interviewer
- What does the current pipeline stack look like, and what's driving a change or a new hire?
- How is on-call handled for pipeline failures, and how often does it actually page someone?
- What's the biggest source of bad data the team deals with today?
- How much of this role is building new pipelines versus maintaining existing ones?
- How does the team decide between batch and streaming for a new use case?
- What does a typical backfill or schema migration look like here?
How to prepare
- Have 2 or 3 pipelines ready to describe in depth. Know the actual volume, tools, and 1 failure you handled.
- Review the difference between ETL and ELT and be ready to defend a choice. Interviewers often ask you to pick one for a given scenario.
- Practice a SQL exercise involving window functions and deduplication. These come up constantly in take-homes and live coding.
- Know 1 real number. A row count, a percentage improvement, or a time saved from a project you actually worked on.
- Read the job posting for the specific tools. Airflow, Dagster, and cloud-native orchestration each have different vocabulary for the same concepts.
Data pipeline roles overlap heavily with data architecture and cloud infrastructure work, so data architect interview questions and cloud architect interview questions are worth reviewing too. If you're weighing this path against a more operations-focused one, data engineer vs. DevOps engineer breaks down how the 2 roles differ.
