Top 10 Deloitte Azure Data Factory & Azure Databricks Interview Questions (2026)

Prepare for Deloitte Azure Data Engineer interviews with the most frequently asked Azure Data Factory, Azure Databricks, ADLS Gen2, Delta Lake, PySpark, SQL and ETL interview questions.

✍️ ANUJ SINGH📅 2026-01-21

#Deloitte#Azure Data Engineer#Azure Data Factory#Azure Databricks#ADF#PySpark#SQL#Interview Questions

Top 20 Deloitte Azure Data Engineer Interview Questions

Deloitte Azure Data Engineer interviews focus on Azure Data Factory, Azure Databricks, PySpark, Delta Lake, ADLS Gen2, SQL, data modeling, Spark optimization, and enterprise data pipeline scenarios. Candidates should be able to explain architecture decisions, performance optimization techniques, and real-world implementation approaches.

1. What is Z-ordering in Spark?

Z-ordering is a Delta Lake optimization technique in Databricks that improves query performance by reorganizing data files based on frequently filtered columns. It stores related data closer together, reducing the amount of data Spark needs to scan during queries.

sql

OPTIMIZE my_table
ZORDER BY (customer_id, order_date);

2. Explain the difference between Spark SQL and PySpark DataFrame APIs.

Spark SQL allows developers to query structured data using SQL syntax, while PySpark DataFrame API provides a Python-based programming approach for data transformations. Both use the same Spark execution engine internally, so performance is generally similar.

python

# Spark SQL Example
spark.sql("""
SELECT *
FROM orders
WHERE total > 1000
""")


# PySpark DataFrame Example
orders.filter(
    orders.total > 1000
).show()

3. How do you implement incremental load in Azure Data Factory?

Incremental loading means processing only new or modified records instead of loading the entire dataset. In ADF, this is commonly implemented using watermark columns such as LastModifiedDate, Lookup activities, Stored Procedures, and dynamic queries.

sql

SELECT *
FROM source_table
WHERE LastModifiedDate >
@pipeline().parameters.lastLoadTime;

After successful loading, the watermark value is updated so the next pipeline execution only processes newly changed records.

4. How do you handle large-scale data ingestion into ADLS Gen2?

Large-scale ingestion into ADLS Gen2 requires optimized parallel processing, efficient file formats, and scalable Azure services. Azure Data Factory Copy Activity, Mapping Data Flows, and Azure Databricks can be used depending on transformation complexity.

text

Large Data Ingestion Flow

Source Systems
      |
      ↓
Azure Data Factory
      |
      ↓
ADLS Gen2 Raw Layer
      |
      ↓
Databricks Processing
      |
      ↓
Delta Lake

5. Write Python code to split a name column into first name and last name.

python

import pandas as pd

df = pd.DataFrame({
    'name': ['John Smith', 'Alice Johnson']
})

df[['first_name', 'last_name']] = (
    df['name']
    .str.split(' ', 1, expand=True)
)

print(df)

6. What are fact and dimension tables in data modeling?

Fact tables store measurable business data such as sales, revenue, and quantity. Dimension tables store descriptive information such as customer, product, and location details. Together they form the foundation of star schema data warehouse design.

text

Fact Table
-----------
Sales_Fact
• product_id
• customer_id
• sales_amount
• quantity


Dimension Table
---------------
Customer_Dim
• customer_id
• customer_name
• location

Product_Dim
• product_id
• product_name
• category

7. How do you design and implement data pipelines using Azure Data Factory?

Designing an ADF pipeline involves identifying source systems, creating linked services and datasets, defining activities, applying transformations, loading data into target systems, and monitoring execution. Parameterization and metadata-driven approaches are used to build reusable pipelines.

text

Source Dataset
      |
      ↓
ADF Pipeline
      |
 ┌───────────────┐
 | Copy Activity |
 | Data Flow     |
 | Stored Proc   |
 └───────────────┘
      |
      ↓
Target System
(SQL / ADLS / Synapse)

8. Explain the concept of PolyBase in Azure Synapse Analytics.

PolyBase allows querying external data stored in Azure Blob Storage or ADLS Gen2 directly using SQL without loading the data into database tables first. It is commonly used for large-scale ELT workloads and external table scenarios.

sql

SELECT *
FROM ExternalTable;

9. Write a SQL query to calculate the cumulative sum of a column.

sql

SELECT
    employee_id,
    salary,
    SUM(salary) OVER(
        ORDER BY employee_id
    ) AS cumulative_salary
FROM employees;

Window functions calculate running totals without requiring additional grouping. Partitioning can also be applied to calculate cumulative values department-wise.

10. How do you manage partitioning in PySpark?

Partitioning helps Spark distribute data across executors and improves parallel processing. repartition() is used to increase or rebalance partitions, while coalesce() reduces partitions efficiently before writing output files.

python

# Check number of partitions
df.rdd.getNumPartitions()

# Increase partitions
df = df.repartition(8)

# Reduce partitions
df = df.coalesce(4)

💡

Interview Tip

For Deloitte interviews, explain not only the definition but also where you used the concept in real projects, why you selected a particular approach, and how it improved performance, scalability, or cost.

11. Explain the use of Delta Lake for data versioning.

Delta Lake provides ACID transactions and data versioning capabilities on top of data lakes. Every insert, update, delete, or merge operation creates transaction logs, allowing users to access previous versions of data using time travel.

sql

-- Read previous Delta table version

SELECT *
FROM table_name
VERSION AS OF 5;


-- Read data using timestamp

SELECT *
FROM table_name
TIMESTAMP AS OF '2024-04-01T00:00:00';

Delta versioning is useful for auditing, rollback scenarios, debugging data issues, and maintaining historical records without creating multiple copies of data.

12. How do you monitor and troubleshoot Spark jobs?

Spark jobs can be monitored using Spark UI, cluster metrics, and application logs. Spark UI provides details about stages, tasks, execution plans, shuffle operations, and failed jobs.

text

Spark Job Monitoring

Spark UI
   |
   ├── Stages
   ├── Tasks
   ├── DAG Execution
   ├── Shuffle Read/Write
   └── Execution Time


Common Issues:
• Data skew
• Out of memory errors
• Long garbage collection time
• Slow joins

Performance can be improved by enabling Adaptive Query Execution (AQE), optimizing joins, tuning partitions, and analyzing Spark execution plans.

13. Write a SQL query to find employees with the highest salary in each department.

sql

SELECT *
FROM (
    SELECT *,
    RANK() OVER(
        PARTITION BY department_id
        ORDER BY salary DESC
    ) AS rank
    FROM employees
) ranked
WHERE rank = 1;

The RANK() window function identifies the highest-paid employees in each department and also handles cases where multiple employees have the same highest salary.

14. How do you optimize joins in PySpark for large datasets?

Large joins can cause expensive shuffle operations. PySpark join optimization techniques include broadcast joins, proper partitioning, handling data skew, selecting the correct join type, and caching frequently used datasets.

python

from pyspark.sql.functions import broadcast

# Broadcast small table
result = large_df.join(
    broadcast(small_df),
    "id"
)

result.show()

Broadcast joins are effective when one dataset is small enough to fit into executor memory, reducing network shuffle and improving execution speed.

15. Describe the process of setting up CI/CD for Azure Data Factory.

CI/CD implementation in Azure Data Factory uses Azure DevOps or GitHub Actions to automate deployment between environments such as Development, QA, and Production.

text

Developer
   |
   ↓
Git Repository
   |
   ↓
ADF Development Branch
   |
   ↓
Publish Branch
(adf_publish)
   |
   ↓
CI Pipeline
(Create ARM Template)
   |
   ↓
CD Pipeline
(Deploy to QA/Prod)

Best practices include parameterized linked services, datasets, and pipelines so the same code can be deployed across multiple environments.

16. Write Python code to reverse a string.

python

text = "Hello Deloitte"

reversed_text = text[::-1]

print(reversed_text)

# Output:
# etioleD olleH

Python slicing allows strings to be reversed easily. Another approach is to iterate through characters and build the reversed string manually.

17. What are the key features of Databricks notebooks?

Databricks notebooks provide an interactive environment for data engineering, analytics, and machine learning workloads. They support multiple languages, collaboration, visualization, scheduling, and integration with MLflow.

text

Databricks Notebook Features

• Multi-language support
  (%python, %sql, %scala, %bash)

• Data visualizations

• Job scheduling

• Notebook collaboration

• Widgets for parameters

• MLflow integration

• Role-based access control

18. How do you handle late-arriving data in Azure Data Factory?

Late-arriving data occurs when records arrive after the scheduled pipeline execution. It can be handled using watermarking, reprocessing windows, retry mechanisms, Delta Lake merge operations, and monitoring alerts.

text

Approaches:

1. Watermark Column
   Track last processed timestamp

2. Reprocessing Window
   Reload previous days data

3. Tumbling Window Trigger
   Handle delayed arrivals

4. Delta MERGE
   Update late records

5. Alerts and Monitoring

19. Explain the concept of Data Lakehouse.

A Data Lakehouse combines the flexibility of a data lake with the reliability and performance features of a data warehouse. It supports analytics, reporting, and machine learning workloads on the same platform.

text

Data Lakehouse Architecture

Raw Data
   |
   ↓
Data Lake Storage
   |
   ↓
Delta Lake
   |
   ├── BI Analytics
   ├── Machine Learning
   └── Reporting

Key features include open file formats like Parquet, ACID transactions, schema enforcement, governance, and reduced data duplication.

20. How do you implement disaster recovery for ADLS Gen2?

Disaster recovery for Azure Data Lake Storage focuses on protecting data availability and enabling recovery during failures. This includes redundancy, backup strategies, replication, and recovery testing.

text

ADLS Gen2 Disaster Recovery

1. Geo-Redundant Storage (GRS)
   - Replicates data to another region

2. Snapshots
   - Point-in-time recovery

3. Soft Delete
   - Recover deleted files

4. Versioning
   - Maintain file history

5. Cross-region Replication
   - Copy critical data

6. Backup Solutions
   - Azure Backup / Third-party tools

💡

Interview Tip

Deloitte interviewers often evaluate architecture thinking. Explain scalability, performance optimization, security, monitoring, cost control, and disaster recovery whenever describing Azure data engineering solutions.

Top 10 Deloitte Azure Data Factory & Azure Databricks Interview Questions (2026)

Prepare for Deloitte Azure Data Engineer interviews with the most frequently asked Azure Data Factory, Azure Databricks, ADLS Gen2, Delta Lake, PySpark, SQL and ETL interview questions.

✍️ ANUJ SINGH📅 2026-01-21

#Deloitte#Azure Data Engineer#Azure Data Factory#Azure Databricks#ADF#PySpark#SQL#Interview Questions

Fact Table ----------- Sales_Fact • product_id • customer_id • sales_amount • quantity Dimension Table --------------- Customer_Dim • customer_id • customer_name • location Product_Dim • product_id • product_name • category

Spark Job Monitoring Spark UI | ├── Stages ├── Tasks ├── DAG Execution ├── Shuffle Read/Write └── Execution Time Common Issues: • Data skew • Out of memory errors • Long garbage collection time • Slow joins

Databricks Notebook Features • Multi-language support (%python, %sql, %scala, %bash) • Data visualizations • Job scheduling • Notebook collaboration • Widgets for parameters • MLflow integration • Role-based access control

Approaches: 1. Watermark Column Track last processed timestamp 2. Reprocessing Window Reload previous days data 3. Tumbling Window Trigger Handle delayed arrivals 4. Delta MERGE Update late records 5. Alerts and Monitoring

ADLS Gen2 Disaster Recovery 1. Geo-Redundant Storage (GRS) - Replicates data to another region 2. Snapshots - Point-in-time recovery 3. Soft Delete - Recover deleted files 4. Versioning - Maintain file history 5. Cross-region Replication - Copy critical data 6. Backup Solutions - Azure Backup / Third-party tools