Technology Lead | Energy Oil & Gas - Midstream | ETRM - Products

Description: Job Title: Technology Lead | Energy Oil & Gas - Midstream | ETRM - Products

Work Location: Houston, TX 77079

Vendor Rate: ***/hr

Contract duration: 6months

Target Start Date: 15 Jun 2026

Job Details:

Must Have Skills

We need 1 ADF Data Engineer at Houston USA location with below JD -

Experience: 5–8 years in data engineering with at least 3+ years on Azure.

Azure Data Factory: Pipelines, data flows (mapping/wrangling), triggers, integration runtimes, parameterization, error handling & retries.

Azure Databricks: Databricks Notebooks, Jobs/Workflows, PySpark/Scala, *** Lake, cluster configuration and optimization.

Programming: Advanced Python or Scala (at least one), strong coding standards, modularization, and unit testing (e.g., pytest).

SQL: Expert-level SQL (CTEs, window functions, query tuning), working with large datasets.

Data Storage & Formats: ADLS Gen2, *** Lake, Parquet, partitioning, Z-Ordering, OPTIMIZE/VACUUM.

Ops & CI/CD: Git, branching strategies, CI/CD for ADF/Databricks; environment promotion; Infrastructure-as-Code exposure is a plus.

Minimum Years Of Experience

8-10 years

Top 3 responsibilities you would expect the Subcon to shoulder and execute

Coding and Development

UI testing

Stakeholder management

Custom Fields:

Name: Intake Call Completed

Value: true

Name: Intake Call Requested

Value: true