Data Engineer / Lead Data Engineer

External listing

MSR Cosmos

United States•REMOTE•CONTRACT•SENIOR•DEV

ETRM Role Details

Commodity Focus: Power, Gas, Oil

Salary Range

USD 65 – 70 /hr (est.)

Job Description

Position: Data Engineer / Lead Data Engineer

Location: Houston, TX - Hybird - Need Locals Mostly

need someone with Oil & Gas experience recently

Job Description:

We are seeking an experienced Data Engineer to design and build robust data foundations and end-to-end analytics solutions for a complex energy trading environment. The role will create trusted, reusable data products and clear reporting that enable informed operational and commercial decision-making. The successful candidate will lead technical design and delivery across the full data engineering lifecycle, while collaborating with business, product, architecture and engineering stakeholders and providing technical guidance to colleagues.

Key responsibilities:

  • Lead the design and development of scalable data ingestion and transformation pipelines using standard data lake and cloud engineering frameworks.

  • Engineer solutions for structured and unstructured data in high-volume, high-velocity and high-variety environments.

  • Develop transformation notebooks, physical and tabular data models, data endpoints, semantic layers and effective visualisations.

  • Design and deliver reusable data products that improve data accessibility, consistency and business value.

  • Lead ETL/ELT activities across source systems, data platforms and downstream business intelligence solutions.

  • Create and publish dashboards and reports that provide stakeholders with accessible, actionable information.

  • Perform trend and correlation analysis; identify data discrepancies and implement controls that protect data quality across platforms.

  • Define and enforce engineering standards, reusable patterns, documentation and development practices.

  • Build automated continuous integration and release pipelines and apply disciplined source-control practices.

  • Gather functional and non-functional requirements, translate features and Product Backlog Items into technical designs, and establish delivery direction.

  • Collaborate with cross-functional teams in a dynamic business environment and mentor engineers where required.

Mandatory experience and skills:

  • Demonstrable data engineering experience within energy trading, commodities trading or an ETRM/CTRM environment. This is mandatory.

  • Strong hands-on experience designing production-grade ingestion, storage, transformation, modelling and serving layers on Microsoft Azure.

  • Advanced SQL capability, including the development and optimisation of complex queries for data transformation and analysis.

  • Hands-on experience with Azure Data Factory, Azure SQL, Azure Databricks, Azure Blob Storage and Azure Data Lake Storage.

  • Strong ETL/ELT and data warehousing experience, including delivery of business intelligence solutions.

  • Experience with dimensional, physical and tabular data modelling and performance optimisation.

  • Experience with GitHub and/or Azure DevOps, including automated build, test and release pipelines.

  • Ability to gather requirements, shape technical solutions and communicate clearly with technical and business stakeholders.

  • Proven ability to lead workstreams, manage priorities and deliver effectively in a collaborative team environment.

Preferred experience

  • Experience with dbt, Azure Analysis Services, Azure Table Storage and Azure Logic Apps.

  • Experience with Power BI and the design of clear, decision-oriented dashboards and reports.

  • Exposure to Power Apps and broader Microsoft Power Platform capabilities.

  • Familiarity with IBM DataStage or comparable enterprise integration tooling.

  • Basic understanding of AI engineering, including how governed data products support analytics and AI use cases.

  • Knowledge of data governance, lineage, metadata management, observability and automated data-quality controls.

Pay: $65.00 - $70.00 per hour

Work Location: Hybrid remote in Houston, TX 77015