Data Engineer – Mid & Senior Level

This is a remote position.

Role Overview

We are looking for Data Engineers at Senior and Mid-Level to join our team in building a privacy-preserving data platform where data engineering meets production-grade software engineering.

You will work on designing, developing, and maintaining reliable data pipelines that transform operational data into high-quality, secure, and AI-ready datasets.

Key Responsibilities

• Build and maintain production-grade data pipelines on AWS.
• Extract, transform, validate, and curate large-scale Parquet datasets.
• Implement data de-identification, masking, and privacy-preserving transformations.
• Design and maintain data pipeline orchestration, scheduling, retries, and backfill mechanisms.
• Implement comprehensive data quality checks, monitoring, and alerting.
• Work with workflow orchestration tools such as Airflow, Dagster, or AWS Step Functions.
• Contribute to CI/CD pipelines and Infrastructure as Code (IaC) practices.
• Manage schema evolution and schema drift across data sources and pipelines.
• Provide production support, troubleshooting, and root cause analysis for data pipeline issues.
• Maintain data catalogs, metadata, and data lineage.
• Follow software engineering best practices including Git, code reviews, automated testing, and maintainable code.
• Build reliable and idempotent data pipelines capable of handling retries and large-scale backfills.

Required Skills & Experience

• Strong proficiency in Python and SQL.
• Hands-on experience with AWS data services and production data pipelines.
• Experience with Apache Spark or equivalent distributed data processing technologies.
• Practical experience with Airflow, Dagster, AWS Step Functions, or similar orchestration tools.
• Strong understanding of data pipeline architecture, ETL/ELT, and data transformation.
• Experience working with Parquet and large-scale datasets.
• Understanding of data quality, schema management, monitoring, and alerting.

• Strong software engineering practices including:

  • Git and version control
  • Code reviews
  • Automated testing
  • Idempotency
  • Error handling
  • Retries and backfills
• Experience supporting and troubleshooting production data pipelines.
• Ability to work effectively with cross-functional engineering and data teams.

Requirements

Required Skills

Python | SQL | AWS | Data Engineering | Data Pipelines | ETL/ELT | Apache Spark | Parquet | Airflow | Dagster | AWS Step Functions | Data Orchestration | Data Quality | Schema Management | Data Transformation | Production Support | Git | CI/CD | Automated Testing | Data De-identification | Data Lineage | Data Catalog

Good to Have

Debezium | AWS DMS | Apache Iceberg | Delta Lake | Apache Hudi | Data Masking | Data Tokenization | Terraform | CloudFormation | ML/AI Training Data | Privacy-Preserving Data