Jobgether
QA Automation Engineer – Enterprise Data & AI
TLDR
Hands-on data quality automation on Databricks, validating enterprise-scale data pipelines and integrating tests into CI/CD.
Accountabilities:
- Execute and extend automated data validation tests within Databricks using Python, PySpark, SQL, and notebook-based frameworks.
- Validate end-to-end data pipelines, including ingestion, batch and incremental loads, transformations, joins, and business rule accuracy.
- Perform data reconciliation between source systems and target datasets to ensure completeness and consistency.
- Enhance and maintain existing data quality frameworks, including rule sets for accuracy, completeness, and reliability.
- Implement and monitor validation checks, thresholds, alerts, and exception handling mechanisms.
- Develop reusable and scalable automated test scripts aligned with enterprise data testing standards.
- Integrate automated tests into CI/CD pipelines (e.g., Azure DevOps) and ensure reliable execution across environments.
- Support testing activities across QA and staging environments, including defect triage and root cause analysis.
- Collaborate with data engineering and analytics teams to ensure data integrity for reporting and visualization tools such as Tableau.
- 5+ years of experience in QA automation, SDET, or data validation engineering roles.
- Strong hands-on experience with Databricks, including notebook development and data pipeline validation.
- Advanced proficiency in Python, PySpark, SQL, and data processing workflows.
- Proven experience in data reconciliation and large-scale data validation across enterprise systems.
- Experience building, extending, or maintaining data quality frameworks in complex environments.
- Familiarity with CI/CD pipelines such as Azure DevOps for test integration and execution.
- Strong analytical, debugging, and problem-solving skills with attention to detail.
- Ability to collaborate effectively with data engineers, QA teams, and cross-functional stakeholders.
- Experience with tools such as Azure Purview or Profisee MDM is a plus.
- Competitive compensation aligned with experience and expertise.
- Fully remote opportunity within the United States.
- Exposure to enterprise-scale data platforms and modern cloud data engineering practices.
- Opportunity to work with Databricks and advanced data quality frameworks.
- Collaborative engineering culture focused on data innovation and automation.
- Career growth in enterprise data engineering, QA automation, and AI-driven data ecosystems.
- Inclusive and flexible work environment supporting work-life balance.
Requirements:
Benefits:
How Jobgether works:
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1
Benefits
Flexible Work Hours
Inclusive and flexible work environment supporting work-life balance.
Learning Budget
Exposure to enterprise-scale data platforms and modern cloud data engineering practices.
Career development opportunities
Career growth in enterprise data engineering, QA automation, and AI-driven data ecosystems.
Remote-Friendly
Fully remote opportunity within the United States.
Jobgether runs the largest remote job platform, effectively linking job seekers with over 200,000 flexible and remote opportunities that match their unique skills and preferences. Our focus is on enhancing the hiring process, ensuring efficiency while prioritizing the candidate experience, particularly in the growing health and wellness sector.
- Founded
- Founded 2020
- Employees
- 11-50 employees
- Industry
- Professional Services
Quality Assurance (QA) Automation Engineer