Senior/Lead Data Engineer

This is a position we're hiring on behalf of our client in the digital banking industry.

Description

In this role, you can expect to:

• Design, develop, test, deploy and monitor data pipelines in Databricks on AWS from a wide variety of data sources.
• Design, develop, test, deploy and monitor scalable code with PySpark and SQL in Databricks.
• Identify opportunities to improve internal process through code optimisation and automation.
• Build data quality dashboards, lineage flows / and or monitoring tools to utilize the data pipeline, providing active monitoring and actionable insight into overall data quality and data governance.
• Assist in migrating data from legacy systems onto newly developed solutions.
• Follow and lead best practices on all data security, retention, and privacy policies.

Requirements

• Bachelor’s degree.
• 6+ years of experience, with 3+ years’ experience of building ETL/ELT pipelines.
• Proven competency in solution design, development, implementation, reporting and analysis.
• Proficiency in Apache-Spark, Python and SQL languages.
• Proficiency in working with Text, Delta, Parquet, JSON, CSV, and XML data formats.
• Working knowledge of Spark structured streaming.
• AWS infrastructure experience, specifically working with S3.
• Solid understanding of git-based version control, DevOps, and CI/CD. Experience of working on Atlassian stack a plus.
• Knowledge of common web API frameworks and web services.
• Strong teamwork, relationship, and client management skills, and the ability to influence peers and senior management to accomplish team goals.
• Willingness to embrace modern technology, best practice, and ways of work.

Benefits

• Performance-based bonus and equity participation
• 13th month salary pro-rata
• 15-day annual leave+ 3-day sick leave + 1 birthday leave + 1 Christmas leave
• Meal and parking allowance
• Full benefits and salary rank during probation.
• Insurances as Vietnamese labor law and premium health care for you and your family
• Overseas travel opportunities for training and working related.
• Hybrid work mode, work-life balance 40-hr per week from Mon to Fri.