Start Your Search Here

Job Search

ALLTECH CONSULTING SVC INC

Mississauga / Global

Data Pipeline Engineer with Databricks Focus

Job Description

Become a Data Pipeline Engineer leveraging Databricks, Python, and SQL for robust data solutions. This role involves close interaction with business analysts for effective data management.

In this position, your main responsibility will be to build and optimize data pipelines for high-quality data processing. Your expertise in Spark programs and data architecture will ensure reliable integration and thorough monitoring of Databricks jobs, enhancing data governance throughout the cloud environment.

Key Responsibilities:

• Build data pipelines ensuring quality and efficiency

• Develop and deploy Spark applications in Databricks

• Troubleshoot and optimize job performance

• Implement best practices in data security and management

• Review code for optimal execution and compliance

Requirements:

• 5+ years of experience in Python coding

• Extensive SQL development expertise

• Proven experience in ETL with Databricks Pyspark

• Familiarity with cloud data warehouses like Redshift

• Architectural experience in enterprise data migrations

Utilize your skills in Python and SQL as a Data Pipeline Engineer in a collaborative environment.

#J-18808-Ljbffr

Apply Now

Similar Opportunities

View all jobs