Azure Data Engineer
- Job Category: Engineering/Technical
- Education: N/A
- FT/PT: Full-Time
- Company: CGI
- More Jobs from this Company
Position ID: J0421-0555
Seeking a sr data engineer with 8-10 years of industry experience including developing and deploying applications on Microsoft Azure Databricks platform. This individual should be self-motivated to drive solutions and proactively create and present (in sufficient detail) the documentation required to support and describe technical solutions. The candidate will work with cloud architects to establish pipeline to Azure data stores using either NiFi or DataBricks. Will build templates for engineering processes in cloud. Will act as the lead data engineer on the project.
• The data engineer will be responsible for development, deployment, maintenance, diagnostics and support of spark ETL jobs on the MS Databricks platform.
• Will streamline code, rationalize datasets and tables to arrive at single source of truth
• Will build detailed architecture diagrams or Entity Relationship Diagrams for all workflows and processes
• 8+ years of hands on experience with cloud automation and scripting in an Azure environment.
• Experience in development of apache Spark code using PySpark or Scala
• Experience in using Azure Databricks Platform
• Experience in using Azure Data Factory to call Azure Databricks notebook activities
• Experience in using Git based repositories
• Good fundamental knowledge about distributed computing, RDBMS and Dimensional modeling concepts
• Exposure to Azure DevOps
• Exposure to Azure Storage (Blob/Data Lake Store), Azure Catalog, Databricks Delta.
• Strong communication and presentations skills required.
SQL, TeraData SQL 10+ years 5
Cloud Platforms (Azure) 8+ years 5
NiFi, DataBricks 8+ years 4
Scripting languages (HTML, Javacsript, jquery, Angular js, .Net, Python etc) 8+ years 5
Hive 6+ years 4
Azure Container Services
500 West Summit Hill Drive
Knoxville TN 37902
Job Link: Apply Here!