Data Engineer

The Data Engineer role is responsible for designing implementing optimising and managing the data infrastructure framework with an emphasis on security and reliability and meet the needs of reporting solutions that drive decision-making and improve performance across the business. This role will contribute to the development of Aurecons Data Engineering standards and patterns as well as Data Governance framework and standards.

Key responsibilities :

• Develop data models and pipelines to meet the needs of reporting solutions that drive decision making and improve performance across all departments of the business.
• Responsible for the data modelling profiling and pipeline development of the Analytical data asset
• Ensuring data practices integrate into overall data architectures and data management principles (e.g. data governance data security metadata data quality)
• Collect aggregate analyse and cleanse structured / unstructured data from multiple sources
• Advising on data architecture creating data models delivering data migration integration and pipelines
• Troubleshooting and solving complex technical problems on large datasets
• Contribute to the development of Aurecons Data Engineering standards and patterns
• Contribute to the development of Aurecons Data Governance framework and standards
• Designing implementing optimising and managing the data infrastructure framework on Azure with an emphasis on security and reliability.
• Managing data infrastructure in a CI / CD (Continuous Integration / Continuous Development) pipeline with appropriate controls and documentation.
• Liaise with Reporting analysts and data analysts and other developers to ensure all pipelines are supported by the correct architectural implementation.
• Work closely with Data Analysts reporting analysts and domain architects to develop the Data & Analytics framework and Enterprise Data Lakehouse solution
• Comply with the procedures and processes used within the Data & Analytics team ensuring all work undertaken is executed consistently and professionally.
• Contribute to the triage analysis and resolution of identified issues and service requests for the Data Lakehouse and Enterprise Reports

Essential Skills and Experience :

Qualifications :

Key Skills

Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala

Employment Type : Full-Time

Experience : years

Vacancy : 1