Darumatic is an IT Consultancy that focuses on cloud native technologies. Our mission is to help clients "Build better software".
We are passionate about working with clients to transform their development and infrastructure teams to become cloud native. Our supportive culture will open up a variety of opportunities for you to develop your knowledge and skills whilst providing opportunities to connect with a range of IT professional networks. We believe in mutually beneficial professional working relationships and your employment with us is no exception.
For information about us visit https://www.darumatic.com
About the role
This is an exciting opportunity to work on a project with great social impact. Our client is currently seeking a Senior Azure Data Engineer to join their team. You will be responsible for developing and maintaining the data architecture, data models and standards for various Data Integration & Data Warehouse projects in Azure. Ensure new features and subject areas are modelled to integrate with existing structures and provide a consistent view. In addition, develop and maintain the documentation of the data architecture, data flow, and data warehouse data models appropriate for both technical and business audiences.
We are looking for someone who is willing to make a big impact and be equipped with good new ideas to benefit the project and the team in the right way.
Salary range: $1000 - $1100 plus super per day.
Accountabilities and deliverables:
- Provide technical expertise in Azure Data Platform
- Experienced in Data Warehouse / Data Lake / Lake House implementations
- Develop ELT pipelines in and out of data lake using Azure Data Factory, Databricks and Delta Lake.
- Promote changes between environments using Azure Devops Pipelines
- Integrate on-premise infrastructure with public cloud AZURE infrastructure.
- Partner with the Data Architects, Product Managers, Data Modellers, Business Users and Scrum Masters to deliver data integrations and BI solutions required for various projects.
- Review the work provisioned by vendor and ensure it is compliant with the guidelines
- Proactively identify improvement on the way of working within the team, with stakeholders to achieve a better outcome.
Experience and skills
- Bachelor’s Degree in computer science or Engineering or similar (preferred)
- Azure Data Engineer (preferred)
- Must have a minimum 5+ years in IT and 4+ years of hands-on experience working as an Azure Data Engineer, and 4+ years in Data Warehousing, ETL/ELT, BI, and Analytics projects.
- Extensive design and implementation experience in Azure cloud data platform and modern data warehousing, including data security, Azure Logging and Monitoring, Azure Databricks, Azure Blob Storage, Azure Data Lake, Azure Data Factory, Azure SQL Database, Azure Devops
- Expertise in data modelling, ELT using ADF, implementing complex views, stored procedures and standard DWH and ETL/ELT concepts.
- Proven experience working with Spark on large data volumes (preferably in Databricks)
- Proven understanding of the Lakehouse concept and how Delta Lake is used to realise this
- Deep understanding of relational and NoSQL data stores, methods, and approaches (star and snowflake, dimensional modelling).
- Experience with data security and data access controls and design.
- Proficiency in Python (Pyspark), SQL (ANSI and Spark SQL) and SCALA performance tuning and troubleshooting.
- Proficiency in building and optimising Azure Devops Pipelines is desired
- Experience working in DataOps model preferred
- Provide resolution to an extensive range of complex data pipeline-related problems proactively and as issues surface.
- Experience with Agile development methodologies
Submit Your Application
You have successfully applied
- You have errors in applying