Job Title:  Consulting - BO - Cloud Engineering - Consultant - Azure Data Engineer

What impact will you make?

Every day, your work will make an impact that matters, while you thrive in a dynamic culture of inclusion, collaboration, and high performance. As one of the leading professional services organisations, Deloitte is where you will find numerous opportunities to succeed and realise your full potential.

The team
Deloitte is working with global customers on cloud technologies to help them spot unseen business trends, transform, and curate data to find meaningful insights and so on. Due to continued expansion, we’re searching for like-minded individuals to help us ‘take it to the next level’.

This is an exciting opportunity for an experienced developer of large-scale data solutions. You will join a team delivering a transformative cloud hosted data platform for key clients

The ideal candidate will have a proven track record in implementing data ingestion and transformation pipelines for large scale organizations. We are seeking someone with deep technical skills in a variety of technologies to play an important role in developing and delivering early proofs of concept and production implementation.

You will get good experience in building solutions using a variety of open source tools a Microsoft Azure services and a proven track record in delivering high quality work to tight deadlines.

Scope of work

Your main responsibilities will be:

• Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure Databricks
• Delivering and presenting proofs of concept of key technology components to project stakeholders.
• Developing scalable and re-usable frameworks for ingesting and enriching datasets
• Integrating the end to end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times
• Working with event based / streaming technologies to ingest and process data
• Working with other members of the project team to support delivery of additional project components (API interfaces, Search)
• Evaluating the performance and applicability of multiple tools against customer requirements
• Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints.

Required Skills

• Strong knowledge of Data Management principles
• 4 to 6 years of total years of experience
• Experience in building ETL / data warehouse transformation processes
• Direct experience of building data pipelines using Azure Data Factory and Apache Spark (preferably Databricks).
• Experience using Apache Spark and associated design and development patterns
• Microsoft Azure Big Data Architecture certification is an advantage.
• Hands-on experience designing and delivering solutions using Azure Storage, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics
• Experience with Apache Kafka / Nifi for use with streaming data / event-based data (Nice to have but not mandatory)
• Experience with other Open Source big data products Hadoop (incl. Hive, Pig, Impala)
• Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB, Cassandra, Neo4J)
• Experience working in a Dev/Ops environment with tools such as Microsoft Visual Studio Team Services, Terraform etc.

Your role as a leader

At Deloitte India, we believe in the importance of leadership at all levels. We expect our people to embrace and live our purpose by challenging themselves to identify issues that are most important for our clients, our people, and for society, and make an impact that matters.

In addition to living our purpose, managers across our organisation:

• Develop self by actively seeking opportunities for growth, share knowledge and experiences with others, and act as a strong brand ambassadors
• Understand objectives for clients and Deloitte, align own work to objectives and set personal priorities
• Seek opportunities to challenge self
• Collaborate with others across businesses and borders to deliver and take accountability for own and team results
• Identify and embrace our purpose and values and put these into practice in their professional life
• Build relationships and communicate effectively in order to positively influence peers and other stakeholders

Professional growth

At Deloitte, our professional development plan focuses on helping people at every level of their career to identify and use their strengths to do their best work every day. From entry-level employees to senior leaders, we believe there’s always room to learn.From on-the-job learning experiences to formal development programs at Deloitte University, our professionals have a variety of opportunities to continue to grow throughout their career. Explore Deloitte University, The Leadership Center.

Benefits

At Deloitte, we know that great people make a great organisation. We value our people and offer employees a broad range of benefits. Learn more about what working at Deloitte can mean for you.

Our purpose                                                                                                            

Deloitte is led by a purpose: To make an impact that matters.

Every day, Deloitte people are making a real impact in the places they live and work. We pride ourselves on doing not only what is good for clients, but also what is good for our people and the communities in which we live and work—always striving to be an organisation that is held up as a role model of quality, integrity, and positive change. Learn more about Deloitte's impact on the world