Agility Logistics Jobs 2023 – Data Engineer Vacancy in Dubai, UAE

As a Data Engineer, you are responsible for the development, performance, quality, and scaling of our data pipelines, with a special focus on distributed processing and data quality.

You will be working with stakeholders throughout the company to ensure we have high quality data to power our Data Science initiatives.

You will be collaborating closely with Data Scientists to ingest and efficiently process required data and facilitate the deployment of data science models into production.

Your Responsibilities:

• Build solutions and tools to enable, organize and process large amounts of data.
• Design and implement data ingestion and transformation in cloud-based environment and in on-premise platform.
• Create and maintain optimal data pipeline architecture for both batch and real-time processing.
• Developing scalable, flexible, and high-performance distributed data pipelines with real time stream processing.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
• Work closely with data scientists to define the best way to ingest and aggregate required data and process it efficiently, and how to get their data science models to run in production.
• Collaborate with business stakeholders to ingest new sources, cleaning and preparing the data and writing the code that connect it to the data lake or to specific application(s).
• Evaluate new technologies and build prototypes for continuous improvements in Data Engineering