data engineer in real-time big data

Описание:

GiG’s Data department develops and supports real-time big data projects, using information to support strategic decision-making and organisational growth.

Задачи:

• Design, create, and maintain ETL processes, data integrations, and data pipelines
• Ensure seamless handling of large volumes of data
• Develop high-performance services to process, store, and analyse large datasets using optimised code
• Design and implement data validation and quality checks tailored to business needs
• Troubleshoot live production systems and resolve issues
• Test and implement new technologies relevant to current and future projects
• Review code, promote best practices, and implement robust solutions across big data projects

Требования:

• Strong expertise in big data technologies such as Spark, Kafka, Nifi, ClickHouse, and big data datastores
• Strong understanding of schema design methodologies and data models
• Expertise in extracting large datasets using SQL and familiarity with SQL databases
• Object-oriented programming skills, ideally in Python and/or Java
• Strong experience with cloud computing and/or distributed systems

Условия:

• Great career development opportunities
• International health insurance
• Health and Wellbeing Package (350 EUR per year)
• Birthday day off
• Me Time: 1 day off per year