Overview
As part of the Data Engineer team, you will be responsible for design, development and operations of large-scale data systems operating at petabytes scale. You will be focusing on real-time data pipelines, streaming analytics, distributed big data and machine learning infrastructure. You will interact with the engineers, product managers, QA, BI developers and architects to provide scalable robust technical solutions.
Responsibilities
- Design, develop, implement and tune large-scale distributed systems and pipelines that process large volume of data; focusing on scalability, low-latency, and fault-tolerance in every system built.
- Provide and support the implementation and operations of the data pipelines and analytical solutions
- Experience in REST API data service – Data Consumption
- Experience in managing work teams
- Advanced project management
Mandatory Skills
8 to 10 years in technology project implementationEnglish conversational (Advanced)Demonstrates up-to-date expertise in Data Engineering, complex data pipeline developmentExperience in agile modelsExperience with Python, Java to write data pipelines and data processing layersExperience in Advanced Pipelines with AirflowExperience with Continuous Integration, DevOps, GithubPerformance tuning experience of systems working with large data setsProven working expertise with Big Data Technologies Hadoop, Hive, Kafka, Presto, SparkHighly proficient in SQLExperience with Cloud TechnologiesGCP – DataProc, BigQuery, Cloud FunctionsExperience with relational models, memory data stores desirable (Sql Server, Oracle, Cassandra, Druid)Knowledge in implementing advanced analytics models using ML / AI (Desirable)Knowledge of BI tools (Power BI, Tableau, Looker, etc.) DesirableRetail experience is a huge plusBenefits
Legal benefits and superior legal benefitsTraining and learning pathsSeniority level
AssociateEmployment type
Full-timeJob function
Information TechnologyIndustries
IT System Data Services and Data Infrastructure and Analytics#J-18808-Ljbffr