Maintain efficient data infrastructure including data warehouse and data lake implementation
Data lake should be able ingest relevant data without impacting data source performance
Ingest relevant data from internal and external sources
Create data solutions such as data integrations and automated jobs
Create streamlined jobs to speed up data processing and optimize data pipeline
Maintain efficient data infrastructure including data warehouse and data lake implementation
Data lake should be able ingest relevant data without impacting data source performance
Ingest relevant data from internal and external sources
Create data solutions such as data integrations and automated jobs
Create streamlined jobs to speed up data processing and optimize data pipeline
Can create data models using Python, Scala, or R
Experience in utilizing Pandas or Numpy (or any equivalent is a plus)
Can setup data pipeline using Airflow (or any equivalent)
Experience in implementing the data engineering architecture in a cloud environment such as Amazon Web Services or Google Cloud Services Can manage...
Graduate of any STEM or business bachelor's degree
Has at least 3 to 5 years of data engineering experience and 2 to 4 years of people management experience
n/a
• Responsible for developing, testing, and maintaining data pipelines.
• Work with the data architect in designing, building and maintaining data systems specified by the data architect’s data framework.
• Work alongside data scientists to build scalable data analytics pipeline.