12200273 - Data Engineer (Python/pyspark) - Singapore - Capgemini
Description
Responsibilities:
- Develop enterprise grade Data Products using Spark and other Big Data frameworks
- Benchmark existing solutions, identify gaps, and propose solutions to eliminate them
- Be able to work within a team of data engineers, data scientists, Devops engineers, business analysts, project managers around the proposed solution and contribute to the team efforts
- Provide operational assistance and guidance for the resulting data product including monitoring, management, disaster recovery, security compliance/auditing, networking, storage, service brokers and build packs
- Establish and maintain continuous delivery pipelines for the deployment of Data Products and related products infrastructure
- Design and implement continuous integration and continuous delivery processes to deliver customer apps to production, fostering a culture of continuous process improvement
- Continuously learn and be at the leading edge of industry trends
- Become an agent of change within organizations
Requirements:
- Bachelor's degree in Computer Engineering or equivalent
- Strong handson experience in distributed data architectures is a must
- Strong knowledge of PySpark is a must
- Working knowledge of Python is a must
- CI/CD experience (Jenkins, GitHub) is a must
- Working knowledge of Bash scripts is a must
- A clear understanding of cloud service and deployment models. AWS/GCP experience is nice to have
- Kubernetes or Docker experience is nice to have
- Experience in release management or production operations, leading teams performing systems automation and integration (leveraging an agile methodology / lean techniques)
More jobs from Capgemini
-
Linux/unix Administrator
Singapore - 1 week ago
-
Application Middleware Infrastructure Sme
Singapore - 1 week ago
-
Fullstack Developer
Singapore - 5 days ago
-
Principal Lead Consultant
Singapur, Singapore - 1 week ago
-
Solution Analyst
Singapore - 1 week ago
-
Solution Analyst
Singapore - 1 day ago