Data Engineer - Singapore - NodeFlair

NodeFlair

Verified Company

Singapore

1 week ago

Posted by:

Wei Jie

beBee Recruiter

Description

Job Summary:

Salary
S$4,500 - S$9,000 / Monthly

Job Type

Seniority
Junior

Years of Experience
At least 1 year

Tech Stacks
AWS Analytics Kinesis CI Microsoft pySpark Apache Databricks Git Azure Airflow kafka Flink SQL

Avensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation, and managed services.

Given our decade of success, we have evolved to become one of the top trusted providers in Singapore and service a client base across banking and financial services, insurance, information technology, healthcare, retail, and supply chain.

We are currently looking to hire a Data Engineer. This is an exciting opportunity to expand your skill set, and achieve job satisfaction and work-life balance. More details as below.

We are seeking a highly skilled Data Engineer with a strong background to join our dynamic team.

Responsibilities:

Data Pipeline Development:

Develop and maintain data pipelines that extract, transform, and load (ETL) data from various sources into a centralized data storage system, such as a data warehouse or data lake.

Data Integration:
Integrate data from multiple sources and systems, including databases, APIs, log files, streaming platforms, and external data providers.
Contribute to common frameworks and best practices in code development, deployment, and automation/orchestration of data pipelines.
Implement data governance in line with company standards.
Partner with Data Analytics and Product leaders to design best practices and standards for developing and productionalising analytic pipelines.

Partner with Infrastructure leaders on architecture approaches to advance the data and analytics platform, including exploring new tools and techniques that leverage the cloud environment (Azure, Databricks, others).

Monitoring and Support:
Monitor data pipelines and data systems to detect and resolve issues promptly. Develop monitoring tools, alerts, and automated error handling mechanisms to ensure data integrity and system reliability.

Requirements:

Must haves

Proven experience as a Data Engineering role (3 years +), with a strong track record of delivering scalable data pipelines.

Extensive experience designing data solutions including data modelling is required.

Extensive hands-on experience developing data processing jobs (PySpark / SQL) that demonstrate a strong understanding of software engineering principles is needed.

Experience orchestrating data pipelines using technology like ADF, Airflow etc is necessary.
Experience working with both real-time and batch data is important.
Experience building data pipelines on Azure is crucial. AWS data pipelines will be beneficial.
Fluency in SQL (any flavour), with experience using Window functions and more advanced features is required.
Understanding of DevOps tools, Git workflow and building CI/CD pipelines is essential.

Nice to Have

Domain knowledge of commodities, covering Sales, Trading, Risk, Supply Chain, Customer Interaction, etc. is highly desirable.
Experience with streaming data processing technologies such as Apache Kafka, Apache Flink, or AWS Kinesis can be beneficial. This includes the ability to design and implement real-time data processing pipelines.

WHAT'S ON OFFER
You will be remunerated with an excellent base salary and entitled to attractive company benefits. Additionally, you will get the opportunity to enjoy a fun and collaborative work environment, alongside a strong career progression.

CONSULTANT DETAILS

Consultant Name:
Meenakshi

Avensys Consulting Pte Ltd

EA License 12C5759

Privacy Statement:
Data collected will be used for recruitment purposes only.

Personal data provided will be used strictly in accordance with the relevant data protection law and Avensys' personal information and privacy policy.