Azure Data Engineer - Singapur, Singapore - SMRT Corporation Ltd

    SMRT Corporation Ltd
    SMRT Corporation Ltd Singapur, Singapore

    3 weeks ago

    Default job background
    $80,000 - $120,000 per year Technology / Internet
    Description

    Job Purpose

    We are looking for an experienced data engineer to join our team. You will use various methods to transform raw data into useful data systems. For example, you will create and maintain optimal data pipelines architecture in cloud. Overall, you will strive for efficiency by aligning data systems with business goals.

    To succeed in this data engineering position, you should have strong analytical skills and the ability to combine data from different sources. Data engineer skills also include familiarity with several programming languages and knowledge of learning machine methods.

    If you are detail-oriented, with excellent organizational skills and experience in this field, we would like to hear from you.

    Responsibilities

    The duties and responsibilities are listed below. The list is not comprehensive and related duties and responsibilities may be assigned from time to time.


    • Understand the data landscape to formulate and complete end-to-end data strategy that includes data gathering, data processing, analysis, ongoing scaled deliverables, and presentations to facilitate decision-making and forward planning.

    • Identify opportunities for data acquisition, work with system owners to automate data connection.

    • Analyse and organize raw data, combine raw information from different sources, build data systems and data pipelines, perform extract transform and load (ETL) to prepare data for prescriptive and predictive modeling.

    • Able to analyze, implement and integrate the right services in cloud based on different data formats (Structured, Semi-Structured and Unstructured)

    • Evaluate business needs and objectives, interpret trends, patterns, and conduct complex data analysis and report on results

    • Build algorithms and prototypes, explore ways to enhance data quality and reliability; Develop analytical tools and programs

    • Collaborate with data scientists and architects on several projects

    Qualifications & Work Experience


    • Degree in Computer Science, IT, or similar engineering field; a Master is a plus

    • Previous experience as a data engineer or in a similar role

    • Data engineering certification is a plus

    Skills

    Technical skills include:

    • Strong analytical skills with the ability to collect, clean, organise, analyse, and disseminate significant amounts of information with attention to detail and accuracy.

    • Strong technical knowledge of programming languages (Python, SQL & Spark)

    • Experience in Azure Cloud technologies (i.e., Azure Data Factory, Azure Synapse Analytics, Azure Data Lake, Azure Functions, Event Hub, Azure Logic Apps and ARM Templates)

    • Experience with CI/CD tool: Azure DevOps.

    • Experience in IaC (ARM Templates, Bicep).

    • Strong knowledge about Git and branching.

    • Technical expertise with data models, data mining, and segmentation techniques.

    • Familiarity with the technology stack available in the industry for metadata management: Data Governance, Data Quality, MDM, Lineage, Data Catalog etc.

    • Hands-on experience with SQL database design.

    • Nice to have Skills: Azure API Management, Azure API gateway, Scala, Snowflake, and Databricks.

    Generic skills include:

    • Critical thinker and problem-solving skills

    • Team player

    • Good time-management skill

    • Great interpersonal and communication skills