About the role
The Senior Data Engineer will build the architecture and implementation of the data platform, data pipeline and infrastructure.
- Responsible for the development, maintenance, improvement, cleaning, and manipulation of data in the context of the data platform.
- Work with data analytics teams, data scientists, and other data warehouse engineers in order to understand and aid in the implementation of enterprise data warehouse/enterprise data platform requirements, analyze performance, and troubleshoot any existing issues.
- Have to be an expert in ETL technologies, SQL development, and Shell Scripting, further spearheading the data and analytics database design (Logical Models and Physical Data Models), creation of master data and maintaining data flow activities.
- Lead as a pioneer in a data-driven organization and this role is instrumental in making this happen.
Key Results Area:
- Design and create enterprise data platform systems optimized for performance, implementing schema changes, and maintaining data architecture standards across all of the business functions
- Additionally tasked with designing and developing scalable ETL packages from the business source systems and the development of ETL routines/jobs in order to populate databases from sources and also to create FACTs, Aggregates and dimensions
- In this capacity, the Data Engineer is also responsible for enabling and running data migrations across different databases and servers in the ETL data pipelines. For example, data migration from MySQL/Oracle to SQL servers. She/He defines and implements data stores based on the system requirements and consumer requirements
- Strive to ensure proper data governance and quality across the Data and Analytics department and the business as a whole
- Work collaboratively with the entire Data and Analytics team, providing support to the entire department for its data-centric needs.
Competencies & Behaviors:
- Technical: Keeps up to date with the latest trends in data and data engineering (Both on-premises and Cloud Practices)
- Passion for data: Belief that data and insight can grow and transform an organization
- Partnership: Successful professional approaches to collaboration with all stakeholders and teams
- Communication: Creating and promoting an enabling environment for open communication; Constructively challenging those with power and authority
- Governance: Work well with the Management, regardless of its composition; contribute to Management; Adhere to clear lines of responsibility and accountability
- Management: Create a positive and productive work environment, Model proper staff behaviour and promote inclusive practices; create a sense of shared responsibility/credit for accomplishments and shared responsibility for challenges or failures; Lead an efficient and effective organizational operation according to best practices
- Decision-making: Delegate appropriate decisions and responsibilities; Make clear and timely decisions; Fair and transparent decision-making
- Organizational Development: Create a work environment in which learning is; create an emotionally intelligent organization and staff competence; Build and promote effective teams, creating an environment of creativity and innovation
Job Requirements:
- 5+ Years experience in data engineering
- Experience with high data volumes, both digital and transactional
- Experience in building and planning a new data infrastructure for a sizable organization
- Must have hands-on experience in full project development lifecycle related to Data Warehouse and engineering
- Understanding Agile development will be an added advantage
- University degree in a quantitative subject
- Language Requirement: Thai, English (written and verbal)
Technology and Methodology:
- Essential: Strong Relational Database experience (MySQL, Microsoft SQL and Oracle)
- Essential: Experience in analytical and quantitative data pipeline creation and data processing with one or more big-data components such as Hadoop, Spark, NoSQL databases
- Essential: Familiarity in building, configuring and monitoring data platforms using highly scalable distributed computing solutions in AWS or equivalent
- Knowledge of data warehousing principles. (Data Wrangling, Data Profiling, Integration, etc.)
- Data modelling skills (Entity Relationships, Logical Data Model, Physical Data Model, Dimensional Modeling, etc.)
- Expert-level hands-on experience in ETL and Reporting technologies like SSIS, SSRS, SSAS and Power BI 3-dimensional cube design
- Ability to develop and update detailed technical documentation
- Awareness of IoT platforms/devices used in FinTech and Digital Money/Payments
Originally posted on Himalayas