About the role
Senior Data Engineer
Experience: 6+ Years
Location: WFH / Remote (India)
Open Positions: Multiple
Qualification: B.Tech / M.Tech / MCA orequivalent
Work Timings: 1:30 PM IST – 10:30 PM IST
Functional Area: Data Engineering
About the Role
Weare seeking an experienced Senior Data Engineer to design, develop, and maintain scalable, reliable, andhigh-performance data platforms and pipelines that support businessintelligence, analytics, and data-driven decision-making.
Theideal candidate will have strong hands-on expertise in Python,Apache Spark, Apache Airflow, Amazon Redshift, SQL, and modern data engineeringpractices, along with mandatory experience in the Healthcare domain. The candidate should haveexperience building end-to-end ETL/ELT pipelines, processing large-scaledatasets, implementing data quality frameworks, and supporting enterpriseanalytics and reporting needs.
This rolerequires a technically strong professional who can collaborate with businessstakeholders, data analysts, architects, and application teams to deliverrobust and scalable data solutions.
Key Responsibilities
Design, develop, and maintain scalable data pipelines and ETL/ELT workflows for structured and unstructured data.
Build and optimize data ingestion, transformation, and data integration processes using Python and Apache Spark.
Develop, schedule, and monitor workflows using Apache Airflow.
Design and manage enterprise data warehouse solutions using Amazon Redshift.
Implement data models, data marts, and reporting datasets to support analytics and business intelligence requirements.
Optimize data processing jobs and database performance for scalability and efficiency.
Work with large healthcare datasets, ensuring data accuracy, consistency, and compliance with industry standards.
Collaborate with business stakeholders to understand reporting and data requirements.
Implement data quality checks, validation frameworks, and monitoring processes.
Troubleshoot data pipeline issues and perform root cause analysis.
Ensure data security, governance, and compliance requirements are adhered to.
Participate in code reviews, development best practices, and continuous improvement initiatives.
Create and maintain technical documentation, data flow diagrams, and data mapping documents.
Work closely with cross-functional teams including Data Science, Analytics, Product, and Engineering teams.
Skills, Knowledge & Experience
Mandatory Skills
6+ years of experience in Data Engineering, Big Data Engineering, or related roles.
Mandatory experience working in the Healthcare domain.
Strong hands-on experience in Python development for data engineering solutions.
Strong hands-on experience with Apache Spark (PySpark) for large-scale data processing.
Hands-on experience with Apache Airflow for workflow orchestration and scheduling.
Strong experience designing, developing, and optimizing solutions on Amazon Redshift.
Advanced SQL skills with experience in query tuning and performance optimization.
Experience building ETL/ELT pipelines and modern data integration frameworks.
Strong understanding of data warehousing concepts, dimensional modelling, and data architecture.
Experience handling large-scale datasets and complex data transformation requirements.
Strong debugging, troubleshooting, and performance tuning skills.
Experience working with cloud-based data platforms and services (AWS or Azure) is preferred.
Excellent analytical and problem-solving skills.
Strong communication and stakeholder management capabilities.
Preferred Skills
Experience with AWS data services such as S3, Glue, EMR, Athena, Lambda, or Kinesis
Experience with Azure Data Factory, Azure Databricks, ADLS, Synapse Analytics, or other Azure data services
Exposure to Databricks, Snowflake, or modern Lakehouse architectures
Experience with CI/CD for Data Engineering pipelines
Knowledge of Healthcare data standards such as HL7, FHIR, EHR/EMR, Claims, Provider, Member, or Clinical Data
Experience with data governance, data quality, and compliance frameworks
Experience with Git and DevOps best practices for data engineering
This jobdescription is not intended to cover or contain a comprehensive listing of allresponsibilities, duties, or activities that are required. Responsibilities,duties, and/or activities may change, or new ones may be added at any time withor without notice.
If you are amotivated professional with a passion for delivering impactful solutions, we'dlove to hear from you. Apply today to be part of a dynamic and forward-thinkingteam at NextGen Invent.
Originally posted on Himalayas