Senior Data Engineer

Senior Data Engineer

Experience: 6+ Years

Location: WFH / Remote (India)

Open Positions: Multiple

Qualification: B.Tech / M.Tech / MCA orequivalent

Work Timings: 1:30 PM IST – 10:30 PM IST

Functional Area: Data Engineering

About the Role

Weare seeking an experienced Senior Data Engineer to design, develop, and maintain scalable, reliable, andhigh-performance data platforms and pipelines that support businessintelligence, analytics, and data-driven decision-making.

Theideal candidate will have strong hands-on expertise in Python,Apache Spark, Apache Airflow, Amazon Redshift, SQL, and modern data engineeringpractices, along with mandatory experience in the Healthcare domain. The candidate should haveexperience building end-to-end ETL/ELT pipelines, processing large-scaledatasets, implementing data quality frameworks, and supporting enterpriseanalytics and reporting needs.

This rolerequires a technically strong professional who can collaborate with businessstakeholders, data analysts, architects, and application teams to deliverrobust and scalable data solutions.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows for structured and unstructured data.
• Build and optimize data ingestion, transformation, and data integration processes using Python and Apache Spark.
• Develop, schedule, and monitor workflows using Apache Airflow.
• Design and manage enterprise data warehouse solutions using Amazon Redshift.
• Implement data models, data marts, and reporting datasets to support analytics and business intelligence requirements.
• Optimize data processing jobs and database performance for scalability and efficiency.
• Work with large healthcare datasets, ensuring data accuracy, consistency, and compliance with industry standards.
• Collaborate with business stakeholders to understand reporting and data requirements.
• Implement data quality checks, validation frameworks, and monitoring processes.
• Troubleshoot data pipeline issues and perform root cause analysis.
• Ensure data security, governance, and compliance requirements are adhered to.
• Participate in code reviews, development best practices, and continuous improvement initiatives.
• Create and maintain technical documentation, data flow diagrams, and data mapping documents.
• Work closely with cross-functional teams including Data Science, Analytics, Product, and Engineering teams.

Skills, Knowledge & Experience

Mandatory Skills

• 6+ years of experience in Data Engineering, Big Data Engineering, or related roles.
• Mandatory experience working in the Healthcare domain.
• Strong hands-on experience in Python development for data engineering solutions.
• Strong hands-on experience with Apache Spark (PySpark) for large-scale data processing.
• Hands-on experience with Apache Airflow for workflow orchestration and scheduling.
• Strong experience designing, developing, and optimizing solutions on Amazon Redshift.
• Advanced SQL skills with experience in query tuning and performance optimization.
• Experience building ETL/ELT pipelines and modern data integration frameworks.
• Strong understanding of data warehousing concepts, dimensional modelling, and data architecture.
• Experience handling large-scale datasets and complex data transformation requirements.
• Strong debugging, troubleshooting, and performance tuning skills.
• Experience working with cloud-based data platforms and services (AWS or Azure) is preferred.
• Excellent analytical and problem-solving skills.
• Strong communication and stakeholder management capabilities.

Preferred Skills

• Experience with AWS data services such as S3, Glue, EMR, Athena, Lambda, or Kinesis
• Experience with Azure Data Factory, Azure Databricks, ADLS, Synapse Analytics, or other Azure data services
• Exposure to Databricks, Snowflake, or modern Lakehouse architectures
• Experience with CI/CD for Data Engineering pipelines
• Knowledge of Healthcare data standards such as HL7, FHIR, EHR/EMR, Claims, Provider, Member, or Clinical Data
• Experience with data governance, data quality, and compliance frameworks
• Experience with Git and DevOps best practices for data engineering

This jobdescription is not intended to cover or contain a comprehensive listing of allresponsibilities, duties, or activities that are required. Responsibilities,duties, and/or activities may change, or new ones may be added at any time withor without notice.

If you are amotivated professional with a passion for delivering impactful solutions, we'dlove to hear from you. Apply today to be part of a dynamic and forward-thinkingteam at NextGen Invent.