About the role
Role Overview
We are seeking a hands-on Data Engineer to join our team, focusing on building the
robust foundations required for modern analytics and AI. In this role, you will bridge
the gap between raw data and production-ready AI solutions, architecting scalable
pipelines that power machine learning models and business intelligence. You will be
responsible for the engineering discipline that ensures data reliability, performance, and
security within the Databricks ecosystem.
Key Responsibilities
AI/ML Pipeline Support: Architect, build, and maintain scalable data pipelines
specifically designed to support AI/ML workflows, including feature
engineering and model training datasets.
Design and implement robust data pipelines, data lakes, and data warehousing
solutions.
Partner with data scientists and analysts to deploy machine learning models and
analytics solutions on Databricks.
Enhance Databricks cluster performance, ensuring cost-effectiveness and efficient
resource utilization.
Implement Databricks security features, enforce data governance, and maintain
compliance with industry standards.
MLOps Integration: Collaborate with Data Scientists to deploy and
operationalize machine learning models into production environments securely.
Contribute to the technical direction and best practices of data engineering
initiatives.
Stay current with Databricks innovations, best practices, and industry trends,
and share knowledge across the team.
Required Skills & Qualifications
Total Experience: 1-4 years in data engineering or related fields.
Databricks Expertise: Solid experience with Databricks, focusing on platform
engineering, data engineering, and analytics.
Technical Skills: Proficiency in Python, Scala, or Java; strong experience with
Databricks APIs, Apache Spark, and Delta Lake.
Data Engineering Background: Solid experience in data warehousing, ETL
processes, and data governance frameworks.
AI/ML Exposure: Practical experience supporting AI/ML lifecycles, including
familiarity with ML frameworks (e.g., TensorFlow, PyTorch) or MLOps practices.
Collaboration & Ownership: Proven ability to work independently, take
ownership of projects, and collaborate effectively with a team.
Communication: Excellent interpersonal and communication skills for effective
collaboration with stakeholders.
Good to Have
Cloud Experience: Hands-on experience with AWS, Azure, or GCP.
Certifications: Databricks Certified Professional or equivalent.
Machine Learning: Familiarity with ML concepts and popular libraries (e.g.,
TensorFlow, PyTorch, Scikit-learn).
What we are offering
Best-in-industry learning and growth opportunities.
Comprehensive Health and wellness benefits, including medical insurance for
self and family.
Employee rewards program and paid parental leave.
Flexible schedule and remote working benefits.
Originally posted on Himalayas
