Data Engineer ID89384
AgileEngine
We are looking for a Data Engineer with strong Databricks and GCP experience to develop scalable ETL/ELT pipelines and productionize agentic workflows for data engineering automation.
The mandatory requirements are 5+ years of experience with Databricks and PySpark, advanced SQL, hands-on GCP experience, and experience with Delta Lake and modern lakehouse architectures.
RECRUITMENT PROCESS
1. Application — Share a few details about your experience and background.
2. Coding Challenge — If applicable, complete it in the coding language you are most comfortable with.
3. Video Interview — Record a short video introduction in English.
4. Technical Interview or Hiring Manager Interview — Discuss your experience and fit for the role.
5. Offer — If it’s a match, you’ll receive an offer.
WHAT YOU’LL GAIN
• Remote work — Work from where you feel most productive.
• Local presence in India — Work in a structured and compliant environment aligned with Indian regulations.
• Competitive compensation in INR — Receive compensation in INR, plus support for learning, education, and wellness.
• Exciting projects — Work with modern technologies for global clients and fast-growing companies.
MUST HAVES
• 5+ years of strong hands-on experience with Databricks and PySpark.
• Advanced SQL and data-processing skills.
• Hands-on experience with GCP, particularly BigQuery .
• Experience with Delta Lake and modern data lake/lakehouse architectures .
• Strong understanding of ETL/ELT, data pipeline design, performance optimization, and data quality.
• Experience building reliable, scalable, production-grade data solutions.
• Strong analytical and troubleshooting skills.
• Understanding of software engineering practices, including testing, version control, deployment, monitoring, and production support.
• Upper-intermediate English level.
NICE TO HAVES
• Experience developing or integrating AI/agentic workflows, AI agents, or workflow automation solutions.
• Experience applying AI to automate data engineering, validation, troubleshooting, or operational processes.
• Familiarity with orchestration and automation frameworks.
• Experience developing reusable data engineering frameworks and platform components.
• Exposure to productionizing AI-enabled solutions with appropriate validation, monitoring, and human oversight.
WHAT YOU WILL DO
• Design, develop, and optimize scalable ETL/ELT data pipelines using Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering and operational processes such as data validation, issue identification, troubleshooting, and workflow execution.
• Integrate agentic capabilities with existing Databricks, GCP, BigQuery, and Delta Lake environments.
• Develop data pipelines and reusable frameworks and components to support new business requirements and datasets across multiple use cases.
• Implement data quality checks, monitoring, validation, exception handling, and production controls.
• Optimize PySpark and SQL workloads for performance, reliability, and scalability.
• Support testing, deployment, productionization, troubleshooting, and ongoing enhancement of data and agentic solutions.
• Collaborate with business, data engineering, and platform teams to identify further automation opportunities.
- ...help establish standards for mobile payments. Brainstorm and create new product and business directions. Influence Razorpay’s engineering culture and technical direction. Requirements ~5+ years of experience in technology. ~ Strong product design sense. ~...SuggestedFull timeWorldwideWeekend work
- ...Job Description Summary This opening is for a Staff Engineer - Mechanical Design - Airfoils in GE9X HPC module within High Pressure Compressor... ...and externally Leverage digital tools & analyze the data insights to drive appropriate actions Provide technical guidance...SuggestedFull timeRelocation package
- ...enabling deep customer proximity and a uniquely global innovative engine. As Japan’s semiconductor powerhouse and a top global player in... ...(example: Spotfire) to analyze large amounts of characterization data and determine and set production test limits and predict yield data...SuggestedFull timeLocal area
- ..., and a better working world for all. EY GDS | PRINCIPAL DATA SCIENTIST Oil & Gas | Senior Manager Job Title Principal... ...from measurement artefacts. Combine learned models with engineering equations, mass and energy balances, simulation outputs, or physical...SuggestedHybrid workShift work
- ...strong team building not just products, but the financial infrastructure of the future. The Role: A lead software development engineer at Razorpay is well-grounded - innovative, quality focussed, product thinker, and business conscious - not mere coders, of course, with...SuggestedWorldwideWeekend work
- ...Thermo Fisher Scientific seeks an experienced clinical data management professional to lead a team managing data operations for pharmaceutical and biotech clients. The role involves staff supervision, process oversight, and coordination across multiple clinical trials globally...
- ...Developer to design, develop, and maintain reporting solutions and data integration services. The role involves creating dashboards,... ...solutions Bachelor's degree in Computer Science, IT, Software Engineering, or related field Strong communication and analytical skills...
- ...Location: Bengaluru We are looking for a talented and analytical Data Scientist to develop data-driven solutions, build predictive... ...data analysis, data cleaning, transformation, and feature engineering to prepare datasets for modelling. Develop, train, evaluate,...Full time
- Responsibilities Build, maintain, and improve highly automated build, release, and testing infrastructure. Create scripts, tools, and automation for internal use and continuous delivery. Collaborate with Development, QA, and Operations to enhance build and automation...Full timeHybrid work
Rs 30 lakh p.a.
Enterprise Sales Specialist – Cloud Summary We are seeking a seasoned Enterprise Sales Specialist to drive strategic growth within the IT Services sector. In this pivotal role, you will leverage your extensive background in cloud technologies to lead high-value sales...Long term contractFull time- ...and third-party services. Work with databases such as MySQL, PostgreSQL, or MongoDB. Collaborate with frontend developers, QA engineers, and other team members. Identify, troubleshoot, and resolve application bugs and performance issues. Participate in code reviews...Full timeHybrid workRemote jobFlexible hours
- ...for elderly, low-literacy or first-time smartphone users, and the accessibility and readability choices you made.- Performance-tier engineering: an app that adapts to low-end devices and slow networks, and the measurements you used to segment it.- Comfort going native when...Work at officeRemote job
- ...deep technical leadership with organizational management responsibilities. The successful candidate will be responsible for building engineering teams, enabling the Architects and Development Leads to drive technical strategy, establishing development processes and standards,...Work at office
- Job Summary:We are looking for an experienced Lead Databricks Engineer with strong expertise in Azure Databricks, PySpark, Python, SQL, Azure Data Factory (ADF), and Delta Lake to develop and optimize scalable data engineering solutions.Key Responsibilities:- Design, develop...
- ...Responsibilities :- Design, develop, and deploy Machine Learning and Deep Learning models for business use cases.- Perform data analysis, feature engineering, model development, validation, and performance optimization.- Build predictive models using algorithms such as...
- Senior Software Engineer - AI & Product EngineeringJob Location : BangaloreExperience : 3 to 10 yearsJob Type : Full TimeWork mode : HybridWere... ...features end-to-end- Collaborate closely with product, design, data, and business teams in a fast-paced startup environment- Drive...Full time
- Job Description - Senior DevOps Engineer:Experience : 5 - 8 YearsLocation : BangaloreEmployment Type : Full-TimeJob Summary : We are looking for an experienced Senior DevOps Engineer to manage cloud infrastructure, automate deployments, and ensure system reliability.Key Responsibilities...Full time
- ...the 5G network edge, enabling broader 5G coverage, multi-gigabit data speeds, low latency and strong security to deliver highly reliable... ...in the harshest industrial environments.Position Overview :Staff Engineer - Data is responsible for architecture, design and implementation...
- ...4/IPv6 Packet Flow, Tagging Concepts.Python (Basic & Advanced): Data Types, Classes, Multi-threading.Strong analytical and problem-solving... ...skills.Experience working with global, cross-functional engineering teams.Hands-on experience with specialized Python libraries: python...Work at officeWeekday work
- Job Description :Currently we have an urgent position for Senior Azure Hybrid Infrastructure and Identity Architect role with one of our projects, location based out of remote.The ideal candidate should have deep expertise in Azure Local / Azure Stack technologies, SAP on Azure...Hybrid workLocal areaRemote job
- ..., Discovery implementation and troubleshooting, Service Mapping, data quality management, and infrastructure integration. The candidate... ...services.- Configure and maintain the Identification and Reconciliation Engine (IRE) and CI Class Manager.- Perform CMDB Health assessments and...Full timeImmediate start
- ...Overview : We're looking for someone with 6+ years of Software Engineering experience and hands-on experience building LLM-powered or agent... ...software engineering fundamentals : system design, API design, data modelling, and clean, maintainable code.- Experience building and...
- Role Overview:We are looking for an experienced Forward Deployed AI Engineer to design, build, and deploy AI-powered solutions that address... ...and agentic workflows.- Develop backend services, REST APIs, and data pipelines to integrate AI capabilities with existing enterprise applications...
- Role & responsibilities:- Understanding the individual product requirements for future product release development and innovation.- Being conversant across front and back-end development to make modifications and enhancements across all aspects of the system.- Ongoing interaction...
- About the role : We're hiring a Principal AI Engineer at KreditBee, Bangalore. What if your next role meant building the AI platform that every... ...- Technical leadership : you influence direction across Product, Data Science and ML teams.Bonus points for : - Building AI platforms...Work at officeLocal area
- Description :GSPANN is hiring a POS Automation Test Lead to guide quality engineering for retail store checkout systems, focusing on test automation,... ...delivery, including task dependencies, test schedules, test data creation, and test environment utilization.- Develop and monitor...Flexible hours
- ...Azure cloud services.- Implement authentication, authorization, data protection, and secure secrets management.- Write unit, integration... ...reliability.- Collaborate with frontend developers, DevOps engineers, QA, and product stakeholders.- Participate in code reviews and maintain...Hybrid work
- ...Serialization/Deserialization and Immutability concepts- Good fundamental knowledge in Enums, Collections, Annotations, Generics, Auto boxing and Data Structure- Database RDBMS/No SQL (SQL, Joins, Indexing)- Multithreading (Re-entrant Lock, Fork & Join, Sync, Executor Framework)-...
- ...applications. Design and optimize database schemas for performance and scalability. Write complex queries and perform efficient data transformations in Snowflake. Integrate Snowflake with data ingestion and analytics tools. Serve as a subject matter expert and contribute...Full timeWork at office
Rs 3 - 6 lakhs p.a.
...Job Summary We are seeking a detail-oriented and analytical Data Analyst to support our manufacturing operations by collecting, analyzing... ...'s degree in Data Analytics, Statistics, Computer Science, Engineering, Mathematics, Operations Management, or a related field. ~2–5...Full time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer ID89384. Be the first to apply!
