(Senior) Data Engineer (ML, Big Data)
Luxoft
Provide End to End Development solution to one of the largest investment banking products by scaling up the latest technology stack by adopting the cloud based services under Client Life Cycle Management area
Responsibilities
Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights.
Constructing infrastructure for efficient ETL processes from various sources and storage systems.
Collaborating closely with Product Managers and Business Managers to design technical solutions aligned with business requirements.
Leading the implementation of algorithms and prototypes to transform raw data into useful information.
Architecting, designing, and maintaining database pipeline architectures, ensuring readiness for AI/ML transformations.
Creating innovative data validation methods and data analysis tools.
Ensuring compliance with data governance and security policies.
Interpreting data trends and patterns to establish operational alerts.
Developing analytical tools, programs, and reporting mechanisms.
Conducting complex data analysis and presenting results effectively.
Preparing data for prescriptive and predictive modeling.
Continuously exploring opportunities to enhance data quality and reliability.
Applying strong programming and problem-solving skills to develop scalable solutions.
Skills
Must have
10+ years of hands-on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines.
High proficiency in Scala/Java and Spark for applied large-scale data processing.
Expertise with big data technologies, including Spark, Data Lake, Delta Lake, and Hive.
Solid understanding of batch and streaming data processing techniques.
Proficient knowledge of the Data Lifecycle Management process, including data collection, access, use, storage, transfer, and deletion.
Expert-level ability to write complex, optimized SQL queries across extensive data volumes.
Experience with RDBMS and OLAP databases like MySQL, Redshift.
Familiarity with Agile methodologies.
Obsession for service observability, instrumentation, monitoring, and alerting.
Knowledge or experience in architectural best practices for building data lakes.
Nice to have
Passion for testing strategy, problem-solving, and continuous learning.
Willingness to acquire new skills and knowledge.
Possess a product/engineering mindset to drive impactful data solutions.
Experience working in distributed environments with global teams.
Other
Languages
English: C2 Proficient
Seniority
Senior
- Job Title : Data Engineer (GenAI)Location : Gurugram / Bangalore (Hybrid)Experience : 5+ YearsEmployment Type : Full-TimeMandatory Skills :- Python- PySpark- SQL- Data Engineering- GenAI, LLMs or RAG- Azure/AWS (Any Cloud)Job Responsibilities :- Build and optimize scalable...SeniorFull timeHybrid work
- Job Description :We are looking for an experienced Senior Data Engineer to build, maintain, and optimise scalable data platforms. The ideal candidate should have strong expertise in designing ETL/ELT pipelines, data modelling, cloud data warehouses, streaming systems, and data...Senior
- ...Role Overview We are looking for a Data Engineer to support and enhance the CS-AT platform, which processes large-scale machine log data (15–20 years) to enable predictive and proactive maintenance solutions. The role involves building and optimizing data pipelines, log...Senior
- Senior Data Engineer - Microsoft FabricExperience : 8+ YearsLocation : BangaloreNP : Immediate - 20 days(Serving)Work Mode : HybridRequired Skills :- Strong hands-on experience in ADF, Azure Synapse, Microsoft Fabric- Expertise in Python and PySpark- Experience in building scalable...SeniorImmediate start
- Job Details : - Job Title : Senior Data Engineer - Microsoft Fabric- Experience : 8+ Years- Location : Bengaluru- Employment Type : Full-TimeJob Summary : - We are seeking an experienced Senior Data Engineer with strong expertise in Microsoft Fabric, Azure Data Factory (ADF)...SeniorFull time
Rs 40 - Rs 45 per hour
...Roles & Responsibility Key Responsibilities: Building and optimizing big data pipelines, architectures, and datasets to handle large-scale data. Enhancing infrastructure for scalability, automation, and data delivery improvements. Developing real-time and batch...Senior- As a Senior Data Engineer on the Data Science team, you will be embedded with data scientists and ML engineers to build the data infrastructure that powers Veho's models, experiments, and operational decisions. You will spend most of your time in DBT and Databricks - designing...Senior
- ...Job Title: Senior Data Engineer (5+ Years Experience) Location: Bangalore/Hyderabad/Gurgaon/Indore/Raipur Experience: 5+ Years Employment Type: Full-time Job Summary: We are looking for a highly skilled Data Engineer with 5+ years of experience in building...SeniorFull time
- ...Java/Scala. 2. Proficiency in Spark/Flink/Hive/Hadoop/Azure/Databricks for batch and real-time processing. 3. Solid understanding of data pipeline, data warehousing and data modelling. 4. Experience with data visualization tools like QlikSense or PowerBI. 5. Experience...Senior
- We are looking for a highly skilled Senior Data Engineer to design, build, and operate scalable cloud-native data platforms for global customers. You will work on modern data architectures, large-scale ETL/ELT systems, streaming pipelines, and AI-driven data platforms using...SeniorHybrid workFlexible hours
- .... The transformative platform uses cutting-edge technology, big data, machine learning, and AI to seamlessly bring together OEMs, retailers... ...the automotive industry. This role goes beyond traditional data engineering and offers the opportunity to shape the enterprise data...Senior
- Designation : Senior Data EngineerExperience : 7 to 14 yearsQualification : BE/B tech, ME/M tech, Bsc, Msc.Location : Hyderabad, Bangalore and MumbaiJob Description : As a Senior Data Engineer, you will own the end-to-end data platform, from raw data ingestion through transformation...Senior
- Senior Data EngineerHumyn Labs & KGen - Full-timeMultimodal PipelinesAWS Data LakeGPU ValidationAI Training DataHumyn Labs converts human... ...where it is built.Role Overview:We are looking for a Senior Data Engineer to own, extend, and harden the multimodal data infrastructure...SeniorFull time
- ...traders of all levels from beginners to institutional-level clients. Let's talk about the cool stuff you do at Axi! As the Senior Data Engineer for the Customer Data Platform, you will play a pivotal role improving Axi’s data capabilities which deliver positive...SeniorLocal area
- ...of the world’s largest companies. Our Data Services practice sits at the centre of everything... ...Data Factory — where your broader data engineering experience will add immediate value.... ...skills; able to work directly with senior client-facing stakeholders ~ Experience...SeniorImmediate start
- ...safe, secure, and seamless for anyone logging in. The Auth0 Data Engineering Team Within Auth0, the Data Engineering team builds... ...sure everyone has access to consistent and reliable data The Senior Data Engineer Opportunity Reporting to the manager of the Data...SeniorFull timeHybrid workLocal areaWorldwide
- ...Manager Your role and responsibilities In this role, you will have the opportunity to design, build, and maintain scalable data engineering solutions for Operations Data & Automation initiatives. You will develop end-to-end ETL/ELT pipelines, data integrations, and data...SeniorFull time
- ...Join us and build an exceptional experience for yourself, and a better working world for all. EY- GDS - Data and Analytics – Senior – Forward Deployed Engineer As part of our EY-GDS AI & Data team, we help our clients solve complex business challenges with the help...SeniorLong term contract
- ...73 Strings is an innovative platform providing comprehensive data extraction, monitoring, and valuation solutions for the private... ...Broadhaven Ventures. About the Role: We are looking for a Senior Data Engineer to build and own the data pipelines, integrations, and...SeniorFull timeWork at office
- About the Role :Radancys Data Engineering team is seeking a Senior Data Engineer to join our Bangalore Product Engineering team and help build the next generation of our data platform, insights products, and AI-powered data agents. This role is part of our evolution from serving...Senior
- ...reliable, secure, and performs as intended. You’ll work closely with engineers and product partners to prevent defects early, strengthen... ...help teams ship with confidence. Key responsibilities - ✅ Data Engineering & Development • Design, develop, and maintain scalable...SeniorLocal area
- Job Description : Senior Data Scientist & Machine Learning EngineerRole Overview : We are seeking a highly experienced, versatile Senior Data Scientist & Machine Learning Engineer to bridge the gap between advanced data science, full-stack software engineering, and production...Senior
- ...Communication, Employee Groups and Celebration. Position Overview Purpose & Scope: As part of Ralph Lauren's Data Strategy initiative, we are seeking a Senior Data Engineer to lead the design and delivery of scalable data products that power analytics across commercial,...Senior
- ...Global Sustainability Business Proc Your role and responsibilities: In this role, you will lead and support enterprise-level data engineering initiatives across the Business Analytics division. You will be responsible for designing, developing, optimizing, and...SeniorFull timeHybrid work
- ...what runs the world. This Position reports to: R&D Principal Engineer Your role and responsibilities: At ABB's Common Technology Center in Bangalore, we are looking for an experienced Senior Data Engineer to drive the development of scalable cloud-based data platforms...SeniorFull time
- ...disciplines. About the team The mission of Roku’s Data Engineering team is to develop a world-class big data platform that empowers... ...to the success of new and existing initiatives. As a Senior Data Engineer on Roku's Content Data Engineering team , you...SeniorHybrid workWork at officeLocal areaRemote jobWorldwideMonday to ThursdayFlexible hours
- ...and a better working world for all. Job Description of Data Engineer Job description At EY, you’ll have the chance to build... ...Data Engineer – Technology Assurance As Risk Assurance Senior, you’ll contribute technically to Risk Assurance client engagements...SeniorFull time
- ...everyday experiences of living, working and playing remarkable - giving us back our most valuable asset, time. Who you are As a Senior Data Engineer at Metropolis, you will play a key role in shaping both customer-facing and business-facing data products that align with our...SeniorLocal areaAfternoon shift
- Senior Software Engineer -Data Engineer Position Description Founded in 1976, CGI is among the largest independent IT and business consulting services firms in the world. With 94,000 consultants and professionals across the globe, CGI delivers an end to end portfolio of...SeniorFull timeLocal areaShift work
- ...next generation of customer experiences, your work will help transform industries. We are proud of: The role As a Data Engineer , you are passionate about experience innovation and eager to push the boundaries of what’s possible. You bring 4+YEARS of experience...SeniorFull timeHybrid workRemote job
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to (Senior) Data Engineer (ML, Big Data). Be the first to apply!
