Infosys - Big Data Engineer - Apache Spark/Hadoop Ecosystem
infosys ltd
Education :- Bachelor's or master's degree in Computer Science, Information Technology, Engineering, or a related field.Experience and Technical Requirements :- Strong experience in big data engineering, data processing, and distributed computing.- Proficiency in Apache Spark, PySpark, and Hadoop ecosystem technologies.- Experience developing scalable data pipelines for batch and real-time data processing.- Strong programming skills in Python, Scala, Java, or similar languages.- Hands-on experience with Hive, HDFS, Kafka, and SQL.- Knowledge of ETL/ELT processes, data ingestion, transformation, and data integration.- Experience working with structured, semi-structured, and unstructured datasets.- Understanding of distributed systems, data partitioning, performance tuning, and optimization.- Familiarity with cloud-based data platforms such as AWS, Azure, or GCP.- Experience with Git, CI/CD pipelines, and workflow orchestration tools such as Airflow.Job Responsibilities:- Design, develop, and maintain scalable big data pipelines and processing frameworks.- Build efficient batch and streaming data workflows to process large volumes of data.- Develop data ingestion and transformation processes from multiple source systems.- Optimize Spark jobs and distributed data processing workloads for performance and scalability.- Implement data quality checks, validation, and error-handling mechanisms.- Integrate data from various databases, applications, and external sources.- Monitor pipeline performance and troubleshoot data processing issues.- Collaborate with data architects, analysts, and engineering teams to support data platform requirements.- Maintain technical documentation and follow data engineering best practices.Required Skills :- Apache Spark, PySpark, Hadoop, SQL, Python/Scala/Java, ETL, Big Data ProcessingSupported Skills :- Kafka, Hive, HDFS, Airflow, AWS/Azure/GCP, Data Pipeline Optimization, CI/CD (ref:hirist.tech)
Vacancy posted 5 hours ago
Similar jobs that could be interesting for youBased on the Infosys - Big Data Engineer - Apache Spark/Hadoop Ecosystem in Hyderabad vacancy
- ...DeliveryAbout the Role :We are seeking a Senior Solutions Architect - Data to lead customer engagements and deliver end-to-end data and AI... ...solution architecture skills with hands-on expertise in data engineering, Snowflake, modern data transformation frameworks, semantic...Suggested
- ...Platform Development (ITSM, CMDB, Integrations), Python, PySpark, Data Engineering, AI-Assisted EngineeringRole Overview : We are looking for a... ...CMDB, SLA data) on a distributed processing platform.- Optimize Spark jobs for performance and cost, including partitioning, join...Suggested
- Role Overview : We are looking for an experienced GCP Data Engineer to design, develop, and optimize scalable cloud-based data solutions. The... ..., data models, and analytics platforms using GCP, dbt, and Apache Airflow.Responsibilities : - Design, develop, and maintain scalable...Suggested
- ...Overview:We are looking for an experienced Data Architect with strong expertise in... ...end data architecture, design scalable data engineering and analytics solutions, and translate complex... ...and processing pipelines using SQL, Python, Spark, and relevant cloud-native services.-...Suggested
- ...Overview PepsiCo operates in an environment undergoing immense and rapid change. Big-data and digital technologies are driving business transformation that is unlocking new capabilities and business innovations in areas like eCommerce, mobile experiences and IoT. The key to...Big Data
- ...Req ID: 392456 NTT DATA strives to hire exceptional, innovative and passionate individuals... ...seeking a AWS Sagemaker/ Bedrock, Data Engineer, ETL, Snowflake to join our team in... ...We also offer clients access to a robust ecosystem of innovation centers as well as...Hybrid workWork at officeRemote jobFlexible hours
- ...We are an Artificial Intelligence (AI) focused product engineering company , providing our customers in healthcare, retail & e-commerce... ...hospitality sectors with cutting-edge products & solutions, harnessing Big Data Analytics, Vision Analytics, and IoT. Ever since our...Big DataFull time
- ...Quest Global is hiring for the role of Data Analyst - Tableau developer! Responsibilities of the Candidate: Tableau: Tableau dashboard development, including data modeling, complex calculations, performance optimization and troubleshooting. SQL & Data Analysis: nderstand...
- ...About the Team Data is at the foundation of DoorDash success. The Data Engineering team builds database solutions for various use cases including reporting, product... ...solutions using data lake/ Snowflake or similar ecosystem ~ Expert in Database fundamentals, SQL and performance...Local area
- ...Thermo Fisher Scientific seeks a Senior Data Engineer to design, develop, and maintain enterprise data pipelines and platforms supporting... ...experience, preferably AWS, GCP, or Azure ~ Experience with Apache Spark, Hadoop/Hive, Kafka, or similar distributed technologies ~ Strong...
- ...Johnson & Johnson seeks a Senior Principal Data Scientist to lead strategy and scale reusable machine learning, generative AI, optimization... ...Requirements ~ Master's degree in computer science, engineering, biostatistics, health economics, or quantitative discipline; PhD...
- ...Responsibilities : - Lead the planning and execution of Azure data engineering projects from inception to completion.- Manage project budgets,... ...programming languages such as Python, SQL, and Scala.- Knowledge of big data technologies and data modeling techniques.- Excellent...Big Data
- ...:- Own the technical architecture for data analytics and visualization applications... ...stack.- Lead, mentor, and grow a team of engineers across maintenance, new feature builds,... ..., or similar).- Experience with big-data tools (Spark, Dask) for scaling beyond single-node Pandas...Big DataShift work
- ...role :We are seeking a deeply technical, hands-on, and strategic Data Science Leader to spearhead complex predictive modeling,... ...data science lifecycle - from data sourcing, cleaning, feature engineering, and exploratory analysis to developing, validating, and deploying...Full time
- ...Artificial Intelligence (AI) focused product engineering company , providing our customers in... ...cutting edge products & solutions, harnessing Big Data Analytics, Vision Analytics, and IoT.... ...is a plus and Experience with Dataflow, Apache Beam, and Dataproc ● Designs, develops,...Big DataFull time
- ...We are an Artificial Intelligence (AI) focused product engineering company , providing our customers in healthcare, retail & e-commerce... ...hospitality sectors with cutting edge products & solutions, harnessing Big Data Analytics, Vision Analytics, and IoT. Ever since our...Big DataHybrid work
Rs 2 - 5 lakhs p.a.
Big Data Executive Secunderabad Hyderabad (Hybrid) Start Date... ...analysis to help solve real business problems Skill(s) required Hadoop Java Machine Learning Python SQL Earn certifications in these...Big DataInternshipHybrid workImmediate start- Role Overview:As a Data Scientist in our Applied ML team, you will sit at the intersection of financial innovation and advanced predictive... .... You will collaborate closely with product managers, software engineers, and business stakeholders to translate ambiguous financial data...Hybrid work
- ...responsibilities include optimizing sales business processes managing data and leveraging technology and coordinating efforts to improve... ...Attributes Functional Skills Trade Marketing Data Analysis Big Data Analytics Capability Development Customer Value...Big DataFull timeRelocation package
Dr Reddy's Laboratories Limited
Hyderabad4 days ago - ...Medtronic seeks a Senior AI Data Science Engineer to design and deploy enterprise-grade AI applications combining full-stack engineering, large language models, and workflow automation. The role is based in Hyderabad, India with flexible work arrangements and requires expertise...Flexible hours
- ...Amgen seeks a Senior Data Scientist to develop computational and AI/ML methods for perturbation biology and functional genomics within the ARIA organization. The role focuses on analyzing high-throughput screening data, building predictive models, and collaborating with experimental...
- Data Engineer (Snowflake & Analytics Engineering)Core Mission : Architecting the Modern Data StackPrimary Engine : Snowflake + dbt / MatillionLocation... ..., ensuring a 360-degree data view.2. Orchestration & Cloud Ecosystem :- Azure Integration : You will leverage Azure Data Factory...Hybrid workWork at officeImmediate start
- Job Title : Data Engineer (Snowflake & dbt/Matillion Specialist)Location : Hyderabad (On-site, 5 Days/Week)Experience : 3-5 YearsEducation : B.E./B.Tech (Mandatory)About the Role :We are seeking an experienced and results-driven Data Engineer to join our growing Data Engineering...
- Description :Data Engineering Solution ArchitectCore Mission : Designing the Future of Fact-Based... ...high-performance Snowflake and Databricks ecosystems. You will own the transition from... ...technology roadmap.- Complexity : Solve "Big Four" level problems with the agility of...Hybrid workImmediate start
- Job Title : Lead Data Scientist / Senior Data ScientistLocation : Hyderabad, Telangana (On-... ...technologies powered by a proprietary Biometric Engine. We are seeking a highly skilled Data... ...SQL/NoSQL databases, data warehouses, and Big Data technologies.- Familiarity with cloud...Big DataFull timeWork at officeWorking Monday to Friday
- Role Overview : We are seeking a seasoned Data Scientist to join our advanced analytics team in Hyderabad, Bangalore, or Pune. In this... ...will collaborate closely with cross-functional product teams, engineering leads, and senior stakeholders to translate ambiguous business problems...Long term contractHybrid work
- Job Title: Senior Data Engineer Company: Lead Data Analytics US Based company Location: Hyderabad (Max Budget: 22 LPA) | Ahmedabad (Max Budget: 20 LPA) Work Mode: 5 Days Work From Office (No Hybrid/Remote) Notice Period: Immediate to 15 Days Max Location Criteria: Strictly...Full timeWork at officeLocal areaImmediate startRelocation
- ...We are an AI-led platform-driven Digital Engineering and Enterprise Modernization partner combining... ...innovators across the healthcare ecosystem. Our disruptors mindset commitment to... ...We are looking for an experienced Senior Data Engineer to design modernize and optimize...Full timeHybrid workWork at officeFlexible hours
Persistent Systems
Hyderabad3 days ago - ...Title: Big Data Engineer Location: Hyderabad Exp: 5Yrs Job Description: Utilize your software engineering skills including Java Spark Python Scala to Analyze disparate complex systems and collaboratively design new products and services Integrate new...Big DataFull time
NR Consulting
Hyderabad3 days ago - ...Senior Data Engineer Job ID#: 26-2233 Location: Hyderabad/Vijaywada India. Who We Are: Since our inception back in 2006, Navitas... ...an understanding of the Databricks Lakehouse architecture, Apache Spark, Delta Lake, and the Medallion Architecture (Bronze, Silver, and...Full timeLocal area
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Infosys - Big Data Engineer - Apache Spark/Hadoop Ecosystem. Be the first to apply!
