Data Engineer - PySpark
HARP Technologies And Services Pvt Ltd
Data Engineer - Pyspark ETL Hadoop Python SQLWe are seeking a skilled PySpark Developer to join our Data Engineering team. The ideal candidate should possess strong hands-on experience in PySpark, Python, and SQL, with expertise in developing scalable data processing solutions and ETL pipelines. The candidate will be responsible for designing, developing, optimizing, and maintaining high-performance data applications that support enterprise analytics and business intelligence initiatives.Key Responsibilities :- Design, develop, and maintain scalable ETL/ELT pipelines using PySpark.- Develop data processing solutions for large-scale structured and unstructured datasets.- Build, optimize, and troubleshoot Spark applications for performance and scalability.- Perform data extraction, transformation, validation, and loading activities.- Develop reusable and efficient data engineering frameworks and components.- Collaborate with business analysts, data architects, and cross-functional teams to understand data requirements.- Ensure data quality, integrity, and governance across data platforms.- Monitor and support production data pipelines and resolve performance bottlenecks.- Participate in code reviews and follow best practices for development and deployment.Required Skills :- Strong hands-on experience with PySpark- Proficiency in Python- Strong knowledge of SQL- Experience with ETL/Data Engineering projects- Good understanding of Data Warehousing concepts- Experience with Spark SQL, DataFrames, and distributed data processing- Knowledge of performance tuning and optimization techniques in Spark- Experience working in Linux/Unix environments- Strong analytical and problem-solving skills (ref:hirist.tech)
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Data Engineer - PySpark in Pune vacancy
- ...Engineers in this role build and operate scalable data pipelines on an AWSbased data platform working closely with data science teams to productionise models. They support multiregion operations troubleshoot production issues and act as a technical bridge between data science...DataFull timeLocal areaFlexible hours
NielsenIQ
Punemore than 2 months ago - ...you can push the limits of whats possible. As a Lead Software Engineer at JPMorganChase within the Consumer and Community Banking you are... ...development and maintenance of robust scalable cloud-based data processing pipelines and infrastructure ensuring adherence to engineering...DataFull time
JPMorganChase
Punemore than 2 months ago - ...to 30 days..!!Key Responsibilities & Skillsets : Common Skillsets : - 3+ years of experience in analytics, Pyspark, Python, Spark, SQL and associated data engineering jobs.- Must have experience with managing and transforming big data sets using pyspark, spark-scala, Numpy...DataImmediate start
- Job Summary : We are hiring a Python PySpark Data Engineer to develop scalable data processing applications and high-performance data pipelines. The ideal candidate should have strong programming expertise in Python and PySpark along with experience in distributed data processing...Data
- Role Overview :Our client is looking for a highly skilled Senior Data Engineer to design, build, and optimize scalable data platforms and... ...based data platforms- Large-scale data processing- Databricks & PySpark ecosystems- Real-time streaming systems- AI-driven data engineering...Data
- Role Overview :As a Senior Data Engineer, you will serve as a technical cornerstone in building and scaling robust data pipelines that power... ...Design and implement scalable data processing pipelines using PySpark to handle large-scale datasets, ensuring optimal performance and...DataLong term contractHybrid work
- Senior Data Engineer (PySpark) :Location :Pune (Work from Office - 5 Days a Week).Job Summary :We are looking for a highly skilled and experienced Senior Data Engineer to join our team in Pune. The ideal candidate will have 8-10 years of experience in designing, building, and...DataWork at office
- ...Description Sr Data Management Analyst - Informatica (ETL)- Advanced Data Engineer Advanced Data Engineer is responsible for implementing the data solutions for transformation and migration for ERP deployments by utilizing established processes. The Advanced Data Engineer...DataFull time
Honeywell
Punemore than 2 months ago - Data Pipeline Development & Operations :- Design, build, and operate scalable and reliable... ...legacy code and data pipelines to PySpark for improved performance and scalability-... ...minimize disruption during code transitionsData Engineering Excellence :- Implement data quality...DataImmediate start
- ...PRIVATE LIMITED is a technology services firm specializing in data engineering, cloud transformation, and advanced analytics. We partner with... ...Responsibilities:- Develop and optimize scalable ETL pipelines using PySpark and Apache Spark to process large volumes of structured and...DataHybrid work
- ...Script and have 6+ years of experience with big data frameworks like Spark, Hadoop or Kafka.- You excel in Apache Spark/PySpark fundamentals.- You excel in using stream and... ...data governance practices.- Data Engineering Expertise: You have advanced skills in designing...DataFull timeWork from homeFlexible hours
- Job Title : Databricks on AWS and PySpark EngineerJob SummaryWe're seeking an experienced Databricks on AWS and PySpark Engineer to join our team. The ideal candidate will have a strong... ..., building, and maintaining large-scale data pipelines and architectures using...Data
- Job Title : Azure Data Engineer - PySpark - RemoteJob Description : The engineer will be working within an established Microsoft Fabric data platform, supporting the delivery of a large number of data transformation workflows. The platform is already built - the focus is on...DataFull timeWork at officeRemote jobWorking Monday to Friday
- Job Title : Azure Data EngineerLocation : PuneExperience : 6+ YearsJob Summary :We are seeking... ...a skilled and experienced Azure Data Engineer to design, develop, and optimize scalable... ...have strong expertise in Azure Databricks, PySpark, Azure Data Factory (ADF), Azure Data Lake...Data
- ...UBS- 16th May F2F drive Databricks & Data Engineer Please support for immediate to serving June Candidates B3 Databricks & Data Engineer profiles for F2F drive on Saturday 16 May As we have received the demand today only pls support by adding profiles here for Sat...DataFull timeImmediate start
TalentOla
Punemore than 2 months ago - ...are looking for a skilled Reltio MDM Developer to join our team. The ideal candidate should have hands-on experience in Reltio Master Data Management (MDM), data integration, data quality, and customer/domain data management solutions.Key Responsibilities :- Design, develop...Data
- ...guidelines Ability to communicate clearly and logically and present developed features from time to time10 Years of Experience in ETL Tools Data Warehouse Experience Good in SQL Job Scheduling . Abinitio Architect Certified in Abinitio DI Minimum 4 years experience in Leading...DataFull time
Virtusa
Punea month ago - ...This role targets a Technical Lead (6-8 years) who bridges the gap between high-level data science and robust AI Engineering. You will move beyond static modeling to architect Agentic AI systemsautonomous workflows that reason act and automate complex analytics at scale....DataFull time
Virtusa
Punemore than 2 months ago - ...Lead Software Engineer Tier 2 Having 6 years of experience Lead Software Engineer to design develop and maintain NLP and text analytics... ...Agentic AI / AI-powered analytics. You will collaborate with data engineers AI specialists and product teams to build scalable intelligent...DataFull time
Virtusa
Punemore than 2 months ago - ...project Implementing best practices Co-ordinate with Cross- functional teams Preferred Skills & Experience: Mandatory Skills: Data Warehousing Bigdata Testing ETL testing Preferably Experience with SQL ETL tools Data Warehousing Tool Experience: JIRA ADO Test...DataFull time
NewVison
Punemore than 2 months ago - 6 years of experience in the following: Design develop and maintain robust ETL pipelines to extract transform and load data from multiple sources into enterprise data systems. Build and enhance reporting and analytics solutions using Microsoft Power BI and the broader Microsoft...DataFull time
Trigent Software Private Limited
Punea month ago - ...Join Roche where every voice matters. The Position As the Data Transfer Delivery Lead you will take on a vital role driving the... ...technical challenges in close collaboration with system owners engineering teams and third-party vendors. Act as the primary point of...DataFull time
Roche
Pune18 days ago - ...Datasphere professional with an AI-first mindset to build intelligent data-driven enterprise solutions. Requirements Required... ...Generation (RAG) Large Language Models (LLMs) Prompt Engineering AI Agent Development Data Integration & ETL Lakehouse Architecture...DataFull time
Vrinda International
Pune6 days ago Rs 20 - 30 lakhs p.a.
...Role Senior Databricks Engineer Role Summary Senior Databricks Engineer with strong... ...in developing optimizing and maintaining data engineering solutions on the Databricks platform... ...have deep experience with Apache Spark (PySpark) Databricks notebooks Delta Lake and Spark...DataFull timeNewVison
Pune5 days ago- Role- Data Engineer Status- On contract(1 Year) Experience Range: Salary Range Design build and maintain ETL/ELT data pipelines using Azure Data Factory Azure Synapse GCP Dataflow BigQuery or similar cloud-native tools. Develop optimized and scalable BigQuery...DataFull timeContract work
Randstad India
Pune18 days ago - ...Job Description: Strong experience in Power BI Desktop & Service DAX Power Query and data modelling Handson with SQL Server (SSMS) SSIS SSRS and complex SQL queries Expertise in Azure Data Factory and Microsoft Fabric for data integration Experience...DataFull time
Acme Services Private Limited
Punemore than 2 months ago - ...Description We are seeking a highly skilled Senior Data Engineer with 812 years of experience in designing developing and managing scalable data platforms and enterprise data solutions. The ideal candidate will have strong expertise in data engineering cloud technologies...DataFull time
Zensar Technologies
Pune1 day ago - ...Kinaxis RapidResponse / Maestro combined with deep experience in data modeling integrations and advanced supply chain planning. The... ...Data Modeling Hierarchy metrics planning models 10 SQL / Data Engineering Querying pipelines ETL 10 Supply Chain Domain Demand Planning...DataFull time
TalentOla
Pune17 days ago Rs 15 - 20 lakhs p.a.
...Developer Role Summary: Senior Databricks Engineer with strong expertise in developing optimizing and maintaining data engineering solutions on the Databricks... ...should have deep experience with Apache Spark (PySpark) Databricks notebooks Delta Lake and Spark performance...DataFull timeNewVison
Pune3 days ago- ...Job Description Data Management & Analytics Developer Business Data & Analytics Role Summary We are seeking a DMA Developer... ...reliability and usability. Key Responsibilities Data Engineering & Development Design develop and maintain SAS (9 and Viya) and...DataFull time
SAS
Punemore than 2 months ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer - PySpark. Be the first to apply!
