Spark developer jobs in work from home

 ...large data sets using distributed processing tools like Akka and Spark. Understanding and critically reviewing existing data...  ...multiple products and features we have. 7+ years of experience in developing highly scalable Big Data pipelines. In-depth understanding of... 

 ...with clients to understand their data needs and requirements, and develop customized solutions to normalize, cleanse, and fine-tune their...  ...techniques. Familiarity with ETL tools and frameworks such as Apache Spark, Talend, or Informatica. Excellent communication and... 

 ...Responsibilities: Data Pipeline Architecture: Design, develop, and optimize end-to-end data pipelines to extract, transform, and load (ETL) data from various sources into our data warehouse. Ensure data quality, reliability, and performance throughout the pipeline. Data... 

Description:Senior Data Engineer (Spark & Lakehouse)Location: Remote, India (Preferred: Bangalore/Pune)Experience: 6+ YearsDomain: Data Engineering / Big DataAbout the Role:We are seeking a Senior Data Engineer to drive the development of our next-generation Data Lakehouse... 

 ...Develop scalable data collection, storage, and distribution platform to house data from vendors, research providers, exchanges, PBs, and web-scraping. Make data available to systematic & fundamental PMs, and enterprise functions: Ops, Risk, Trading, and Compliance. Develop... 

 ...data modeling Knowledge of data warehousing concepts Experienced building data pipelines and micro services Experience with  Spark ,  Airflow and other streaming technologies to process incredible volumes of streaming data A willingness to accept failure, learn... 

We are looking for a Senior Data Engineer with expertise in SQL, Python, AWS, and containerization to build and maintain a scalable data platform . The role involves working with web scraping, data pipelines, and DevOps practices while collaborating with cross-functional...

 ...Collaborate with other engineers, customers to brainstorm and develop impactful data products tailored to our clients. Leverage AI and machine learning techniques to integrate intelligent features into our offerings. Develop, and optimize end-to-end data pipelines... 

 ...fundamental portfolio managers (PMs) , as well as enterprise teams including Ops, Risk, Trading, and Compliance . The role involves developing internal data products and analytics while optimizing data pipelines. Key Responsibilities: Web Scraping & Data Acquisition... 

 ...Job Responsibilities: Develop and maintain data pipelines for large-scale data processing. Work with streaming data technologies, including...  ...experience in real-time/streaming data pipeline development using Apache Spark, StreamSets, Apache NiFi, or similar frameworks.... 

 ...building solutions on Data Science platforms and applications • 2-3 years' Experience with Databricks/Snowflake, AWS Data Science services, Spark, Python, Scala, R, Neo4J, Linux and ML processes. • Experience with Life Sciences related business process domains, such as R&D,... 

 ...office applications , including regulatory reporting, settlements, and reconciliation . We are seeking an experienced ETL/Java Developer to join a mixed Luxoft/client team , focusing on new functionality development . This role provides an excellent opportunity to... 

 ...documentation, supporting audits, and collaborating with stakeholders to drive compliance initiatives.   ⭐Key Responsibilities: × Develop and maintain comprehensive documentation, including network topology diagrams, configuration reviews, and firewall standards... 

 ...our data lake house environment. You will leverage your 10+ years of expertise to develop complex data pipelines, ensure data quality, and drive innovation using Databricks and Apache Spark. Responsibilities: ~ Design and implement scalable and robust data pipelines... 

 ...Requirement: Architect, develop, and maintain scalable and secure data pipelines to process structured and unstructured data from diverse sources. Collaborate with data scientists, BI analysts and business stakeholders to understand data requirements. Optimize data... 

 ...machine-learning techniques and with sensitivity to the limitations of the techniques. Select, acquire and integrate data for analysis. Develop data hypotheses and methods, train and evaluate analytics models, share insights and findings and continues to iterate with... 

 ...permissions and clusters. Collaborate with data engineers to optimize Spark jobs and Delta Lake usage. Ensure proper data ingestion,...  ..., S3, ADLS) Collaboration & Documentation Collaborate with developers, data scientists, and DevOps engineers. Maintain up-to-date... 

Description :- Design, develop, and maintain data ingestion and transformation pipelines in an Azure-based environment.- Work with Kafka for...  ...analytical systems.- Implement data transformation logic using Spark and Python.- Optimize performance of large-scale data pipelines in... 

 ...Experience in leading a team of engineers and a good attitude toward learning the domain and implementation- Strong exposure and expertise in Spark (Primary), Scala/Java (Scala Primary), Airflow Orchestration and AWS.- Finalizing the scope of the system and delivering Big Data... 

 ...Description We are seeking a skilled Snowflake Developer / Data Engineer to join our team in India. The ideal candidate will be responsible for designing and implementing robust data solutions using Snowflake, ensuring high performance and reliability of our data infrastructure... 

 ...Snowflake Data Engineer to join our dynamic team in Mumbai. The ideal candidate will have 10+ years of hands-on experience in designing, developing, and implementing robust data solutions on the Snowflake platform. This role demands a deep understanding of data warehousing... 

 ...such as Redshift, BigQuery, Snowflake, or Delta Lake- Build real-time and batch data pipelines using tools like Apache Airflow, Kafka, Spark, and DBT- Ensure data governance, lineage, quality, and observability- Collaborate with stakeholders to define data strategies,... 

 ...applications. Build and enhance applications using technologies such as React, Python, AWS, Snowflake, and other relevant tools. Develop chatbots or agents leveraging Agentic AI and LLM technologies. Conduct advanced data analytics to extract insights and drive... 

 ...Engineering, or a related field. ~4 to 15 years of experience in data engineering, with a focus on Microsoft technologies. ~ Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF), Azure Databricks, and PySpark. ~ Implement and optimize big... 

 ...Role Overview: We are looking for an experienced Data Architect to design, develop, and optimize enterprise data solutions on Microsoft Azure. Key Responsibilities: Lead architecture, design, and implementation of Data Warehouse and Data Lake solutions. Work across... 

 ...reliable ETL solutions • Design dimensional data models (star schema following the Kimball approach) and partner with BI teams to develop LookML models that support analytics and reporting • Collaborate with business analysts to gather requirements, validate data quality... 

 ...Position: Data Visualization Developer Location: Remote in Canada Employment Type: Full-Time or Contract (Flexible, based on candidate preference) About Collavion Collavion is a forward-thinking recruitment and talent solutions partner serving diverse sectors... 

 ....   Job Title: Principal Clinical Data Science Programmer [Elluminate] Location : Bangalore/Remote What you will be doing: Develop, implement and maintain date review and data cleaning capabilities for sponsors led Phase I-IV clinical trials using sponsors technologies... 

 ...functional teams to understand business requirements and transform them into technical solutions. Data Pipeline Development: Design, develop, and maintain robust data pipelines for efficient data ingestion, transformation, and storage. Optimize and automate data... 

 ...translate requirements into a scalable data architecture. Design, develop, and own robust, efficient, and scalable data models in Snowflake...  ...-Haves) Experience with large-scale data processing using Spark. Direct experience with advanced Snowflake features (e.g.,...