Spark developer jobs in work from home

 ...and support. The ideal candidate would have extensive experience developing and supporting a DW service comprised of multiple Data...  ...Expertise in at least two scripting language (Python, Scala, Spark,Unix or Java) is a mandatory. ~ Proficiency in BI/Visualization... 

 ...help our team deliver data products, analytics, and models quickly and independently. The role is cross-functional and responsible for developing resilient data pipelines and infrastructure for evaluating and deploying data science models. The ideal candidate should... 

 ...Key Responsibilities: Design, develop, and maintain robust ETL/ELT pipelines to collect, clean, transform, and load data from diverse...  ...S3, Glue) or Azure (Data Factory, Synapse) Familiarity with tools like Apache Spark , Kafka , Informatica , or dbt .... 

Job description Qualitest India Private Limited is looking for Data Engineer to join our dynamic team and embark on a rewarding career journey. # Liaising with coworkers and clients to elucidate the requirements for each task. # Conceptualizing and generating infrastructure...

 ...quality, integrating advanced statistical and machine learning models, and driving measurable business outcomes. Architect and develop pipelines with robust validation, quality enforcement, and efficient workflows for model deployment. Partner with data scientists... 

 ...the Role:** We are seeking a skilled and experienced Apache NiFi Developer/Data Engineer to join our team. The ideal candidate will have a...  ...:**   Experience with big data technologies (e.g., Hadoop, Spark). Knowledge of containerization and orchestration tools (e.g.... 

 ...Responsibilities: Design, develop, and maintain data infrastructure, databases, and data pipelines Develop and implement ETL processes to extract, transform, and load data from various sources Ensure data accuracy, quality, and accessibility, and resolve data-related... 

Job Title: Data Engineer Experience: Min 7 - 15 years in SQL & C# (Below 7 years experienced candidates please refrain from applying) Location: Permanent remote Notice period: Immediate joiners preferred (Max 15 days) Work timings- Mon-Fri, 2:00pm to 11:00pm IST ...

 ...Experience in developing REST API services using one of the Scala frameworks. Ability to troubleshoot and optimize complex queries on the Spark platform. Expert in building and optimizing big data, data, and ML pipelines, architectures, and data sets. Knowledge in modeling... 

Must be strong with Python for ML pipelines specifically with Pytorch and scikit-learn AWS is required, building pipelines within Should have a background in LLM (langchain, agents, extensive prompt engineering) The 'strong additional requirements' below are required. ...

 ...~ Proficiency in SQL, Python, and modern data modeling practices ~ Hands-on experience with batch and streaming frameworks (e.g., Spark, Kafka, Kinesis, Hadoop) ~ Proven track record of building and maintaining real-time and batch data pipelines at scale ~ Deep understanding... 

 ...documentation, supporting audits, and collaborating with stakeholders to drive compliance initiatives.   ⭐Key Responsibilities: × Develop and maintain comprehensive documentation, including network topology diagrams, configuration reviews, and firewall standards... 

 ...Requirement: Architect, develop, and maintain scalable and secure data pipelines to process structured and unstructured data from diverse sources. Collaborate with data scientists, BI analysts and business stakeholders to understand data requirements. Optimize data... 

 ...Key Responsibilities Develop & Optimize ETL Pipelines Build robust, scalable data pipelines using Azure Data Factory (ADF), Databricks...  .... Big Data Processing Utilize Azure Databricks, Apache Spark, and Delta Lake for distributed and large-scale data processing.... 

 ...candidate will be responsible for providing Training on designing, developing, and delivering advanced training programs for professionals and...  ...guidance on big data tools and platforms such as Hadoop, Spark, and cloud-based data solutions. Develop training materials,... 

 ...Responsibilities Analyze large datasets to extract actionable insights. Develop and implement predictive models using machine learning algorithms...  .... ~ Knowledge of big data technologies such as Hadoop or Spark is a plus. ~ Excellent problem-solving skills and ability to... 

 ...machine-learning techniques and with sensitivity to the limitations of the techniques. Select, acquire and integrate data for analysis. Develop data hypotheses and methods, train and evaluate analytics models, share insights and findings and continues to iterate with... 

 ...environments Collaborate with threat researchers and engineers to develop and deploy effective ML solutions Conduct model evaluations...  ...GCP ~ Understanding of distributed computing like Ray, Apache Spark ~ SQL (PostgreSQL, MySQL) and NoSQL databases (MongoDB, Elasticsearch... 

 ...community and make an impact supporting the machine learning models of some of the world's largest brands. Key Responsibilities: Develop complex, original question-and-answer pairs based on advanced topics in your area of expertise. Ensure questions involve multi-... 

 ...role in shaping next-gen data systems.What you'll do :- Design & develop robust data pipelines (ETL) using the latest Big Data tech- Optimize...  ...(Product, Data, Design, ML)- Work with modern tools : Apache Spark, Databricks, SQL, Python/Scala/JavaExperience in Scala is mandatoryWhat... 

 ...skilled Data Engineers to design, build, and manage robust data pipelines that power Agentic AI solutions on AWS. The role focuses on developing efficient ETL/ELT workflows, ensuring data quality, security, and scalability to support AI/ML model training, inference, and... 

 ...Experience in leading a team of engineers and a good attitude toward learning the domain and implementation- Strong exposure and expertise in Spark (Primary), Scala/Java (Scala Primary), Airflow Orchestration and AWS.- Finalizing the scope of the system and delivering Big Data... 

 ...queries (especially for Amazon Redshift).- Experience with Apache Spark (preferably on AWS EMR) for big data processing.- Proven...  ...code (e. g., Terraform, CloudFormation).Key Responsibilities : - Develop, maintain, and optimize complex SQL queries, primarily for Amazon... 

Description :- Design, develop, and deploy end-to-end data pipelines on AWS cloud infrastructure using services such as Amazon S3, AWS Glue...  ...data processing and transformation workflows using Apache Spark, and SQL to support analytics and reporting requirements.- Build... 

 ...Role Overview: We are looking for an experienced Data Architect to design, develop, and optimize enterprise data solutions on Microsoft Azure. Key Responsibilities: Lead architecture, design, and implementation of Data Warehouse and Data Lake solutions. Work across... 

 ...help design and build a no-code interface that enables users to develop predictive models for complex supply chain challenges. In this role...  .... Solid understanding of distributed computing frameworks (Spark, Dask, Ray, etc.). Familiarity with Docker, Kubernetes, and CI... 

 ....   Job Title: Principal Clinical Data Science Programmer [Elluminate] Location : Bangalore/Remote What you will be doing: Develop, implement and maintain date review and data cleaning capabilities for sponsors led Phase I-IV clinical trials using sponsors technologies... 

Job Description We are seeking a senior data professional to join the Pre-Clinical Research Lab line of business for a large pharmaceutical client. The successful candidate will be part of a global team and will be responsible for designing and building conceptual and physical...

 ...Position: Data Visualization Developer Location: Remote in Canada Employment Type: Full-Time or Contract (Flexible, based on candidate preference) About Collavion Collavion is a forward-thinking recruitment and talent solutions partner serving diverse sectors... 

 ...transform and load data into our data warehouse and analytics platforms, ensuring data quality and consistency throughout the pipeline. Develop scalable data architectures using cloud-native technologies and modern data platforms, contributing to architectural decisions that...