Azure Databricks Architect - Apache Spark
https://vysystems.com/
Job Description :Key Responsibilities :- Design and architect scalable data engineering solutions using Azure Databricks.- Develop and optimize ETL/ELT pipelines for enterprise data processing.- Build and maintain data lake/lakehouse solutions using Azure services.- Develop data transformation workflows using PySpark/Scala, Python, and SQL.- Implement Delta Lake architecture to support reliable and high-performance data processing.- Collaborate with business and technical stakeholders to deliver robust data solutions.- Ensure data quality, governance, security, and performance optimization.- Implement CI/CD pipelines and follow DevOps best practices for deployment and version control.Required Skills :- 12+ years of experience in Data Engineering, Data Architecture, or related roles.- Strong expertise in Azure Databricks.- Hands-on experience with Apache Spark (PySpark/Scala).- Proficiency in Python and SQL.- Experience with Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS).- Strong understanding of ETL/ELT, Delta Lake, and Data Modeling.- Experience with Git, CI/CD, and DevOps practices.- Excellent analytical, problem-solving, and stakeholder management skills. (ref:hirist.tech)
Vacancy posted 28 days ago
Similar jobs that could be interesting for youBased on the Azure Databricks Architect - Apache Spark in Hyderabad vacancy
- ...Engineering : - Develop cloud-native solutions leveraging Microsoft Azure services.- Work with services including : 1. Azure App Services.... ..., and Retrospectives.- Collaborate with Product Managers, Architects, UX Designers, and Engineering teams.- Contribute to technical documentation...SuggestedContract workWorldwide
- ...cloud-native engineering on any of the cloud hyperscalers such as Azure, AWS, or GCP including their AI/ML services such as Azure OpenAI... ...well as container orchestration (Kubernetes, Docker), Big Data, Databricks, CI/CD at platform scale, and distributed systems.- 1+ years of...SuggestedHybrid workWork at officeWork from home
- ...Commerce Platform will play a critical role in architecting and delivering next-generation e-... ...services (OpenAI, Anthropic, AWS SageMaker, Azure AI) and apply emerging patterns such as... ...well as Big Data technologies such as Apache Spark, Flink, Kafka, Amazon EMR, and Hive.-...SuggestedFull timeHybrid workWork at officeWorldwide
- ...catalog integration. CKA (Certified Kubernetes Administrator) or Certified Backstage Associate (CBA). AWS certification (Solutions Architect / DevOps Engineer). Additional Information All your information will be kept confidential according to EEO guidelines....SuggestedFull time
- ...technologies and industry best practices. Technologies You have experience with several of the following technologies: · AWS / Azure · Kubernetes / Docker · Terraform · GitHub Actions / Jenkins · Prometheus / Grafana · Datadog / Splunk · OS – Linux /...SuggestedFull timeHybrid work
- ...operating, and optimizing cloud platform services. These services must be secure, scalable, and automated. The primary focus is on Microsoft Azure, but the environment spans multiple clouds, including AWS and GCP. You will lead complex technical programs, advances reusable...Long term contractFull timeHybrid work
- ...architecture with Infrastructure-as-Code, cloud migration from on-premises infrastructure and modern DevOps practices.Key Skills : Cloud Architect, AWS, Terraform, Terragrunt, Ansible, Linux, Database Cluster, AWS Landing Zone, Control Tower, Custom Automation, PCI, ISO...Hybrid workWorldwide
- ...Qualifications:- Bachelors degree in Computer Science, Information Systems, Engineering, or a related technical field.- AWS Certified Solutions Architect or Developer certification.- Experience with microservices architecture and serverless computing.- Understanding of networking and...
- Role Overview:We are seeking a forward-looking Architect to design, build, and scale next-generation applications leveraging cloud-native architectures, AI/ML capabilities, and modern engineering practices.The individual owns technical decisions and direction for their portfolio...
- ...Responsibilities : - Create and configure data processing and web application infrastructure using Azure services. That includes setting up storage accounts, event hubs, AKS, Databricks components, gateways, databases, service accounts both on prod and non-prod environments.-...
- Platform Design Engineer IIExperience: 3-6 yrsWhat you will do:- 3-4 yrs hands-on experience in the board architecture and product design documents, schematic design and reviews, layout guidelines and reviews, board bring-up and debug of issues- Hands-on experience in board ...
- ...environments. ~ Strong understanding of cloud-based and distributed systems. ~ Experience with distributed storage systems such as S3 and Apache Iceberg at scale is a strong plus. ~ Proven experience in leading and mentoring engineering teams. ~ Familiarity with ServiceNow...Full timeWork at officeRemote jobFlexible hours
- ...and file-format optimisation. Optimise Spark/Glue/EMR workloads for performance, cost efficiency... ...OpenTofu, CloudFormation, GitHub Actions, Azure DevOps, Jenkins, Docker, Kubernetes/EKS,... ...to mentor engineers, collaborate with architects/security/SRE teams, and adopt newer AI-...
- ...using Microsoft Fabric and the Azure data ecosystem. The role... ...Pipelines, Dataflows, Notebooks, Spark, SQL endpoints, and Power BI semantic... ..., Microsoft Purview, Synapse, Databricks, and enterprise APIs.... ...mentor engineers, collaborate with architects/security/SRE teams, and adopt...
- .... EY-Consulting - Data and Analytics - AWS Data Platform Architect / Engineering Manager - / Manager / Lead Architect EY's Consulting... ...Lead CI/CD and Infrastructure-as-Code adoption using GitHub, Azure DevOps, Jenkins, Terraform/OpenTofu, CloudFormation and policy-...Long term contract
- ...application deployment.- Build and manage cloud infrastructure on AWS, Azure, or Google Cloud Platform.- Develop Infrastructure as Code (IaC)... ...):- AWS Certified DevOps Engineer Professional- AWS Solutions Architect- Microsoft Certified: Azure DevOps Engineer Expert- Certified...Full timeHybrid work
- Role Overview : We are looking for an experienced Automation Architect to design and drive scalable infrastructure and cloud automation solutions... ...compliance activities.- Build automation that works across AWS, Azure, VMware, and API-driven environments.- Integrate Ansible...Hybrid work
- Role Overview : We are seeking an Identity Platform Engineer to build, enhance, and operate enterprise identity solutions that support secure access across employees, applications, and technology platforms.This role focuses on turning identity management into a scalable, automated...
- Job Summary : As a Python Developer with Databricks, you will be responsible for developing and maintaining scalable data pipelines, managing cloud environments on Azure, and ensuring smooth integration with APIs.The ideal candidate will be proficient in Python, Databricks (...
- About the Role : We are looking for GCP / Java Developers with strong hands-on experience in Java, Spring Boot, REST APIs, Microservices and Google Cloud Platform (GCP). The ideal candidate should have experience building and deploying cloud-native applications using modern ...
- Key Responsibilities:- Design, develop, and maintain integrations in OIC.- Build and enhance BI Publisher (BIP) reports, data models (SQL), bursting, and eText/XML outputs (including bank/payment file formats).- Develop SQL for Finance and Procurement modules.- Troubleshoot ...
- Role : Senior Solution Architect Google Cloud (Vertex AI, GCP)Company : Auxo AIExperience : 1015 YearsPositions : 1Location : Hyderabad / Mumbai / Bengaluru / GurugramWork Mode : Hybrid 3 Days Office, 5 Days WorkingAbout the Role : Auxo AI is looking for a Senior Solution Architect...Hybrid workWork at officeImmediate start
- ...are looking for a skilled PySpark Developer with experience in Azure Databricks (ADB) and Azure Data Factory (ADF) to join our team. The... ...PySpark Developer or similar role with a strong understanding of Apache Spark internals.- Hands-on experience with Azure Databricks (ADB)...
- We are looking for an experienced Azure Data Engineer with 8 to 12 years of experience in designing, developing, and maintaining scalable... .... The ideal candidate should have strong expertise in Azure Databricks, Azure Data Factory, ADLS, SQL, Python/PySpark, and ETL processes...
- ...services in the Managed Services model, which includes the following:- Support day-to-day administration and operations of Wiz.io and Azure cloud security platforms.- Monitor cloud security posture, security findings, misconfigurations, and policy violations across Azure environments...Hybrid work
- Position: Identity Platform Engineer (Sailpoint)Overall/Total Experience: 5-8 years onlyLocation: HyderabadWorking Days: 5 Days Working from officeNotice Period Requirement: 30 Days (Maximum)Client's Company Size: Startup / Small EnterpriseRole & Responsibilities:- Design, implement...
- ...who is passionate about building and sustaining platforms for scale and delivering long-term impact. This title may also be called Architect.Key Areas of Responsibility : - Define and drive platform architecture strategy focused on scalability, modularity, security, and resilience...Long term contractRemote job
- Designation : Azure API Lead / ArchitectExperience : 10 Years Location : Across Straive (Hybrid / Remote)Job Description :- Worked as a Technology Architect in the Integration/API space with experience in Azure API management leading client engagements and working on driving...Hybrid workRemote job
- ...Engineering team. The ideal candidate will be responsible for designing, deploying, and supporting cloud infrastructure across AWS and Azure using Infrastructure as Code (IaC) principles. This role involves building and maintaining CI/CD pipelines, automating operational...
- ...application performance. Collaborate with architects, business stakeholders, and engineering... ...Cloud Technologies AWS Microsoft Azure Google Cloud Platform (GCP) Docker... ...Azure Data Factory (ADF) Snowflake Databricks DevOps & Monitoring GitLab CI/CD...Contract work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Azure Databricks Architect - Apache Spark. Be the first to apply!
