Azure Databricks Architect - Apache Spark
https://vysystems.com/
Job Description :Key Responsibilities :- Design and architect scalable data engineering solutions using Azure Databricks.- Develop and optimize ETL/ELT pipelines for enterprise data processing.- Build and maintain data lake/lakehouse solutions using Azure services.- Develop data transformation workflows using PySpark/Scala, Python, and SQL.- Implement Delta Lake architecture to support reliable and high-performance data processing.- Collaborate with business and technical stakeholders to deliver robust data solutions.- Ensure data quality, governance, security, and performance optimization.- Implement CI/CD pipelines and follow DevOps best practices for deployment and version control.Required Skills :- 12+ years of experience in Data Engineering, Data Architecture, or related roles.- Strong expertise in Azure Databricks.- Hands-on experience with Apache Spark (PySpark/Scala).- Proficiency in Python and SQL.- Experience with Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS).- Strong understanding of ETL/ELT, Delta Lake, and Data Modeling.- Experience with Git, CI/CD, and DevOps practices.- Excellent analytical, problem-solving, and stakeholder management skills. (ref:hirist.tech)
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Azure Databricks Architect - Apache Spark in Hyderabad vacancy
- ...Summary We are seeking a highly skilled and hands-on Databricks Solution Architect with deep expertise in designing, building, and optimizing... ...streaming data pipelines using Delta Live Tables (DLT), Apache Spark, Spark SQL, Python, and Scala . Build, optimize, and maintain...SuggestedContract workRemote job
- Job Title : Azure Data EngineerLocation : Pune, Hyderabad, BangaloreExperience : 7 to 12 years.Skills : Databricks, Spark/ Pyspark/ Python, ETL, ADF/ AirflowNotice Period : Immediate to 45 DaysJob Description :- Expert level skills writing and optimizing complex SQL.- Experience...SuggestedImmediate start
- ...implement scalable data architectures using databricks lakehouse, ensuring alignment with... ...driving best practices across delta lake, spark optimization, and workload performance tuning... ...with cloud platforms, preferably microsoft azure including azure databricks, data factory,...SuggestedFull time
- ...seeking an experienced Resident Data Architect with deep expertise in the Databricks Lakehouse Platform to lead... ...Expertise in cloud platforms such as AWS, Azure, or Google Cloud . ~ Strong... ...Lakehouse Architecture ~ Delta Lake ~ Apache Spark ~ Data Warehousing ~ Data...SuggestedWorldwide
- ...hands-on Senior Consultant – Databricks with deep technical expertise... ...using Delta Live Tables (DLT), Spark SQL, Python, or Scala. Optimize... ...Strong understanding of AWS, Azure, or GCP cloud ecosystems.... ...Data Engineer, Machine Learning, or Solutions Architect) preferred....SuggestedContract workRemote job
- ...We’re seeking a hands-on experience in Databricks with deep technical expertise in building and... ...pipelines using Delta Live Tables (DLT), Spark SQL, Python, or Scala. Optimize ETL, streaming... .... Strong understanding of AWS, Azure, or GCP cloud ecosystems. Experience with...Full time
Rs 8 - 20 lakhs p.a.
...Mandatory Skills: Python, PySpark, Etl, Sql, Azure Databricks, Azure Datafactory. Primary Roles... ...Drive technical discussion with client architect and team members ● Orchestrate the... ...hands-on experience in SQL, Python and Spark (PySpark) ● Candidate must have experience...Full timeWeekday work- Job Title : Azure Data Engineer (7+ Years Experience)Location : Hyderabad - HybridExperience... ...integration solutions using Azure Databricks, PySpark, and Spark SQL.- Design and implement data lakes... ..., T-SQL.3. Big Data Technologies :- Apache Spark- Delta Lake- Data Lake...Full time
- ...modern data platforms on Microsoft Azure? We're looking for experienced Microsoft Azure Databricks Engineers to join a high-... ...ELT solutions using PySpark and Spark SQL.- Develop and maintain data... ...Collaborate with Data Engineers, Architects, Data Scientists, and business stakeholders...
- ...wide data management.Roles & Responsibilities :- Architect and maintain robust, scalable data pipelines using Databricks, Spark, and Delta Lake, enabling efficient batch and... ...technologies such as Databricks, PySpark, SparkSQL Apache Spark, AWS, Python, SQL, and Scaled Agile...
- Job Description :Key Responsibilities :- Strong hands-on experience with Apache Kafka/ Confluent Platform and Confluent Cloud.- Proven experience in supporting developer teams in Kafka-based environments.- Solid understanding of Kafka architecture : topics, partitions, brokers...Weekday work
- ...Azure Architect – FTE or C-H For 20- plus years Centroid has helped complex companies accelerate their digital transformation by implementing and managing Cloud-based business solutions. We specialize in advisory, deployment, migration, integration and managed services....Full timeRemote job
- Role : ARCHITECT - AzureLocation : Any where in India.Job Description :Responsibilities :- Hands... ...CI/CD pipelines (GitHub Actions, Azure Pipelines) specifically for containerized... ...sessions.- Support data workloads, including Databricks deployments integrated with cloud and Kubernetes...
- ...development, microservices architecture, and Azure infrastructure. The role involves... ...Collaborate with architects and product teams on solution design and... ...Kubernetes, Docker Apache Kafka / Event-Driven Architecture...
- Remote/WFH opportunity.Experience : 5+ yrs experience.Job Description :Mandatory Skills :- Strong experience in :1. Azure Data Factory (ADF)2. Azure Databricks3. Apache Spark4. Lakeflow Declarative Pipelines5. SQL6. PySpark- Hands-on experience with Azure cloud platform and data...Remote jobWork from home
- ...Type: Full Time Experience: 10-15 Years Shift Timings: 11.30 am to 8:00 pm (OR) 12:00 pm to 8.30 pm Mode: WFO Azure Security Architect The Azure Security Architect is responsible for designing, implementing, documenting and managing security solutions on the...Full timeFlexible hoursShift work
- ...accelerate their cloud adoption and modernize legacy infrastructure through robust, data-driven frameworks.Role Overview :As an Azure DevOps Architect, you will serve as the technical authority for cloud-native infrastructure and automation strategies. You will work closely...Hybrid work
- ...:We are hiring for the role of Team Lead Azure Data Engineer. Location - Hyderabad/Ahmedabad... ...roles.- Expertise in Python, SQL, Spark, Kafka, and Airflow.- Proven experience with... ...GCP).- Hands-on experience with Snowflake, Databricks, or Delta Lake.- Strong understanding of data...Hybrid work
- ...Engineer to design and develop scalable data pipelines on the Azure Databricks platform. You will be responsible for transforming raw data into... ...data models.Technical Skills Required : - Core : Python, PySpark, Spark SQL.- Platform : Azure Databricks (Workflows, Unity Catalog,...
- Job Summary :We are seeking an experienced MSBI Developer with Azure expertise to design, develop, support, and optimize enterprise data integration and business intelligence solutions. The ideal candidate should have strong hands-on experience in Microsoft SQL Server BI Stack...
- ...industry standards.Design and develop scalable, robust, and efficient AI/ML architectures.Design and deploy AI/ML solutions using Microsoft Azure services such as Azure Machine Learning, Azure OpenAI Service, Azure AI Services, and Azure Kubernetes Service (AKS).Identify and...
- ...updated with the latest technologies and best practices. Professional & Technical Skills: - Must To Have Skills: Proficiency in Microsoft Azure IaaS.- Good To Have Skills: Experience with Microsoft Azure PaaS, Python (Programming Language).- Strong understanding of cloud...
Rs 80000 - Rs 100000 per month
...Description Venpa Global Technologies is looking for an Azure AKS Engineer to join our innovative team in Mumbai. This contract position offers a unique opportunity to work remotely, contributing to projects that are pivotal in the Human Resources industry. As an Azure...Contract workRemote job- Role : Azure Cloud EngineerHigh-Level Job Description : We are looking for an experienced Azure Cloud Engineer with strong expertise in... ...(such as Azure Administrator Associate or Azure Solutions Architect Expert) are preferred.Experience : 5+ years Location : Hyderabad...
- ...design and deliver scalable, resilient, and maintainable applications. Design and implement cloud native solutions using Microsoft Azure, including App Services, Functions, Service Bus, Event Grid, and containerized workloads, ensuring high availability and fault tolerance...Full timeContract work
- Optum is hiring Azure Cloud Engineer at Hyderabad location who can join us on short notice.Primary Skills : Build and maintain production... ...automation-first solutions across the stack- Work on ADF and Databricks-based deployments- Debug real-world production issues and improve...Immediate start
- ...users.- Automate data workflows and orchestrate data pipelines using Apache Airflow to improve efficiency and reduce manual effort.- Optimize... ...data processing performance and scalability by leveraging Apache Spark and PySpark to handle large-scale data transformations.- Monitor...
- ...performance.- Implement orchestration workflows using Cloud Composer (Apache Airflow) or similar tools.- Develop and maintain batch and real-... ...Experience in designing ETL/ELT pipelines.- Knowledge of Apache Spark or PySpark.- Experience with version control tools such as Git.-...
- ...1. BigQuery2. Cloud Dataflow3. Cloud Storage4. Cloud Composer5. Apache Airflow6. Dataform/dbt7. Pub/Sub2. Programming : - Advanced expertise... ...distributed computing and processing frameworks, e.g., Apache Spark, PySpark, or Apache Beam.4. Data Engineering Concepts : Solid understanding...
- ...delivered according to established standards, with a focus on both Azure and AWS platforms. The Cloud Delivery engineer will be... ...seamless delivery of cloud resources on both Azure and AWS. Architecting, Designing, Developing and Implementing AI models and algorithms...Hybrid work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Azure Databricks Architect - Apache Spark. Be the first to apply!
