Senior Data Engineer
Rs 40 - Rs 45 per hourFull-time
Skyleaf
Roles & Responsibility
Key Responsibilities:
- Building and optimizing big data pipelines, architectures, and datasets to handle large-scale data.
- Enhancing infrastructure for scalability, automation, and data delivery improvements.
- Developing real-time and batch processing solutions using Kafka, Spark, and Airflow.
- Ensuring data governance, security compliance, and high availability.
- Collaborating with product, business, and analytics teams to support data needs.
Tech Stack:
- Big Data Tools: Spark, Kafka, Databricks (Delta Tables), ScyllaDB, Redshift
- Data Pipelines & Workflow: Airflow, EMR, Glue, Athena
- Programming: Java, Scala, Python
- Cloud & Storage: AWS
- Databases: SQL, NoSQL (ScyllaDB, OpenSearch)
- Backend: Spring Boo
Requirements
Key Responsibilities:
- Building and optimizing big data pipelines, architectures, and datasets to handle large-scale data.
- Enhancing infrastructure for scalability, automation, and data delivery improvements.
- Developing real-time and batch processing solutions using Kafka, Spark, and Airflow.
- Ensuring data governance, security compliance, and high availability.
- Collaborating with product, business, and analytics teams to support data needs.
Tech Stack:
- Big Data Tools: Spark, Kafka, Databricks (Delta Tables), ScyllaDB, Redshift
- Data Pipelines & Workflow: Airflow, EMR, Glue, Athena
- Programming: Java, Scala, Python
- Cloud & Storage: AWS
- Databases: SQL, NoSQL (ScyllaDB, OpenSearch)
- Backend: Spring Boo
Vacancy posted more than 2 months ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Engineer. Be the first to apply!
