Big Data Engineer

Skill: Big Data Engineer (Pyspark + SQL) Exp: 5 to 8 years Loc: Pune & Chennai

JD Lead end to end analysis of largescale multisource datasets using Big Data technologies driving insights for strategic and operational decision making Design develop and optimize complex analytical queries using Starburst Trino Presto to enable fast federated access across enterprise data sources Build manage and optimize analytics workflows in Databricks using SQL PySpark and Python for scalable data processing Partner closely with Data Engineering Architecture and Platform teams to influence data models analytics architecture and performance optimization Translate ambiguous business problems into structured analytical approaches and clearly communicate findings trends and recommendations to senior stakeholders Define and enforce best practices for data quality validation reconciliation and governance within analytics solutions Mentor junior and midlevel analysts providing technical guidance code reviews and analytical best practices Support executive reporting dashboards and advanced adhoc analysis as required Contribute to continuous improvement of analytics processes standards and reusable frameworks710 years of experience as a Data Analyst Senior Data Analyst working in largescale enterprise data environments Strong handson experience with Big Data ecosystems Hadoop Hive Spark distributed data platforms Proven experience using Starburst TrinoPresto for complex high performance analytical queries Extensive experience with Databricks including Databricks SQL Spark based analytics and notebook development Expert level SQL skills query optimization performance tuning complex joins Strong proficiency in Python and or PySpark for data analysis and transformation Solid understanding of data warehousing data modeling and analytics architecture concepts Excellent analytical problem solving and stakeholder communication skills Experience working on cloud platforms such as AWS Azure or GCP Exposure to BI and visualization tools Power BI Tableau or equivalent Knowledge of enterprise data governance metadata management and data quality frameworks Experience working in Agile DevOps delivery models Strong ownership and ability to lead quality outcomes end to end Clear communication and stakeholder management Mentoring mindset and collaboration across QA Dev and DevOps teams