Lead Pyspark Developer
Luxoft
Surveillance Enhancement Program (SEP) is a multi-year program (MYP) to build and enhance the surveillance platform to detect potential instances of market misconduct, market manipulation and market abuse. The Bank met its commitment to FCA (Financial Conduct Authority, FCA) to meet Market Abuse Regulation ("MAR") requirements for critical data sources in September 2017. Following that, subsequent projects were initiated to further enhance, expand and complement the current coverage of the automated surveillance platform. This project will focus on Alerts enhancements, Strategic data sourcing from FM order and trade lakes, UI / workflow enhancements to meet regulatory obligations (i.e. Market Abuse Regulation, Dodd Frank). This also includes Control Room & Suitability Monitoring related data, UI and scenario enhancements.
Responsibilities
Develop software applications based on business requirements
Maintain software applications and make enhancements according to project specifications
Participate in requirement analysis, design, development, testing, and implementation activities
Propose new techniques and technologies for software development.
Perform unit testing and user acceptance testing to evaluate application functionality
Ensure to complete the assigned development tasks within the deadlines
Work in compliance with coding standards and best practices
Provide assistance to Junior Developers when needed.
Perform code reviews and recommend improvements.
Review business requirements and recommend changes to develop reliable applications.
Develop coding documentation and other technical specifications for assigned projects.
Act as primary contact for development queries and concerns.
Analyze and resolve development issues accurately.
Skills
Must have
10+ years of experience in data intensive Pyspark development ,Window Functions , Performance Optimization ,Spark Architecture,Salting ,Column Pruning
Must have experience in leading a Development Team
Proven experience as a Core Python developer
Good understanding of Gen AI
Experience developing Classes, OOPS, exception handling, parallel processing
Strong knowledge of DB connectivity, data loading , transformation, calculation
Extensive experience in Pandas/Numpy dataframes, slicing, data wrangling, aggregations
Lambda Functions, Decorators
Vector operations on Pandas dataframes /series
Application of apply map, apply, map functions
Concurrency and error handling data pipeline batch of size [1-10 gb]
Ability to understand business requirements and translate them into technical requirements
Ability to design architecture of data pipeline for concurrent data processing
Working with Git source control Service-orientated architecture, including the ability to consider integrations with other applications and services
Debugging application
Nice to have
Knowledge of web backend technology
Django, Python, PostgreSQL.
Apache Airflow
Atlassian Jira
Understanding of Financial Markets Asset Classes (FX, FI, Equities, Rates, Commodities & Credit), various trade types (OTC, exchange traded, Spot, Forward, Swap, Options) and related systems is a plus
Surveillance domain knowledge, regulations (MAR, MIFID, CAT, Dodd Frank) and related Systems knowledge is certainly a plus
Other
Languages
English: C2 Proficient
Seniority
Lead
- About the Role :We are seeking a Lead Software Engineer to join a leading organization. The role focuses on building robust Python backend... ...and eventdriven solutions.Key Responsibilities :- Design, develop, and maintain scalable Python backend applications using FastAPI...Suggested
- About the Role:We are looking for a skilled Lead Infrastructure Engineer to join our Technology team at Clarivate. In this role, you will... ...and troubleshoot Linux systems across production environments.- Develop automation solutions using Python and Bash to improve operational...Suggested
- Job Description :We are seeking a highly skilled and experienced Lead Linux Device Driver Developer to join our dynamic engineering team in Bangalore.This pivotal role offers the opportunity to work on advanced consumer SoC platforms within a leading semiconductor organization...Suggested
- Backend LeadAbout the Role : We are looking for an experienced Backend Lead to take ownership of core server-side systems powering a modern, scalable digital platform.The role involves leading backend architecture, designing highly reliable APIs and services, improving system...SuggestedContract workTemporary work
- Lead OIC Consultant | Oracle Integration Cloud & VBCS Start Date Starts Immediately... ...solution teams have successfully implemented cloud solutions and developed tools that seamlessly integrate with ERP systems, reducing manual...SuggestedFull timeImmediate start
- ...build, and deliver top-notch technology products. As a Senior Lead Software Engineer at JPMorganChase within the D igital Communications... ...and applications. Job responsibilities D esign and develop scalable, fault-tolerant microservices and APIs that support rule...
- ...adventure where you can push the limits of what's possible. As a Lead Software Engineer at JPMorganChase within the API Marketplace in... ...approaches to build solutions or breakdown technical problems Develops secure and high-quality production code, and reviews and debugs...
- ...first. We value collaboration, curiosity and commitment. As a Lead Software Engineer at JPMorgan Chase within the Accelerator business... ...Build scalable solutions that avoid single points of failure. Develop secure code to protect customers and the company from malicious...Long term contractWork at office
- ...commercial and residential environments - kitchens, warehouses, hotel floors, homes - to capture the high-fidelity task data that the world’s leading Robotics labs use to train their foundation models. Read more . What you’ll do Lead Workforce & Operations: Drive day-to-day...Hourly payFull timeLocal areaShift work
- ...We are looking for a seasoned, highly technical Senior Salesforce Developer to lead the design, development, and implementation of complex customization within our Salesforce ecosystem. In this role, you will be the technical anchor—writing clean, scalable code, designing...Full time
- ...Max 60daysRole Summary :We are looking for a highly experienced Lead SDET with deep domain expertise in Retail Banking and strong hands... ...- Experience working within SAFe Agile environments.- Ability to develop comprehensive Test Strategy, Test Plans, Test Reviews, Estimations...Work at office
- Job Description:We are looking for an experienced GCP Application Support Lead with strong experience in cloud-based application support and production operations. The candidate will be responsible for managing application support teams, ensuring 24x7 production support, managing...Shift work
- Role Overview : Development of custom software solutions designed to meet specific business needs. Your typical day will involve coding, enhancing components, and collaborating with various teams to ensure the delivery of scalable and high-performing solutions using modern frameworks...
- Responsibilities : - Demonstrate strong understanding of enterprise contact centre architecture.- Understand how agent desktop, routing, skills, queues, and telephony integrations function within enterprise ecosystems.- Support transition planning and migration efforts toward...Hybrid work
- Job Overview : We are seeking a highly experienced and hands-on Senior C#/.NET - Technical Lead to join our team in Bangalore.Key Responsibilities : - Lead the architecture, design, and development of scalable enterprise web applications using .NET (Core/Framework)- Drive end...
- Position : Senior Full Stack Engineer/ Technical Lead .NET Core | Azure | React | AILocation : BangaloreJob Summary :We are looking for... .... The ideal candidate will be responsible for designing, developing, and operating secure, scalable enterprise applications in a global...
- About the Role : Zinnov is Hiring - Tech Lead Development, ServiceNow on behalf of a leading global healthcare and medical technology... ...customizations.- Serve as the first line of technical escalation for developers, mentor junior engineers, and communicate sprint progress, risks...
- Role Overview :As a Custom Software Engineering Lead, you will oversee the technical vision and architectural framework of bespoke software... ...and team productivity.- Mentor and support team members to develop their skills.Tech Stack & Requirements :- Proficiency in Kinaxis....
- Role Overview :We are looking for a Senior Manager DevOps to lead our DevOps function, cloud infrastructure, automation, CI/CD, platform reliability, and operational excellence. The role will be responsible for defining DevOps strategy, owning organization-wide objectives,managing...
- What We're Looking For :This lead-level role owns the design and operation of MLOps and DevOps platforms that support reliable model build, test, deployment, and release workflows. The position combines cloud-native infrastructure, container orchestration, automation, and production...
- ...Responsibilities Lead the design, development, customization, and optimization of Salesforce Sales Cloud solutions for opportunity... ...management, sales automation, forecasting, and territory planning. Develop and enhance Salesforce Platform components using Apex, Lightning...Full time
- Security Testing | Technical Lead Start Date Starts Immediately CTC (ANNUAL) Competitive salary Competitive salary...Immediate start
- ...biopharma contract development and manufacturing. It also has a rapidly developing presence in highly specialised and molecular clinical diagnostic... ...20 years. Job Description POSITION TITLE Module Lead REPORTING TO: Manager REPORTING LOCATION: Bangalore...Full timeContract work
- ...biopharma contract development and manufacturing. It also has a rapidly developing presence in highly specialised and molecular clinical diagnostic... ...20 years. Job Description POSITION TITLE Module Lead REPORTING TO: Manager REPORTING LOCATION: Bangalore...Full timeContract work
- ...to work on a range of state-of-the-art solutions, where you can develop your knowledge and experience every day. Job Description... ...Have oversight into architecture, working with your technical leads to ensure the systems and solutions you develop are flexible, loosely...Full timeWork at officeLocal areaFlexible hours
- ...is flipping the script of the banking world, and we’re excited to lead the small business banking revolution. At Novo, we’re here... ...operating system that makes business banking as easy as iOS. We developed modern bank accounts and tools to help to save time and increase...Full timeFreelance
- ...take your software engineering career to the next level. As a Lead Software Engineer at JPMorganChase within the JPMorgan Wealth Management... ...software code development Gathers, analyzes, synthesizes, and develops visualizations and reporting from large, diverse data sets in...
- ...adventure where you can push the limits of what's possible. As a Lead Software Engineer- Quality Assurance at JPMorganChase within the... ...Experience working effectively with teams and stakeholders to develop relationships and achieve common goals Proficiency in a business...Contract work
- ...are proud of: The role As a Oracle Integration Cloud - Lead , you are passionate about experience innovation and eager to... ...Fusion, SAP Commerce, and other enterprise applications. ~ Develop data mappings , transformations, and orchestration logic for end...Full timeHybrid workRemote job
- ...GitHub and version control- SSO experience- ERP Integration experience- Agile development experienceKey Responsibilities:- Lead a team of Salesforce developers- Own technical design and delivery of Salesforce solutions- Design and implement scalable solutions on the Force.com...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead Pyspark Developer. Be the first to apply!
