Lead Pyspark Developer
Luxoft
Surveillance Enhancement Program (SEP) is a multi-year program (MYP) to build and enhance the surveillance platform to detect potential instances of market misconduct, market manipulation and market abuse. The Bank met its commitment to FCA (Financial Conduct Authority, FCA) to meet Market Abuse Regulation ("MAR") requirements for critical data sources in September 2017. Following that, subsequent projects were initiated to further enhance, expand and complement the current coverage of the automated surveillance platform. This project will focus on Alerts enhancements, Strategic data sourcing from FM order and trade lakes, UI / workflow enhancements to meet regulatory obligations (i.e. Market Abuse Regulation, Dodd Frank). This also includes Control Room & Suitability Monitoring related data, UI and scenario enhancements.
Responsibilities
Develop software applications based on business requirements
Maintain software applications and make enhancements according to project specifications
Participate in requirement analysis, design, development, testing, and implementation activities
Propose new techniques and technologies for software development.
Perform unit testing and user acceptance testing to evaluate application functionality
Ensure to complete the assigned development tasks within the deadlines
Work in compliance with coding standards and best practices
Provide assistance to Junior Developers when needed.
Perform code reviews and recommend improvements.
Review business requirements and recommend changes to develop reliable applications.
Develop coding documentation and other technical specifications for assigned projects.
Act as primary contact for development queries and concerns.
Analyze and resolve development issues accurately.
Skills
Must have
15+ years of experience in data intensive Pyspark development, Window Functions, Performance Optimization, Spark Architecture, Salting, Column Pruning
Must have experience in leading a Development Team
Proven experience as a Core Python developer
Good understanding of Gen AI
Experience developing Classes, OOPS, exception handling, parallel processing
Strong knowledge of DB connectivity, data loading , transformation, calculation
Extensive experience in Pandas/Numpy dataframes, slicing, data wrangling, aggregations
Lambda Functions, Decorators
Vector operations on Pandas dataframes /series
Application of apply map, apply, map functions
Concurrency and error handling data pipeline batch of size [1-10 gb]
Ability to understand business requirements and translate them into technical requirements
Ability to design architecture of data pipeline for concurrent data processing
Working with Git source control Service-orientated architecture, including the ability to consider integrations with other applications and services
Debugging application
Nice to have
Knowledge of web backend technology
Django, Python, PostgreSQL.
Apache Airflow
Atlassian Jira
Understanding of Financial Markets Asset Classes (FX, FI, Equities, Rates, Commodities & Credit), various trade types (OTC, exchange traded, Spot, Forward, Swap, Options) and related systems is a plus
Surveillance domain knowledge, regulations (MAR, MIFID, CAT, Dodd Frank) and related Systems knowledge is certainly a plus
Other
Languages
English: C2 Proficient
Seniority
Lead
- ...Senior frontend role at Genpact in Bengaluru. Needs 5–8 years. What you'll do Lead front-end development on Next.js and React, remaining hands-on with pages, components, routing, data fetching and rendering Own the front-end approach, including component architecture...SuggestedFull time
- ...Role – Lead Software Engineer – MS Power Platform/ Dynamics CRM Where the job is based : Bangalore Experience: 8+ Years... ...Nonprofit and Public Sector organisations . We encourage all our developers and designers to embrace the latest technologies, frameworks,...SuggestedWork at office
- ...customer value and winning GTM motions , this role is for you. Benefits of Joining CloudSEK We provide an environment where you can develop and enhance your skills while delivering meaningful work that matters. You’ll be rewarded a competitive salary as well as a full...SuggestedFull timeWork at officeFlexible hours
- ...Greetings from Encora Innovation Labs! A Leading SaaS Technology Product Engineering Company! Please find the below... ...Build customized solutions that support business requirements by developing in Visualforce, Apex, Java, and AJAX Experience with Classic...SuggestedFull timeWorldwide
- ...engineering with AI consulting. We collaborate with some of the world’s leading data-rich organizations across sectors such as CPG-retail, BFSI,... ...solutions Facilitate in Technical Aspects Design, develop and evolve highly scalable and fault-tolerant distributed components...SuggestedFull time
- ...electronic payment technology industry. Verifone has one of the leading electronic payment solutions brands and is one of the largest providers... ...experience, innovation, and an ambitious spirit. Whether it’s developing the next platform of secure payment systems or searching for new...Work at officeWorldwide
- ...Engineering Manager as a member of the Development team in India, the Lead Software Engineer is responsible for delivering quality and... ...that our customers have today. This role is responsible for Developing Software Solutions by learning information needs, discussing with...Full timeWork at office
- ...Lead Developers act as the primary point of contact for our clients, ensuring that teams are successful in their delivery. In this role, you will bring a strategic mindset to projects and spend time collaborating and negotiating with clients to bring a technical vision to...Full timeHybrid workShift work
- ...collaborative workplace where they can thrive. We offer industry-leading benefits packages to promote a creative and inclusive culture.... ...social good, we’d love to hear from you. Role: Lead Backend Developer Job Summary: Utilize advanced Python technologies and frameworks...Full timeFlexible hours
- ...to the delivery path and the recovery posture, for our trading, banking, payments and settlement products. Key Responsibilities Lead and support a DevOps pod of 10+ engineers. You are accountable for the roadmap, the engineering standards and the cloud budget. You create...
- ...is on the lookout for a talented Security Compliance Operations Lead to manage and mature a security assurance function within Security... ...visibility into security posture and risk reduction. Lead and develop the Security Compliance Operations function, including control assurance...Full timeWork at office
- ...commercial and residential environments - kitchens, warehouses, hotel floors, homes - to capture the high-fidelity task data that the world’s leading Robotics labs use to train their foundation models. Read more . What you’ll do Lead Workforce & Operations: Drive day-to-day...Hourly payFull timeLocal areaShift work
- ...next generation of customer experiences, your work will help transform industries. We are proud of: The role As a Lead Developer - Sitecore , you are passionate about experience innovation and eager to push the boundaries of what’s possible. You bring 6+ YEARS...Full timeHybrid workRemote job
- ...generation of customer experiences, your work will help transform industries. We are proud of: The role As a C++, C Lead Developer , you are passionate about experience innovation and eager to push the boundaries of what’s possible. You bring 8+ YEARS of...Full timeHybrid workRemote job
- ...Job Overview: The Lead Salesforce Developer within the BTT (Business Technology and Transformation) team will collaborate with other developers, architects, admins, QA and release and Engineering managers to deliver new innovative solutions on the Salesforce platform. Toast...Hybrid workWork at office
- ...required for enterprise trust, as we bring Work AI to every employee, in every company. About the Role: Glean is seeking a Lead Salesforce Developer – Revenue Systems to provide hands-on technical leadership for our Salesforce platform and revenue systems. You will own the...Long term contractFull timeHybrid workWork at officeFlexible hours
- ...gives us a competitive advantage. Job Description: Design Lead with Cloud (GCP) (Experience - 8 to 12 Years) • Strong data... ...Cloud Platform skills (as this is where the solutions are being developed/deployed) • Must have good Experience/knowledge on GCP components...Local area
- ...Science, Engineering, or related field. ~ Proven track record leading GCP projects with measurable outcomes (e.g., reduced ETL latency... ...GDPR), and cost management in GCP. Key Responsibilities: Develop, test, and maintain data pipelines and ETL/ELT processes using...Local area
- ...the promise of AI. What is The Role Elastic is looking for a Lead Salesforce Engineer who combines deep architectural intuition,... ...enterprise integration practices. Where the work runs on Salesforce, develop using Lightning Web Components (LWC), Apex, SOQL/SOSL, Flows, and...Full timeLocal areaFlexible hours
- ...services to support Kubernetes and containers. WHAT YOU'LL DO Lead the design, development, and deployment of Portworx's Kubernetes-... ...in every design and implementation decision. Design and develop robust, well-defined APIs that integrate seamlessly with the frontend...Full timeWork at officeFlexible hours
- ...Lead Software Engineer Position Description: As a Lead Software Engineer you will join provide technical leadership to the group responsible for architecting, developing, and owning the Zeta Marketing Platform. You will collaborate with Engineers, Product Managers...
- ...adventure where you can push the limits of what's possible. As a Lead Software Engineer at JPMorgan Chase within the Commercial &... ...approaches to build solutions or breakdown technical problems Develops secure and high-quality production code, and reviews and debugs code...
- ...adventure where you can push the limits of what's possible. As a Lead Software Engineer at JPMorganChase within the Commercial &... ...responsibilities ~ Execute creative LLM assisted software solutions, design, develop, and troubleshoot LLM powered applications and services (e.g.,...Full time
- ...build, and deliver top-notch technology products. As a Senior Lead Software Engineer at JPMorganChase within the Consumer and Community... ...business and its technical teams, contractors, and vendors Develops secure and high-quality production code, and reviews and debugs code...For contractorsLocal area
- DevOps Engineer - Lead Position Description Design and manage AWS infrastructure using Terraform or CloudFormation. Build and improve... ...’s strategy and direction. Your work creates value. You’ll develop innovative solutions and build relationships with teammates and...Full time
Rs 34.86 lakh p.a.
...scalable, reliable, and cost-efficient data platforms processing billions of records Lead the design, development, and optimization of large-scale data pipelines using Spark, PySpark, Python, and SQL Drive platform modernization initiatives, including incremental processing...Full time- ...Senior QA and testing role at Societe Generale in Bengaluru. What you'll do Lead Software Engineer - Test Automation Creating, daring, innovating, and taking action are part of our DNA What you need ~ Test automation Job details Location: Bengaluru Education...Full time
- ...Senior software engineering role at Societe Generale in Bengaluru. What you'll do Lead Software Engineer - Java Full Stack Angular Creating, daring, innovating, and taking action are part of our DNA What you need Java Angular Job details Location: Bengaluru...Full time
- Senior QA and testing role at Societe Generale in Bengaluru. What you'll do Whether you’re joining us for a period of months, years or your entire career, together we can have a positive impact on the future Creating, daring, innovating, and taking action are part of...Full time
- ...Mid-level software engineering role at Aliaxis (Ashirvad Pipes) in Bengaluru. Needs 3+ years. What you'll do Lead multiple projects from conception to completion in accordance with deadlines VAVE execution, Cost Reduction experience What you need AutoCAD GD&...Full time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead Pyspark Developer. Be the first to apply!
