Sign up to access all features of our service
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Lead Pyspark Developer

Luxoft

Surveillance Enhancement Program (SEP) is a multi-year program (MYP) to build and enhance the surveillance platform to detect potential instances of market misconduct, market manipulation and market abuse. The Bank met its commitment to FCA (Financial Conduct Authority, FCA) to meet Market Abuse Regulation ("MAR") requirements for critical data sources in September 2017. Following that, subsequent projects were initiated to further enhance, expand and complement the current coverage of the automated surveillance platform. This project will focus on Alerts enhancements, Strategic data sourcing from FM order and trade lakes, UI / workflow enhancements to meet regulatory obligations (i.e. Market Abuse Regulation, Dodd Frank). This also includes Control Room & Suitability Monitoring related data, UI and scenario enhancements.

Responsibilities

Develop software applications based on business requirements

Maintain software applications and make enhancements according to project specifications

Participate in requirement analysis, design, development, testing, and implementation activities

Propose new techniques and technologies for software development.

Perform unit testing and user acceptance testing to evaluate application functionality

Ensure to complete the assigned development tasks within the deadlines

Work in compliance with coding standards and best practices

Provide assistance to Junior Developers when needed.

Perform code reviews and recommend improvements.

Review business requirements and recommend changes to develop reliable applications.

Develop coding documentation and other technical specifications for assigned projects.

Act as primary contact for development queries and concerns.

Analyze and resolve development issues accurately.

Skills

Must have

15+ years of experience in data intensive Pyspark development, Window Functions, Performance Optimization, Spark Architecture, Salting, Column Pruning

Must have experience in leading a Development Team

Proven experience as a Core Python developer

Good understanding of Gen AI

Experience developing Classes, OOPS, exception handling, parallel processing

Strong knowledge of DB connectivity, data loading , transformation, calculation

Extensive experience in Pandas/Numpy dataframes, slicing, data wrangling, aggregations

Lambda Functions, Decorators

Vector operations on Pandas dataframes /series

Application of apply map, apply, map functions

Concurrency and error handling data pipeline batch of size [1-10 gb]

Ability to understand business requirements and translate them into technical requirements

Ability to design architecture of data pipeline for concurrent data processing

Working with Git source control Service-orientated architecture, including the ability to consider integrations with other applications and services

Debugging application

Nice to have

Knowledge of web backend technology

Django, Python, PostgreSQL.

Apache Airflow

Atlassian Jira

Understanding of Financial Markets Asset Classes (FX, FI, Equities, Rates, Commodities & Credit), various trade types (OTC, exchange traded, Spot, Forward, Swap, Options) and related systems is a plus

Surveillance domain knowledge, regulations (MAR, MIFID, CAT, Dodd Frank) and related Systems knowledge is certainly a plus

Other

Languages

English: C2 Proficient

Seniority

Lead

Vacancy posted 20 hours ago
Similar jobs that could be interesting for youBased on the Lead Pyspark Developer in Bangalore vacancy
  •  ...Senior frontend role at Genpact in Bengaluru. Needs 5–8 years. What you'll do Lead front-end development on Next.js and React, remaining hands-on with pages, components, routing, data fetching and rendering Own the front-end approach, including component architecture... 
    Suggested
    Full time

    Genpact

    Bangalore
    7 days ago
  •  ...Role – Lead Software Engineer – MS Power Platform/ Dynamics CRM   Where the job is based : Bangalore   Experience: 8+ Years...  ...Nonprofit and Public Sector organisations . We encourage all our developers and designers to embrace the latest technologies, frameworks,... 
    Suggested
    Work at office

    Kerv

    Bangalore
    6 hours ago
  •  ...customer value and winning GTM motions , this role is for you. Benefits of Joining CloudSEK We provide an environment where you can develop and enhance your skills while delivering meaningful work that matters. You’ll be rewarded a competitive salary as well as a full... 
    Suggested
    Full time
    Work at office
    Flexible hours

    CloudSEK

    Bangalore
    6 hours ago
  •  ...Greetings from Encora Innovation Labs! A Leading SaaS Technology Product Engineering Company! Please find the below...  ...Build customized solutions that support business requirements by developing in Visualforce, Apex, Java, and AJAX Experience with Classic... 
    Suggested
    Full time
    Worldwide

    Encora

    Bangalore
    6 hours ago
  •  ...engineering with AI consulting. We collaborate with some of the world’s leading data-rich organizations across sectors such as CPG-retail, BFSI,...  ...solutions Facilitate in Technical Aspects Design, develop and evolve highly scalable and fault-tolerant distributed components... 
    Suggested
    Full time

    Sigmoid

    Bangalore
    6 hours ago
  •  ...electronic payment technology industry. Verifone has one of the leading electronic payment solutions brands and is one of the largest providers...  ...experience, innovation, and an ambitious spirit. Whether it’s developing the next platform of secure payment systems or searching for new... 
    Work at office
    Worldwide

    Verifone

    Bangalore
    6 hours ago
  •  ...Engineering Manager as a member of the Development team in India, the Lead Software Engineer is responsible for delivering quality and...  ...that our customers have today. This role is responsible for Developing Software Solutions by learning information needs, discussing with... 
    Full time
    Work at office

    Energy Exemplar

    Bangalore
    6 hours ago
  •  ...Lead Developers act as the primary point of contact for our clients, ensuring that teams are successful in their delivery. In this role, you will bring a strategic mindset to projects and spend time collaborating and negotiating with clients to bring a technical vision to... 
    Full time
    Hybrid work
    Shift work

    Referrals Only

    Bangalore
    6 hours ago
  •  ...collaborative workplace where they can thrive. We offer industry-leading benefits packages to promote a creative and inclusive culture....  ...social good, we’d love to hear from you. Role: Lead Backend Developer Job Summary: Utilize advanced Python technologies and frameworks... 
    Full time
    Flexible hours

    Definitive Healthcare, India

    Bangalore
    6 hours ago
  •  ...to the delivery path and the recovery posture, for our trading, banking, payments and settlement products. Key Responsibilities Lead and support a DevOps pod of 10+ engineers. You are accountable for the roadmap, the engineering standards and the cloud budget. You create... 

    OpenFX

    Bangalore
    6 hours ago
  •  ...is on the lookout for a talented Security Compliance Operations Lead to manage and mature a security assurance function within Security...  ...visibility into security posture and risk reduction. Lead and develop the Security Compliance Operations function, including control assurance... 
    Full time
    Work at office

    Rockstar Games

    Bangalore
    6 hours ago
  •  ...commercial and residential environments - kitchens, warehouses, hotel floors, homes - to capture the high-fidelity task data that the world’s leading Robotics labs use to train their foundation models. Read more . What you’ll do Lead Workforce & Operations: Drive day-to-day... 
    Hourly pay
    Full time
    Local area
    Shift work

    Instawork

    Bangalore
    6 hours ago
  •  ...next generation of customer experiences, your work will help transform industries.   We are proud of:     The role   As a  Lead Developer - Sitecore , you are passionate about experience innovation and eager to push the boundaries of what’s possible. You bring 6+ YEARS... 
    Full time
    Hybrid work
    Remote job

    Valtech

    Bangalore
    6 hours ago
  •  ...generation of customer experiences, your work will help transform industries.   We are proud of:     The role   As a  C++, C Lead Developer , you are passionate about experience innovation and eager to push the boundaries of what’s possible. You bring 8+ YEARS of... 
    Full time
    Hybrid work
    Remote job

    Valtech

    Bangalore
    6 hours ago
  •  ...Job Overview: The Lead Salesforce Developer within the BTT (Business Technology and Transformation) team will collaborate with other developers, architects, admins, QA and release and Engineering managers to deliver new innovative solutions on the Salesforce platform. Toast... 
    Hybrid work
    Work at office

    Toast

    Bangalore
    6 hours ago
  •  ...required for enterprise trust, as we bring Work AI to every employee, in every company. About the Role: Glean is seeking a Lead Salesforce Developer – Revenue Systems to provide hands-on technical leadership for our Salesforce platform and revenue systems. You will own the... 
    Long term contract
    Full time
    Hybrid work
    Work at office
    Flexible hours

    Glean

    Bangalore
    6 hours ago
  •  ...gives us a competitive advantage. Job Description:   Design Lead with Cloud (GCP) (Experience - 8 to 12 Years) • Strong data...  ...Cloud Platform skills (as this is where the solutions are being developed/deployed) • Must have good Experience/knowledge on GCP components... 
    Local area

    Capco

    Bangalore
    6 hours ago
  •  ...Science, Engineering, or related field. ~ Proven track record leading GCP projects with measurable outcomes (e.g., reduced ETL latency...  ...GDPR), and cost management in GCP. Key Responsibilities: Develop, test, and maintain data pipelines and ETL/ELT processes using... 
    Local area

    Capco

    Bangalore
    6 hours ago
  •  ...the promise of AI. What is The Role Elastic is looking for a Lead Salesforce Engineer who combines deep architectural intuition,...  ...enterprise integration practices. Where the work runs on Salesforce, develop using Lightning Web Components (LWC), Apex, SOQL/SOSL, Flows, and... 
    Full time
    Local area
    Flexible hours

    Elastic

    Bangalore
    6 hours ago
  •  ...services to support Kubernetes and containers. WHAT YOU'LL DO Lead the design, development, and deployment of Portworx's Kubernetes-...  ...in every design and implementation decision. Design and develop robust, well-defined APIs that integrate seamlessly with the frontend... 
    Full time
    Work at office
    Flexible hours

    Everpure

    Bangalore
    6 hours ago
  •  ...Lead Software Engineer Position Description: As a Lead Software Engineer you will join provide technical leadership to the group responsible for architecting, developing, and owning the Zeta Marketing Platform. You will collaborate with Engineers, Product Managers... 

    Zeta Global

    Bangalore
    6 hours ago
  •  ...adventure where you can push the limits of what's possible.    As a Lead Software Engineer at JPMorgan Chase within the Commercial &...  ...approaches to build solutions or breakdown technical problems Develops secure and high-quality production code, and reviews and debugs code... 

    JPMorgan Chase & Co.

    Bangalore
    7 hours ago
  •  ...adventure where you can push the limits of what's possible.  As a Lead Software Engineer at JPMorganChase within the Commercial &...  ...responsibilities ~ Execute creative LLM assisted software solutions, design, develop, and troubleshoot LLM powered applications and services (e.g.,... 
    Full time

    JPMorgan Chase & Co.

    Bangalore
    7 hours ago
  •  ...build, and deliver top-notch technology products.  As a Senior Lead Software Engineer at JPMorganChase within the Consumer and Community...  ...business and its technical teams, contractors, and vendors Develops secure and high-quality production code, and reviews and debugs code... 
    For contractors
    Local area

    JPMorgan Chase & Co.

    Bangalore
    7 hours ago
  • DevOps Engineer - Lead Position Description Design and manage AWS infrastructure using Terraform or CloudFormation. Build and improve...  ...’s strategy and direction. Your work creates value. You’ll develop innovative solutions and build relationships with teammates and... 
    Full time
    Bangalore
    9 hours ago
  • Rs 34.86 lakh p.a.

     ...scalable, reliable, and cost-efficient data platforms processing billions of records Lead the design, development, and optimization of large-scale data pipelines using Spark, PySpark, Python, and SQL Drive platform modernization initiatives, including incremental processing... 
    Full time

    Autodesk

    Bangalore
    9 hours ago
  •  ...Senior QA and testing role at Societe Generale in Bengaluru. What you'll do Lead Software Engineer - Test Automation Creating, daring, innovating, and taking action are part of our DNA What you need ~ Test automation Job details Location: Bengaluru Education... 
    Full time

    Societe Generale

    Bangalore
    1 day ago
  •  ...Senior software engineering role at Societe Generale in Bengaluru. What you'll do Lead Software Engineer - Java Full Stack Angular Creating, daring, innovating, and taking action are part of our DNA What you need Java Angular Job details Location: Bengaluru... 
    Full time

    Societe Generale

    Bangalore
    1 day ago
  • Senior QA and testing role at Societe Generale in Bengaluru. What you'll do Whether you’re joining us for a period of months, years or your entire career, together we can have a positive impact on the future Creating, daring, innovating, and taking action are part of...
    Full time

    Societe Generale

    Bangalore
    1 day ago
  •  ...Mid-level software engineering role at Aliaxis (Ashirvad Pipes) in Bengaluru. Needs 3+ years. What you'll do Lead multiple projects from conception to completion in accordance with deadlines VAVE execution, Cost Reduction experience What you need AutoCAD GD&... 
    Full time

    Aliaxis (Ashirvad Pipes)

    Bangalore
    15 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Lead Pyspark Developer. Be the first to apply!