Software Engineer III - AI/ML, Prompt Engineer

Job Description

You are a strategic thinker passionate about building prompt-driven, production-grade AI that measurably improves risk and control outcomes across enterprise workflows. You have found the right team.

As a Prompt Engineering & Applied ML Associate at JPMorgan Chase as a part of our Risk & Controls AI team, you will spend each day translating business intent into grounded, auditable decision support by combining prompt engineering with applied machine learning

 

Job responsibilities 

• Design production-grade prompts for complex enterprise workflows; test, iterate, and optimize based on outcomes.
• Apply familiarity with ML models, including classification, NLP, and transformer-based architectures. Implement LLM integration patterns such as retrieval-augmented generation (RAG), chain-of-thought prompting, and response validation.
• Define and execute prompt/model evaluation criteria (accuracy, consistency, hallucination rate, policy adherence).
• Design and run offline and online experiments to improve prompts and tune model performance.
• Implement guardrails, safety filters, and fallback strategies in production AI/ML workflows. Use ML frameworks such as PyTorch, TensorFlow, scikit-learn, and/or Hugging Face Transformers.
• Build multi-step agent workflows using LangChain or similar orchestration frameworks.
• Leverage modern databases—including vector stores (e.g., Pinecone, pgvector), graph databases, and relational/NoSQL systems—for retrieval and persistence.
• Optimize embeddings usage, tokenization strategies, and context-window management. Integrate model serving infrastructure, API-based model providers, and model routing strategies.
• Prepare data, engineer features, and curate datasets for ML training and evaluation. Monitor AI/ML-assisted production workflows with observability and drift detection practices. Translate complex business requirements into structured prompt, model, and system designs.
• Develop Python-based orchestration, data transformation, and automation scripts. Collaborate effectively with software engineers, product owners, and control stakeholders.
• Communicate clearly in writing and verbally to both technical and business audiences.

Required qualifications, capabilities, and skills

• BS/BA degree in Computer Science, Engineering, Machine Learning, Data Science, Statistics, or equivalent experience
• 5+ years of hands-on experience in machine learning, applied AI, or prompt engineering in production environments
• 5+ years of experience with Python and ML/AI tooling (model development, evaluation, and deployment)
• 3+ years of experience with LLM application development, prompt engineering, or NLP systems
• 1+ years of experience with experimentation frameworks, A/B testing, and production monitoring of AI features

Preferred qualifications, capabilities, and skills

• Experience with responsible AI practices and model governance
• Experience in financial services, technology risk, or controls-oriented environments
• Relevant ML/AI certifications (e.g., AWS ML Specialty, Google Professional ML Engineer)
• Understanding of enterprise risk, controls, and auditability expectations

About Us

JPMorganChase, one of the oldest financial institutions, offers innovative financial solutions to millions of consumers, small businesses and many of the world’s most prominent corporate, institutional and government clients under the J.P. Morgan and Chase brands. Our history spans over 200 years and today we are a leader in investment banking, consumer and small business banking, commercial banking, financial transaction processing and asset management.

We recognize that our people are our strength and the diverse talents they bring to our global workforce are directly linked to our success. We are an equal opportunity employer and place a high value on diversity and inclusion at our company. We do not discriminate on the basis of any protected attribute, including race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, marital or veteran status, pregnancy or disability, or any other basis protected under applicable law. We also make reasonable accommodations for applicants’ and employees’ religious practices and beliefs, as well as mental health or physical disability needs. Visit our  FAQs for more information about requesting an accommodation.

About the Team

J.P. Morgan’s Commercial & Investment Bank is a global leader across banking, markets, securities services and payments. Corporations, governments and institutions throughout the world entrust us with their business in more than 100 countries. The Commercial & Investment Bank provides strategic advice, raises capital, manages risk and extends liquidity in markets around the world.