The Senior Agentic AI Modeler is a hands-on technical expert responsible for designing, building, evaluating, and deploying agentic AI systems that can reason, plan, interact with tools, orchestrate workflows, and autonomously solve complex business problems. This role blends deep LLM expertise, modern agent frameworks, prompt engineering, architectural design, and practical MLOps awareness to build safe, reliable, and high-performing agentic pipelines. You will work closely with product, engineering, data, and governance teams to translate business needs into intelligent AI agents that deliver measurable outcomes at scale.

If you enjoy pushing the boundaries of what AI agents can do—multi-step planning, tool use, memory, RAG orchestration, function calling, autonomous workflows, this role is for you.

*This role requires on-site office presence 1x per week, on average.

About the Company:

Our client is a world-renowned, commercial-stage biotech powerhouse leveraging high-scale AWS/Databricks ecosystems to build autonomous AI agents that will help accelerate life-saving drug discovery and launches.

Key Responsibilities:

Agentic System Design & Development

• Architect and implement autonomous and semi-autonomous agent workflows using modern frameworks

(LangChain, OpenAI Assistants, Semantic Kernel, LlamaIndex, CrewAI, DSPy, etc.)

• Build agents capable of multi-step reasoning, task decomposition, planning, and tool execution.

• Develop tooling ecosystems—function APIs, retrieval endpoints, evaluators, monitoring hooks—for agent

orchestration.

• Prototype, fine-tune, and optimize LLMs, RAG systems, and multi-agent architectures for reliability and low

hallucination rates.

Modeling & Prompt Engineering

• Create robust prompt pipelines, templates, and routing strategies for diverse commercial/analytics use

cases.

• Optimize agent behaviors via prompt distillation, chain-of-thought design, system message tuning,

guardrails, and evaluation frameworks.

• Use datasets and logs to refine agent behavior through supervised fine-tuning or RLAIF where applicable.

Solution Integration & Delivery

• Integrate agents with enterprise systems, APIs, data stores, CRMs, search tooling, and business

applications.• Implement observability for agent decision-making: trace logs, tool call audits, success/failure rates, safety

events, drift.

• Partner with engineering and MLOps to deploy agentic systems in secure, scalable production

environments.

Quality, Safety, & Governance

• Ensure agents operate within approved safety, compliance, and privacy constraints.

• Define repeatable patterns for safe agent behavior: policy enforcement, output validation, citation checking,

guardrail triggers.

• Collaborate with model risk, medical/reg, and product teams to ensure trustworthy outputs.

Collaboration & Thought Leadership

• Work closely with Product, Data Engineering, MLOps, UX, and Governance teams to shape the roadmap.

• Stay current with breakthroughs in agent architectures, reasoning models, tool use, and planning systems.

• Mentor junior modelers and share best practices across the organization.

Qualifications:

• 5–8+ years of experience in ML/AI engineering, NLP/LLMs, or agentic system development.

• 10+ years of SDLC experience a must.

• Strong hands-on experience with LLMs (OpenAI, Anthropic, Meta, Google, Mistral, etc.) and embeddings.

• Demonstrated ability to build agent workflows using frameworks such as LangChain/LangGraph, Semantic

Kernel, LlamaIndex, CrewAI, OpenAI Assistants API, and DSPy.

• Proficiency in Python and modern API design.

• Experience with RAG pipelines, retrieval tuning, and vector databases (Pinecone, Weaviate, Chroma,

Elastic, etc.).

• Familiarity with model deployment and cloud ecosystems (Azure, AWS, GCP).

• Experience creating safety-aware AI systems with guardrails, evaluation harnesses, and human-in-the-loop reviews

Compensation: $90– $110 per hour 

The hourly rate offered will be commensurate with experience, skills, qualifications, and other relevant factors. 

About the Search: This search is being managed by Summit Strategic Search on behalf of our client. This is a contract-based assignment where we the recruitment and employment administration. While you will be engaged through our firm, the end client retains final authority over the selection of personnel for their project team. All offers of engagement are contingent upon the end client’s approval, project requirements, and successful completion of the onboarding process. 

Equal Opportunity Commitment: We do not discriminate on the basis of race, religion, color, national origin, gender, gender identity or expression, sexual orientation, age, marital status, veteran status, disability status, or any other characteristic protected by law.   

Accommodations & Support: We are committed to providing reasonable accommodations for qualified individuals with disabilities during the application and interview process. If you require assistance, please contact us at [email protected]. We maintain a zero-tolerance policy for harassment and provide all contractors with dedicated reporting channels to ensure a safe, professional work environment. 

Onboarding Requirements: Due to the regulated nature of our client’s industry and the requirement for on-site presence in Union County, NJ, all candidates must successfully complete the end client’s onboarding protocols. This typically includes a background check, drug screening, and adherence to site-specific security and health policies.