About the Job
We're on a mission to make good health last a lifetime. More than 1 billion people live with obesity worldwide, driving preventable chronic conditions. We're here to build better long-term care.
Eucalyptus is now part of Hims & Hers, the global leader in personalised health and wellness.
Euc is the company behind Juniper, one of the world's largest weight-management programs combining GLP-1 medication with personalised nutrition, movement support, and clinician-led care from prescribers, nurses, health coaches, pharmacists, and dietitians. Our published clinical research shows that our combined clin...
Key Responsibilities
AI now touches almost every conversation we have with patients — consultations, medical support, health coaching, AI assistants, and patient experience support. As we scale, we need one standardised way to answer: are these conversations safe, accurate, and high quality?
That’s what you’ll build. You’ll own the development of a standardised evaluations layer across all of our support workstreams — clinical interactions (consultations, medical support), health coach conversations, AI assistant conversations, PX support, and health insights (wellness vs. clinical). You’ll build the services that monitor what is said to our patients and flag risky conversations and insights. You’ll also own how our AI assistants integrate with our in-house patient data models, so every assistant responds with accurate, well-grounded patient context.
This is a rare junior role: real ownership of a safety-critical system from day one, with the support of experienced teams around you. You’ll report to the Head of Data and work closely with the ML Engineering, Data Engineering and Data Science teams.
Design and build a standardised evaluation framework covering clinical interactions, health coach conversations, AI assistant conversations, PX support, and health insights
Build and run services that monitor patient-facing conversations and automatically flag risky content for review and escalation
Develop eval datasets, metrics, and thresholds with clinicians, coaches, and support leads — turning “what good looks like” into something measurable
Own the technical integration between our AI assistants and our data models, making patient & business context available to assistants safely, and consistently
Operate your systems in production: monitoring, alerting, and iterating as models and prompts evolve
Own and automate guardrail updates based on internal and patient feedback from interactions with our third-party support tools
Share results with the teams who own each workstream, and help them improve
Required Skills & Abilities
A degree in computer science, engineering, data science, or a related field — or equivalent practical experience
Strong Python skills and the fundamentals to build reliable services, not just notebooks
Hands-on experience with LLMs: prompt engineering, and ideally exposure to eval frameworks or LLM-as-judge approaches (coursework, internships, or personal projects all count)
A careful, safety-first mindset — you care about what an AI system says to a real patient, and you sweat the edge cases
Clear communication — you can explain eval results to a clinician as well as an engineer
Comfortable with ambiguity: this layer doesn’t exist yet, and you’ll help define it
Experience with eval tooling (e.g. promptfoo, Ragas, LangSmith, or in-house harnesses)
Exposure to healthcare, regulated industries, or other high-stakes AI applications
Familiarity with GCP, SQL, or modern data stacks
