NY, SF, Remote
Why RoVR
At RoVR, we’re on a mission to turn practices into AI-Native, Self-Optimizing Learning Health Clinics. At RoVR, we are building a multimodal, agentic operating system to launch, manage, and scale A-LHCs nationwide. We collaborate with large health organizations as well as local physician groups, modernizing legacy operations to launch these next-generation clinics.
Central to this vision are at RoVR, autonomous agents that orchestrate and optimize clinical, administrative, and patient engagement processes. By alleviating repetitive tasks, RoVRs free providers and admins to focus on proactive decision-making guided by actionable insights, laying the groundwork for a more responsive, outcome-driven healthcare ecosystem.
By uniting clinical and operational intelligence in one cohesive platform, we're building the infrastructure for a Learning Health System that can collaborate with and empower clinicians to redefine the patient experience and transform health outcomes.
Based in the heart of NYC and backed by leading venture capital firms/visionary healthcare leaders, RoVR is on a mission to transform the future of healthcare. We are seeking mission-driven engineers, designers, and operators to join our team and help us build the foundational operating system that will drive medicine for the next century.
The Role
We are looking for an infrastructure engineer to design, build and operate the platform that supports our research and runs our production agents: the cloud and GPU fleet, orchestration, deployment and observability, and the serving path for fine-tuned open-weight models. As the first person responsible for this platform, you would set the standards it runs on and decide what it becomes as training and inference scale. The focus is making the stack faster, cheaper and more reliable, so that researchers can concentrate on models and agents can be trusted with clinical work.
Responsibilities
Own the cloud platform: container orchestration, GPU inference infrastructure, networking, storage, and infrastructure as code across training and production.
Build the deployment path for models and agents: CI/CD, artifact and checkpoint versioning, canary and rollback, and promotion to production without manual steps.
Operate model serving (vLLM, SGLang or similar): batching, caching, multi-model routing, and latency and cost targets per workload.
Set the standard for reliability and observability: SLOs, on-call, incident response, tracing and metrics across every service.
Own routing and capacity: Build request routing, queue management, GPU capacity allocation, and autoscaling to meet workload demands.
Optimize inference performance: Improve latency, throughput, and cost through batching, caching, concurrency, streaming, and GPU memory optimization.
Secure multi-tenant inference: Implement access controls, secrets management, audit logging, and isolation across requests, caches, and model-serving infrastructure.
Extend orchestration and research tooling so researchers can launch, monitor and reproduce runs on their own.
Minimum qualifications
Bachelor's degree or equivalent experience in computer science or a related technical field.
Experience running production infrastructure on a major cloud, including container orchestration and GPU workloads.
Strong infrastructure-as-code and CI/CD practice (Terraform or similar, GitHub Actions or similar).
Experience serving large models in production, or deep enough systems background to pick it up quickly.
Proficiency in Python and one of Go, Rust or C++.
Experience owning reliability and security for systems people depend on.
Preferred qualifications
Experience with distributed inference, GPU performance optimization, or serving multiple fine-tuned variants of a base model.
Experience implementing security controls for regulated systems, supporting HIPAA compliance, or preparing for SOC 2 audits.
Contributions to open-source infrastructure or model-serving projects.
Logistics
Location: San Francisco or New York, on-site.
Employment type: Full-time.
Compensation: competitive salary and meaningful equity.
Visa sponsorship is available.
RoVR is an equal opportunity employer and does not discriminate on the basis of race, gender, sexual orientation, gender identity/expression, national origin, disability, age, genetic information, veteran status, marital status, pregnancy or related condition, or any other basis protected by law.
Apply for this Job