Jobgether
Senior MLOps Engineer - SRE | DevOps
TLDR
Own end-to-end ML deployment and production-grade inference across multi-tenant AI platforms, driving reliability, scalability, and efficient infrastructure.
Accountabilities:
- Design, build, and operate scalable ML and inference infrastructure supporting real-time and batch workloads across multiple tenants.
- Own the end-to-end ML deployment lifecycle, including model registry, versioning, rollout strategies (canary, A/B, shadow), and safe rollback mechanisms.
- Operate and optimize production-grade AI and LLM workloads, managing inference providers, throttling, quotas, and fallback strategies under load.
- Develop and maintain reproducible ML pipelines for training, evaluation, and deployment with full lineage and automation.
- Implement Infrastructure-as-Code practices using Terraform, ensuring scalable multi-account cloud architectures.
- Manage GitOps workflows using tools such as ArgoCD to ensure reliable and consistent deployments across environments.
- Operate Kubernetes-based infrastructure (AWS EKS), including GPU scheduling, workload isolation, and cost-aware scaling strategies.
- Define and enforce SRE best practices, including SLOs, observability, incident response, and performance monitoring for ML systems.
- Drive cost optimization initiatives across ML workloads, including resource right-sizing and efficient infrastructure utilization.
- Improve automation across the ML lifecycle using modern engineering and agentic coding tools.
- 5+ years of experience in Platform Engineering, SRE, DevOps, or MLOps roles, operating production systems at scale.
- Strong hands-on experience deploying and managing ML/AI workloads in production environments.
- Deep SRE expertise, including SLO definition, incident response, postmortems, and reliability engineering practices.
- Advanced experience with Infrastructure-as-Code using Terraform in complex, multi-account environments.
- Strong GitOps experience with declarative infrastructure and deployment workflows.
- Deep expertise in Kubernetes, including production operations and failure-mode troubleshooting.
- Strong AWS knowledge, including networking, IAM, compute, storage, and distributed architectures.
- Experience building CI/CD pipelines using tools such as GitHub Actions, GitLab CI, CircleCI, or similar.
- Strong automation mindset with ability to eliminate manual operational work through engineering solutions.
- Familiarity with agentic coding tools and ability to use them effectively in infrastructure and pipeline development.
- Strong communication skills to articulate technical decisions, trade-offs, and incident analysis clearly.
- Experience with GPU/accelerator scheduling and node lifecycle management (e.g., Karpenter).
- Experience operating LLM inference systems at scale, including quota management, caching, and guardrails (e.g., AWS Bedrock or similar).
- Experience with ML orchestration tools such as Argo Workflows, Kubeflow, Airflow, or SageMaker Pipelines.
- Familiarity with ML observability tools, drift detection, and model monitoring practices.
- Background in FinOps and cost attribution for large-scale inference systems.
- Experience with multi-tenant infrastructure and isolation strategies.
- Exposure to feature stores, model registries, and experiment tracking tools such as MLflow or Feast.
- Experience scaling ML platforms in high-growth or startup-to-enterprise environments.
- Fully remote work model with flexibility.
- Opportunity to work on cutting-edge AI and ML infrastructure at scale.
- High ownership environment with direct impact on platform architecture and evolution.
- Exposure to modern cloud-native technologies, Kubernetes, and distributed systems at production scale.
- Collaborative engineering culture focused on automation, reliability, and innovation.
- Work aligned with global time zones (EST/PST) for structured collaboration.
- Continuous technical challenges involving LLMs, ML systems, and large-scale infrastructure.
- Strong emphasis on engineering autonomy and senior-level decision-making.
Requirements:
Nice to have:
Benefits:
How Jobgether works:
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1
Benefits
Engineering autonomy and decision-making
Strong emphasis on engineering autonomy and senior-level decision-making.
Remote-Friendly
Fully remote work model with flexibility.
Jobgether runs the largest remote job platform, effectively linking job seekers with over 200,000 flexible and remote opportunities that match their unique skills and preferences. Our focus is on enhancing the hiring process, ensuring efficiency while prioritizing the candidate experience, particularly in the growing health and wellness sector.
- Founded
- Founded 2020
- Employees
- 11-50 employees
- Industry
- Professional Services
Operations Engineer