Principal AI Architect - LLM Infrastructure

Role Overview : - Design the complete private LLM serving architecture : model selection, serving framework (vLLM, TensorRT-LLM, Triton), and runtime topology.- Define intelligent CPU vs GPU routing policies : which model sizes and prompt types route to which compute tier based on latency, cost, and throughput targets.- Architect multi-cloud model serving : AWS, Azure, GCP - provider-agnostic, no managed AI lock-in.- Design on-premises serving for enterprise customers : RedHat OpenShift, VMware - containerised model serving on customer hardware.- Architect per-tenant model isolation, data-residency compliance, and air-gapped sovereign deployment patterns.- Define the release architecture for model versions : rollout, staged deployment, rollback, and promotion gates.- Design the LLM governance framework : model behaviour monitoring, inference audit logging, guardrail architecture.- Design the DevSecOps pipeline architecture and self-service deployment automation standards.- Evaluate open-source models (Llama, Mistral, Qwen, Phi) against closed models for specific enterprise use cases.- Produce architecture sign-off documents and review all AI system designs before implementation.Must Have : - Deployed LLMs to production in a real enterprise environment - not just API consumption.- Hands-on with vLLM, TensorRT-LLM, or Triton in a production serving context.- Designed CPU cluster inference (Intel Xeon / AMD EPYC) for open-source models.- Kubernetes at production scale (EKS, AKS, GKE, or OpenShift) - not just local k8s.- Designed multi-cloud, provider-agnostic AI architectures.- Experience with model quantization (GPTQ, AWQ, GGUF) and routing trade-offs.- 8+ years in AI/software architecture with at least 3 years in production LLM systems. (ref:hirist.tech)