kubernetes
-
The third part of this blog post series covered migration from the RayCluster CRD to the RayService CRD. However, much more has happened since I wrote the second part. Therefore, this post tells you the rest of the story, including the challenges we faced and the improvements we made to enhance the system. Autoscaling often…
-
Initially, I didn’t think about writing another part of this blog post series, and the way I wrapped up the second one could give that impression. However, recent updates have made this topic worth revisiting, so here we are. In the first blog post, I mainly talked about RayCluster and its Custom Resource Definition (CRD)…
-
As described in the first part of this series, we set up our CI/CD to deploy an application running as a Ray Serve Deployment to the RayCluster, and came up with a passable workaround for the external Global Control Service (GCS). Slowly but surely, the production K8s cluster came to life and started providing the…
-
In this blog post, I would like to share my experiences running AI applications on Kubernetes. Our client, Mixedbread, leverages KubeRay Operator to run their AI workloads. We help them migrate their applications from one of the hyperscalers to a multi-cloud solution using an open-source tool named Claudie, and cut the infrastructure cost by 70%.…