This use of the native Kubernetes API also helps new users to seamlessly transition from single-cluster to multi-cluster architecture. More expansive details on these improvements are listed in the associated blog post. This permits users of KubeAdmiral federated clusters to see a consolidated view of deployments across multiple clusters, rather than having to look at each individual cluster in turn. KubeAdmiral also goes beyond its predecessors by fully using native Kubernetes APIs rather than a separate federation API for cluster management. KubeAdmiral also refines KubeFed's approach to rescheduling pods for optimum utilization by reducing pod churn and minimising changes made to rebalance clusters. In contrast to KubeFed's replica rescheduling algorithm, which could often lead to skewed and inconsistent utilization depending on how frequently a cluster was deployed to, KubeAdmiral has a more refined algorithm based on real-time cluster utilization instead. The system also introduces dynamic weight scheduling based on real-time cluster resource utilization, achieving dynamic load balancing across member clusters. KubeAdmiral also addresses challenges related to unschedulable workloads by automatically migrating these to other clusters, ensuring that temporary issues such as node outages, resource shortages and affinity problems don't compromise reliability. This design simplifies the scheduler logic and reduces overall complexity. Each stage is handled by individual plugins, promoting modularity and allowing users to add or remove their own scheduling plugins without affecting others. Taking inspiration from kube-scheduler's design, KubeAdmiral introduces a flexible framework dividing scheduling into four distinct stages: Filter, Score, Select, and Replica. ![]() Automatic dependency scheduling is also a notable feature, allowing dependencies such as ConfigMaps to automatically follow their Deployment to corresponding member clusters. Unlike its predecessor KubeFed, KubeAdmiral has comprehensive scheduling semantics, supporting more flexible and fine-grained mechanisms for cluster selection and scoring via labels and taints, and scoring of clusters based on load and affinity. The scheduler, often referred to as the "brain" of KubeAdmiral, plays a pivotal role in computing the desired placement of workloads in member clusters. One of the standout features of KubeAdmiral is its rich multi-cluster scheduling capabilities. In a comprehensive post, ByteDance's Gary Liu has delved into the details of KubeAdmiral's objectives and functionality. The surge in the size and number of Kubernetes clusters at ByteDance, coupled with the diversification of workloads beyond stateless microservices to include stateful services, storage, offline, and machine learning jobs, prompted the need for a more robust and scalable cluster federation system. Having run KubeFed for a number of years, ByteDance witnessed phenomenal growth in its cloud-native infrastructure. It maintains a stable deployment rate of 95-98%, significantly reducing the need for manual intervention. ![]() Currently managing over 100,000 microservices with more than 10,000,000 pods across dozens of federated Kubernetes clusters at ByteDance, KubeAdmiral handles 30,000 upgrade and scaling operations daily. KubeAdmiral scales to run more than 10 million pods across dozens of federated Kubernetes clusters.īased on the foundations of the older KubeFed system, KubeAdmiral boasts enhanced multi-cluster orchestration and scheduling capabilities, specifically tailored for various mainstream business scenarios. This is designed to manage multiple clusters with the efficiency and effectiveness comparable to "a seasoned navy admiral commanding a fleet". ByteDance, the company behind popular global platforms like TikTok, has unveiled KubeAdmiral, its next-generation cluster federation system for Kubernetes.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |