Reinforcement Learning based Autoscaling for Kafka-centric Microservices in Kubernetes
- Title
- Reinforcement Learning based Autoscaling for Kafka-centric Microservices in Kubernetes
- Creator
- Joyce J.E.; Sebastian S.
- Description
- Microservices and Kafka have become a perfect match for enabling the Event-driven Architecture and this encourages microservices integration with various opensource platforms in the world of Cloud Native applications. Kubernetes is an opensource container orchestration platform, that can enable high availability, and scalability for Kafkacentric microservices. Kubernetes supports diverse autoscaling mechanisms like Horizontal Pod Autoscaler (HPA), Vertical Pod Autoscaler (VPA) and Cluster Autoscaler (CA). Among others, HPA automatically scales the number of pods based on the default Resource Metrics, which includes CPU and memory usage. With Prometheus integration, custom metrics for an application can be monitored. In a Kafkacentric microservices, processing time and speed depends on the number of messages published. There is a need for auto scaling policy which can be based on the number of messages processed. This paper proposes a new autoscaling policy, which scales Kafka-centric microservices deployed in an eventdriven deployment architecture, using a Reinforcement Learning model. 2022 IEEE.
- Source
- 2022 IEEE 4th PhD Colloquium on Emerging Domain Innovation and Technology for Society, PhD EDITS 2022
- Date
- 2022-01-01
- Publisher
- Institute of Electrical and Electronics Engineers Inc.
- Subject
- custom metrics; Horizontal Pod Autoscaling (HPA); Kubernetes
- Coverage
- Joyce J.E., Christ (Deemed to Be University), Deparatment of Computer Science, Bangalore, India; Sebastian S., Christ (Deemed to Be University), Deparatment of Computer Science, Bangalore, India
- Rights
- Restricted Access
- Relation
- ISBN: 978-166546143-6
- Format
- Online
- Language
- English
- Type
- Conference paper
Collection
Citation
Joyce J.E.; Sebastian S., “Reinforcement Learning based Autoscaling for Kafka-centric Microservices in Kubernetes,” CHRIST (Deemed To Be University) Institutional Repository, accessed February 23, 2025, https://archives.christuniversity.in/items/show/20206.