Scaling Patterns

Horizontal Scaling Pattern

• Horizontal Scaling or “scaling out,” involves adding more instances of a microservice to distribute the load.
• This pattern improves the system’s ability to handle increased traffic and provides better fault tolerance.
• Instances can be added or removed dynamically based on the current load, which is particularly effective in cloud environments where resources can be provisioned on demand.

Vertical Scaling Pattern

• Vertical Scaling or “scaling up,” involves adding more resources (CPU, memory) to an existing microservice instance.
• This pattern can enhance the performance of a microservice without changing the number of instances.
• However, it has limitations as there is a maximum capacity for a single instance and it may lead to higher costs for more powerful hardware or virtual machines.

Auto-Scaling Pattern

• Auto-Scaling automatically adjusts the number of microservice instances based on predefined metrics and thresholds, such as CPU usage, memory usage or request rate.
• This pattern ensures that the system can handle varying loads efficiently, scaling out during peak times and scaling in during low-demand periods.
• Auto-scaling helps in optimizing resource usage and cost, maintaining performance without manual intervention.

Service Mesh Pattern

• Service Mesh provides a dedicated infrastructure layer for managing service-to-service communication.
• It includes features such as load balancing, traffic management, service discovery, observability and security policies.
• A service mesh abstracts the communication logic out of the microservices, enabling better observability, resilience and control over how services interact.
• It is particularly useful in complex microservices architectures, ensuring consistent and secure communication between services.

A service mesh has two planes:

1. Data Plane: A network of lightweight proxies (usually Envoy) deployed as sidecars alongside each microservice. They intercept all network traffic.
2. Control Plane: The “brain” that manages and configures the proxies. It provides the API for humans to define policies (e.g., Istio’s istiod).

Service Mesh vs API Gateway

Use a Service Mesh if:

• There are 20+ microservices communicating frequently.
• There is a need of zero-trust security (mTLS) between all pods.
• There is a need of standardized observability across different programming languages.

Avoid it if:

• There’s a small cluster (the resource overhead will outweigh the benefits).
• The team is small (managing the control plane adds significant operational toil).

Read more at:

• https://www.redhat.com/en/topics/microservices/what-is-a-service-mesh
• https://aws.amazon.com/what-is/service-mesh/