Problem
How do the clients of Microservices-based applications access the individual services ?
Soution
- In a distributed environment, services need to communicate with each other. However, this is interservice communication. We also have use-cases where a client outside our domain wants to hit our services for the API. So, either we can expose the address of all our microservices which can be called by clients OR we can create a Service Gateway which routes the request to various microservices and responds to the clients.
- Creating a Gateway is much better approach here. There are two major advantages.
- The security for each individual services does not need to maintained.
- And, cross-cutting concerns, for example, addition of meta-information can be handled at a single place.
- Netflix Zuul and Spring Cloud Gateway are two well-known Cloud Gateways which are used to handle such situations. In this tutorial, we will use Spring Cloud Gateway.
Spring Cloud Gateway
- Spring Cloud Gateway provides a library to build an API Gateway. This is the preferred gateway implementation provided by Spring Cloud. It's built with Spring 5, Spring Boot 2, and Project Reactor.
- To understand the offerings of Spring Cloud Gateway we must understand the API Gateway pattern in detail.
- Let's assume, we are implementing the microservices architecture for our e-commerce system. One of the microservices in the system is Product Catalog Service, which is responsible to manage product lifecycle through — create, update, delete, and get operations. Let’s go through some common scenarios, we might come across —
Scenario 1 — Multiple Versions of Service
- Our Product Catalog Service is updated with the new changes, but the updated service is not compatible with the mobile client.
- Though we cannot achieve the service update completely, we can optimize it by running multiple versions in parallel. Our mobile client will continue using version-V1 while other clients will move to the version-V2.
Scenario 2 — Restricted Access
- Our Product Catalog Service provides APIs to both read and write, the catalog data.
- Our security team suggests to limit the access to write APIs — add, update, and delete, from the vendor portal only — vendor.test-gateway-store.com.
- The rest of the clients, like the e-commerce portal, can only access get product API from Product Catalog Service.
Scenario 3 — Monitoring API Performance
- A lot of customers are complaining about delays while getting the product details. We must monitor the performance of the “get product details” API to assess the severity and plan out the next steps.
Scenario 4 — Updating Response Header
- We received the performance statistics of Product Catalog Service. Get product details API is getting hit too many times. This is not needed as the product details are not updated every minute.
- We can avoid the unnecessary hits by introducing the “Cache-Control” header attribute in the response of “get product details API”.
- The above-discussed scenarios typically fall into the category of cross-cutting concerns and must be dealt with separately. This increases the maintainability and agility of the overall system. The API Gateway pattern provides a cleaner approach to resolve such issues.
How API Gateway Works?
- The API gateway takes all the API calls, from the clients and routes them to appropriate microservice(s) with request routing, composition, and protocol translation.
- API Gateway was introduced to provide the coarse grained APIs (implementing API Composition) which can communicate to the fine-grained APIs, aggregate the responses and return back to the end user. As a single entry point was introduced with this pattern, it became the preferred choice to enable the cross-cutting concerns.
- Exactly what the API gateway does will vary from one provider to another. Some common functions include routing, rate limiting, billing, monitoring, authentication, API composition, policies, alerts, and security.
Spring Cloud Gateway — Core Functions
- Spring Cloud Gateway supports many cross-functional features we discussed above. To support the wide range of such features, the framework is based on three core components — Routes, Predicates, and Filters.
Routes
- This is the primary component consisting of ID, Destination Uri, Predicates, and Filters.
Predicates
- This is more like a condition to match the route. This is done based on the HTTP request, headers, parameters, path, cookie, and other request criteria.
Filters
- They are the plugins to update the request or response. They function similarly to the servlet filters with the pre and post variations. You can use it for multiple purposes including “adding a request parameter”, “adding a response header”, “monitoring”, and many other utilities.
- We will see the examples for each of these aspects through the sample implementations in the next section.
Nextflix Zuul
- A zuul proxy is used as an API gateway which receives all the incoming requests to the system and calls corresponding underlying microservice based on the configurations.
- With this architecture:
- We won’t have CORS issues
- We will have a centralized security implementation which will be applied to all incoming requests
- We can apply filters to all incoming requests
- When needed, we will be able to change the security business logic easily (since it is centralized)
- At the center of Zuul is a series of Filters that are capable of performing a range of actions during the routing of HTTP requests and responses.
The following are the key characteristics of a Zuul Filter:
1. Type:
- Most often defines the stage during the routing flow when the Filter will be applied (although it can be any custom string)
2. Execution Order:
- Applied within the Type, defines the order of execution across multiple Filters
3. Criteria:
- The conditions required in order for the Filter to be executed
4. Action:
- The action to be executed if the Criteria is met
- Zuul provides a framework to dynamically read, compile, and run these Filters. Filters do not communicate with each other directly - instead they share state through a RequestContext which is unique to each request.
- Filters are currently written in Groovy, although Zuul supports any JVM-based language. The source code for each Filter is written to a specified set of directories on the Zuul server that are periodically polled for changes. Updated filters are read from disk, dynamically compiled into the running server, and are invoked by Zuul for each subsequent request.
Filter Types
- There are several standard Filter types that correspond to the typical lifecycle of a request:
1. PRE
- Filters execute before routing to the origin. Examples include request authentication, choosing origin servers, and logging debug info.
2. ROUTING
- Filters handle routing the request to an origin. This is where the origin HTTP request is built and sent using Apache HttpClient or Netflix Ribbon.
3. POST
- Filters execute after the request has been routed to the origin. Examples include adding standard HTTP headers to the response, gathering statistics and metrics, and streaming the response from the origin to the client.
4. ERROR
- Filters execute when an error occurs during one of the other phases.
- Alongside the default Filter flow, Zuul allows us to create custom filter types and execute them explicitly. For example, we have a custom STATIC type that generates a response within Zuul instead of forwarding the request to an origin.
- We have a few use cases for this, one of which is internal endpoints that contain debug data about a particular Zuul instance.
Zuul Features
Resiliency Features
1. Adaptive Retries
- The core retry logic that we use at Netflix to increase our resiliency and availability
2.Origin Concurrency Protection
- configurable concurrency limits to protect your origins from getting overloaded and protect other origins behind Zuul from each other
Operational Features
1. Request Passport
- track all the lifecycle events for each request, which is invaluable for debugging async requests
2. Status Categories
- An enumeration of possible success and failure states for requests that are more granular than HTTP status codes
3.Request Attempts
- Track proxy attempts and status of each, particularly useful for debugging retries and routing
4. Websocket/SSE
- support for side-channel push notifications
5. Throttling and rate-limiting
- protection from malicious client connections and requests, helping defend against volumetric attacks
6. Brownout filters
- For disabling certain CPU-intensive features when Zuul is overloaded
- Configurable routing — file-based routing configuration, instead of having to create routing filters in Zuul
