Jintao Zhang, Apache APISIX Committer, Kubernetes Ingress Nginx Reviewer, and contributor to several cloud-native open source projects.
Apache APISIX Ingress Overview#
Apache APISIX Ingress Definition#
In the K8s ecosystem, Ingress is a resource that represents the entry point for K8s traffic. To make it effective, an Ingress Controller is needed to listen to the Ingress resources in K8s and resolve the rules and actually carry the traffic for those resources. The most widely used Ingress Controller implementations in today's trends are Kubernetes Ingress Nginx.
APISIX Ingress is another implementation of the Ingress Controller. The main difference with Kubernetes Ingress Nginx is that APISIX Ingress uses Apache APISIX as the actual data plane that carries business traffic. As shown in the figure below, when a user requests a specific service/API/web page, the entire business traffic/user request is transferred to the K8s cluster through an external proxy and then processed by APISIX Ingress.
As you can see from the above diagram, APISIX Ingress is divided into two parts. One part is the APISIX Ingress Controller, which is the control plane that will do the configuration management and distribution. The other part is the APISIX Proxy Pod, which is responsible for carrying business traffic and is implemented by means of CRD (Custom Resource Definitions).
Apache APISIX in a nutshell#
We mentioned earlier that APISIX Ingress uses Apache APISIX as the data surface that actually carries business traffic, so what does the Apache APISIX project do?
Apache APISIX is the top open source project of the Apache Foundation and is currently the most active open source gateway project. As a dynamic, real-time, high-performance open source API gateway, Apache APISIX provides rich traffic management features such as load balancing, dynamic upstream, canary release, service meltdown, authentication, observability, and more.
Apache APISIX helps enterprises handle API and microservice traffic quickly and securely with features such as flow-limiting authentication, logging security features, and support for rich custom plug-ins. There are also currently relevant integrations with many open source projects such as Apache SkyWalking, Prometheus and other such components.
APISIX Ingress vs K8s Ingress Nginx#
Since I am involved in the development and maintenance of both APISIX Ingress and K8s Ingress Nginx, many people ask me how to choose between these two projects. Or why do you want to do APISIX Ingress when you have K8s Ingress Nginx.
Configuration level#
In APISIX Ingress, we have added some rich and flexible configurations, such as canary release deployment through a single configuration file. However, in K8s Ingress Nginx, you need at least two Ingress resource files to achieve the above effect.
Richness#
In terms of richness, because Apache APISIX is feature-rich and allows for multiple plugins, using APISIX Ingress eliminates the need to configure additional features yourself, allowing you to spend more time on actual development.
Architecture separation#
APISIX Ingress uses a separate architecture for the data plane and control plane, so users can choose to deploy the data plane inside/outside the K8s cluster. However, K8s Ingress Nginx puts the control plane and the data plane in the same Pod, so if something happens to the Pod or the control plane, the whole Pod will hang, which will affect business traffic.
This architectural separation gives users a more convenient deployment option and facilitates data migration and usage in business architecture adjustment scenarios.
APISIX Ingress Features Explained#
Since Apache APISIX is a fully dynamic, high-performance gateway, APISIX Ingress itself supports full dynamics, including routing, SSL certificates, upstream, plug-ins, and more.
APISIX Ingress also has the following features.
- CRD support for easier understanding of declarative configurations; while state checking ensures quick access to the synchronized state of declarative configurations
- Support for advanced route matching rules and custom resources, which can be extended with more than 50 official Apache APISIX plug-ins & customer-defined plug-ins
- Support for K8s native Ingress configuration
- Support for traffic slicing
- Support for gRPC plaintext and TCP Layer 4 proxy
- Automatic service registration and discovery, no fear of scaling up or down
- More flexible load balancing policy with health check function
The following is a detailed introduction to CRD and custom resources.
CRD Extensions#
We mentioned CRD in the previous introduction, but how does APISIX Ingress use CRD extensions?
At the user level, when a request is made by a Client and arrives at Apache APISIX, the corresponding service traffic is directly transferred to the backend (e.g., Service Pod), thus completing the forwarding process. This process does not need to go through the Ingress Controller, which can ensure that if there are problems, or changes, expansion and contraction or migration processing, it will not affect the users and business traffic.
On the configuration side, users can apply custom CRD configurations to K8s clusters via kubectl apply, and Ingress Controller will continuously watch these resource changes to apply the corresponding configurations to Apache APISIX.
Custom resources#
APISIX Ingress currently supports the following 5 categories of custom resources related to routing, upstream, consumer, certificate related and cluster public configuration.
APISIX Route#
The top-level configuration for the spec attribute in the custom resource APISIX Route is http. But actually spec supports both configurations, spec.http as in the example below, which is mainly used for Layer 7 proxies, and spec.stream, which is used for Layer 4 proxies. In the configuration file, we first customize a rule for it, namely the relevant parameter under match.
As the above back-end configuration example uses the same Service, you can adjust it according to the scenario in actual use. Note that the weight attribute is used to configure the relevant service weight. With the above configuration, a complete set of routing customization resources can be implemented.
APISIX Upstream (Upstream)#
When configuring APISIX Upstream, you need to pay attention to the content of name to be consistent with the Service of K8s cluster, so as to ensure that the subsequent APISIX Ingress Controller can accurately match its corresponding traffic.
In the configuration file, spec.loadbalancer is mainly responsible for setting the load balancing policy, and there are various policy modes to choose from. spec.scheme is for the protocol type configuration, currently only HTTP and gRPC protocols are supported. spec.healthCheck is mainly for setting the health check function, such as setting its active status, effective protocol and path and final feedback and other parameters configuration.
APISIX Consumer#
The APISIX Consumer configuration mainly adds authentication-related features, such as spec.authParameter, which currently supports BasicAuth and KeyAuth, the two more common types of authentication.
You can configure the associated username and password directly with value, or directly with secret, which is more secure than the plaintext configuration of the former.
APISIX TLS (certificate)#
APISIX TLS is mainly for certificate management. As the example shows, users can configure multiple domains via hosts, and the parameters under secret are the corresponding configuration certificates.
Apache APISIX TLS also comes with spec.client for configuring mTLS two-way authentication.
APISIX Config#
The Config types supported by custom resources are described in two ways.
One type is APISIX Cluster Config, which is mainly used for some generic configurations. Currently it supports global use of Prometheus plug-in/global configuration SkyWalking in K8s or Apache APISIX, and we will add some other generic configurations in subsequent development.
Another one is the APISIX Plugin Config that we are currently working on in PR. If you are interested, you can also click the link to join the discussion. Plugin Config is mainly a unified collection of common plug-in configurations, for example, some of the same configuration, users can use APISIX Plugin Config to apply to multiple routes at the same time, eliminating the tedious steps of multiple independent configurations.
APISIX Ingress Hands-On Practice#
Currently, you can deploy APISIX Ingress via Helm Charts. You can deploy both Apache APISIX and APISIX Ingress, including the etcd required for Apache APISIX, with a single command, which is very simple.
Practice Scenario 1: Traffic Segmentation#
By using APISIX Ingress, you can achieve the effect of proportional traffic slicing, as follows.
Step 1: Configure APISIX Upstream#
Step 2: Configure APISIX Route#
Configure subset and weight in backends to split the incoming user request traffic. The example below shows that 90% of the traffic will go to v1 and 10% of the traffic will go to v2.
With the above two steps, it is very easy to slice and dice traffic proportionally to achieve scenarios like canary release. For more details, please refer to: Traffic Segmentation in Apache APISIX Ingress Controller.
Practice Scenario 2: Configuring Authentication#
If you want to configure Basic Auth for certain routes in APISIX Ingress, you can refer to the following actions.
Step 1: Create APISIX Consumer resource#
As mentioned earlier, you can add basicAuth to the APISIX Consumer configuration and specify a username and password for it.
Step 2: Configure the APISIX Route and add authentication-related parameters#
In the custom resource APISIX Route, simply turn it on and specify the authentication type by adding authentication to the backend.
With the above steps, it is possible to use Consumer to complete the relevant configuration authentication.
Practice Scenario 3: K8s Resource Extensions#
As we mentioned at the beginning, APISIX Ingress supports not only custom resources, but also K8s native Ingress resources.
The image above shows the K8s Ingress resource. Normally if you want to do rewrite on a resource, you can add annotation configuration attributes. This way when the user carries the httpbin.org request, it can be redirected to /ip via the path /sample.
When the above requirement uses APISIX Ingress, simply add a kubernetes.io/ingress.class: apisix to Ingress to specify the APISIX Ingress Controller to listen to this resource, and configure k8s.apisix.apache.org/rewrite-target: "/ip" to complete the redirection to the /ip path.
The above example is just one of the ways APISIX Ingress currently supports native K8s Ingress, for more examples you can check the specific documentation for reference and use.
Future Plans#
APISIX Ingress will continue to be updated in terms of functionality and ecology, and the current phase has been completed APISIX Ingress and Cert-manager integration, and the following goals will be achieved step by step.
- complete Kubernetes V1.22+ and CRD V1 version adaptation support (completed, soon to be released in APISIX Ingress V1.3 version)
- support for Gateway API (expected in Q4)
- extend the new architecture to allow users to use APISIX Ingress without the need to use etcd
- enrich the product ecology and expand the APISIX Ingress community
Finally, we hope you can participate in the project more often, for example, there will be an APISIX Ingress community meeting every two weeks on Wednesday at 2 pm, and we will synchronize the current progress of the project or the problems we encountered. Keep an eye on the Apache APISIX video to participate in the live community meetings.
Click here for more details about the APISIX Ingress community meetings.