Lets learn "About kube proxy in iptables mode"

-


In this tutorial, we'll learn about the Kube-proxy in iptables mode. Basically, we'll see how Kube-proxy create iptables that help in load balancing and service discovery. If you are not familiar with Linux IPTables I would recommend to check out my tutorial on IPTables here first and then come back to read this tutorial.

Kube-proxy

Kube-proxy when running in iptables mode basically watch kube-apiserver for new endpoints and create corresponding iptables rules and iptables are fully responsible for service discover and proper routing of traffic to corresponding pods. Kube-proxy needs to run on all k8s worker nodes.

Below is a small video describing the working of kube-proxy in iptables mode. 



Now that we know what Kube-proxy is and how it Kube-proxy works, let deep dive into the types of iptables Kube-proxy adds to the firewall.

Kube-proxy added IPTables

To learn this let's consider an example. Consider a k8s cluster with app FOO running (and listening on 9153 port )and 3 replicas of FOO is running in the cluster. So the 3 pods for app are as follows:

[root@worker1 ~]# kubectl get pods -o wide | grep FOO
FOO-7g26h               1/1     Running   2          3d13h   100.96.0.2    worker1.cluster.com     <none>      <none>
FOO-9r4mj                1/1     Running   2         3d13h   100.96.32.2     worker2.cluster.com   <none>     <none>
FOO-wjvrz                1/1     Running   2          3d13h   100.96.64.2     worker3.cluster.com   <none>     <none>

Let app FOO is exposed by a cluster IP service. And the virtual IP of the service is 100.96.128.2.

[root@worker1 ~]# kubectl get svc | grep FOO

NAME      TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                  AGE

FOO   ClusterIP   100.96.128.2   <none>        9153/TCP   46m


Since 100.96.128.2 is the virtual IP, IP tables should be added in a way that traffic to this VIP should be redirected of one of the pods IP ( 100.96.0.2, 100.96.32.2, and 100.96.64.2 )



Let's see the IPTables rules now.

>> iptables -t nat -L


Chain KUBE-SERVICES (2 references)

target     prot opt source               destination

  • KUBE-MARK-MASQ  tcp  -- !100.96.0.0/17        100.96.128.2         /* kube-system/FOO:foo cluster IP */ tcp dpt:9153

  • KUBE-SVC-QKJQYQZXY3DRLPVB  tcp  --  anywhere             100.96.128.2         /* kube-system/FOO:foo cluster IP */ tcp dpt:9153

  • KUBE-NODEPORTS  all  --  anywhere             anywhere             /* kubernetes service nodeports; NOTE: this must be the last rule in this chain */ ADDRTYPE match dst-type LOCAL


Above highlighted rules means if there is traffic destined to port 9153 will be redirected KUBE-SVC-QKJQYQZXY3DRLPVB chain. Let's see whats in this chain.

Chain KUBE-SVC-QKJQYQZXY3DRLPVB (1 references)

target     prot opt source               destination

KUBE-SEP-3UIFCWUMBYEVXRYP  all  --  anywhere             anywhere             statistic mode random probability 0.33333333349

KUBE-SEP-QSKKIZFW3PNDWYKJ  all  --  anywhere             anywhere             statistic mode random probability 0.50000000000

KUBE-SEP-PIHAV4BKXEANYKSM  all  --  anywhere             anywhere



The above rules state that if traffic comes to KUBE-SVC-QKJQYQZXY3DRLPVB chain there are three chain it can go KUBE-SEP-3UIFCWUMBYEVXRYP, KUBE-SEP-QSKKIZFW3PNDWYKJ or KUBE-SEP-PIHAV4BKXEANYKSM.

The probability of choosing any of the three chains is the same. Let's try to understand a bit about the probability values mentioned in the above rules.

  • As we know that IPTables rules are processed in order, Since there are three options when the control reaches the first rule since there are three options to chose the probability of choosing the first rule is 1/3 = 0.3333
  • When the control reaches the second rule, the choices remaining are 2, hence probability of choosing the second rule is 1/2 = 0.5000
  • And finally, when the control reaches 3rd rule, there are no more choices left so that is the only option. So the probability of choosing that at that point is 1.
Let dive into rules that are part of the above three chains

Chain KUBE-SEP-3UIFCWUMBYEVXRYP (1 references)

target     prot opt source               destination

KUBE-MARK-MASQ  all  --  100.96.0.2           anywhere

DNAT       udp  --  anywhere             anywhere             udp to:100.96.0.2:9153


Chain KUBE-SEP-QSKKIZFW3PNDWYKJ (1 references)

target     prot opt source               destination

KUBE-MARK-MASQ  all  --  100.96.32.2          anywhere

DNAT       udp  --  anywhere             anywhere             udp to:100.32.64.2:9153


Chain KUBE-SEP-PIHAV4BKXEANYKSM (1 references)

target     prot opt source               destination

KUBE-MARK-MASQ  all  --  100.96.64.2          anywhere

DNAT       udp  --  anywhere             anywhere             udp to:100.96.64.2:9153


finally, you must have understood each of the three chains contains a rule to redirect traffic to one of the pods.



HOPE YOU LIKED THE TUTORIAL. FEEL FREE TO COMMENT BELOW IF YOU HAVE ANY DOUBT. AND STAY TUNED FOR MORE TUTORIALS :)

Comments

Post a Comment

Popular posts from this blog

Lets learn "System design for paste bin (or any text sharing website)"

-
Image
  In this tutorial, we'll learn about the system design for PasteBin. If you don't know what Pastebin is here goes its definition. Pastebin is a type of online content hosting service where users can store texts to share with others over the internet.  Pastebin system design requirements  Functional Basic - User should be able to paste the text, generate the link and share the link with other users. There should be the max size of the text that our system will support (say 10Mb) Users should be able to provide the custom URL path for the link created. Like if a user wants to create the links  www.pb.com/letlearn, he can give this URL and pastebin should create a link using this URL to share with others. The link or paste expiry feature should be there for each stored text. There can be default expiry time say 1 year but this should be configurable. Personalization means the user should be able to login to track all the pastes created by him and manage them. Non-Functional Durab
Read more

Lets learn "Factory design pattern"

-
 In this tutorial, we'll learn about the factory design pattern as in what it is, in which use cases this is being used and how it works. If you've are not familiar with what are software design patterns and what are its different types check out this tutorial. Introduction A factory design pattern is a creational design pattern that is related to object creation. In this pattern, we'll have a method to create an object without exposing its creation logic to the client. If a class method is creating an object it has to be static.  The idea is to create a static method ( factory method ) which instantiates a class without exposing its logic to the client and the client can use that method every time it needs to create an object of the class. The reason why the factory pattern is useful is that as it does not expose the logic of object creation, so if logic is changed, the client code need not to be changed. Usecases The place where this pattern is mostly used is library cod
Read more