Lets learn "About kube proxy in iptables mode"

-


In this tutorial, we'll learn about the Kube-proxy in iptables mode. Basically, we'll see how Kube-proxy create iptables that help in load balancing and service discovery. If you are not familiar with Linux IPTables I would recommend to check out my tutorial on IPTables here first and then come back to read this tutorial.

Kube-proxy

Kube-proxy when running in iptables mode basically watch kube-apiserver for new endpoints and create corresponding iptables rules and iptables are fully responsible for service discover and proper routing of traffic to corresponding pods. Kube-proxy needs to run on all k8s worker nodes.

Below is a small video describing the working of kube-proxy in iptables mode. 



Now that we know what Kube-proxy is and how it Kube-proxy works, let deep dive into the types of iptables Kube-proxy adds to the firewall.

Kube-proxy added IPTables

To learn this let's consider an example. Consider a k8s cluster with app FOO running (and listening on 9153 port )and 3 replicas of FOO is running in the cluster. So the 3 pods for app are as follows:

[root@worker1 ~]# kubectl get pods -o wide | grep FOO
FOO-7g26h               1/1     Running   2          3d13h   100.96.0.2    worker1.cluster.com     <none>      <none>
FOO-9r4mj                1/1     Running   2         3d13h   100.96.32.2     worker2.cluster.com   <none>     <none>
FOO-wjvrz                1/1     Running   2          3d13h   100.96.64.2     worker3.cluster.com   <none>     <none>

Let app FOO is exposed by a cluster IP service. And the virtual IP of the service is 100.96.128.2.

[root@worker1 ~]# kubectl get svc | grep FOO

NAME      TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                  AGE

FOO   ClusterIP   100.96.128.2   <none>        9153/TCP   46m


Since 100.96.128.2 is the virtual IP, IP tables should be added in a way that traffic to this VIP should be redirected of one of the pods IP ( 100.96.0.2, 100.96.32.2, and 100.96.64.2 )



Let's see the IPTables rules now.

>> iptables -t nat -L


Chain KUBE-SERVICES (2 references)

target     prot opt source               destination

  • KUBE-MARK-MASQ  tcp  -- !100.96.0.0/17        100.96.128.2         /* kube-system/FOO:foo cluster IP */ tcp dpt:9153

  • KUBE-SVC-QKJQYQZXY3DRLPVB  tcp  --  anywhere             100.96.128.2         /* kube-system/FOO:foo cluster IP */ tcp dpt:9153

  • KUBE-NODEPORTS  all  --  anywhere             anywhere             /* kubernetes service nodeports; NOTE: this must be the last rule in this chain */ ADDRTYPE match dst-type LOCAL


Above highlighted rules means if there is traffic destined to port 9153 will be redirected KUBE-SVC-QKJQYQZXY3DRLPVB chain. Let's see whats in this chain.

Chain KUBE-SVC-QKJQYQZXY3DRLPVB (1 references)

target     prot opt source               destination

KUBE-SEP-3UIFCWUMBYEVXRYP  all  --  anywhere             anywhere             statistic mode random probability 0.33333333349

KUBE-SEP-QSKKIZFW3PNDWYKJ  all  --  anywhere             anywhere             statistic mode random probability 0.50000000000

KUBE-SEP-PIHAV4BKXEANYKSM  all  --  anywhere             anywhere



The above rules state that if traffic comes to KUBE-SVC-QKJQYQZXY3DRLPVB chain there are three chain it can go KUBE-SEP-3UIFCWUMBYEVXRYP, KUBE-SEP-QSKKIZFW3PNDWYKJ or KUBE-SEP-PIHAV4BKXEANYKSM.

The probability of choosing any of the three chains is the same. Let's try to understand a bit about the probability values mentioned in the above rules.

  • As we know that IPTables rules are processed in order, Since there are three options when the control reaches the first rule since there are three options to chose the probability of choosing the first rule is 1/3 = 0.3333
  • When the control reaches the second rule, the choices remaining are 2, hence probability of choosing the second rule is 1/2 = 0.5000
  • And finally, when the control reaches 3rd rule, there are no more choices left so that is the only option. So the probability of choosing that at that point is 1.
Let dive into rules that are part of the above three chains

Chain KUBE-SEP-3UIFCWUMBYEVXRYP (1 references)

target     prot opt source               destination

KUBE-MARK-MASQ  all  --  100.96.0.2           anywhere

DNAT       udp  --  anywhere             anywhere             udp to:100.96.0.2:9153


Chain KUBE-SEP-QSKKIZFW3PNDWYKJ (1 references)

target     prot opt source               destination

KUBE-MARK-MASQ  all  --  100.96.32.2          anywhere

DNAT       udp  --  anywhere             anywhere             udp to:100.32.64.2:9153


Chain KUBE-SEP-PIHAV4BKXEANYKSM (1 references)

target     prot opt source               destination

KUBE-MARK-MASQ  all  --  100.96.64.2          anywhere

DNAT       udp  --  anywhere             anywhere             udp to:100.96.64.2:9153


finally, you must have understood each of the three chains contains a rule to redirect traffic to one of the pods.



HOPE YOU LIKED THE TUTORIAL. FEEL FREE TO COMMENT BELOW IF YOU HAVE ANY DOUBT. AND STAY TUNED FOR MORE TUTORIALS :)

Comments

Post a Comment

Popular posts from this blog

Lets learn "System design for paste bin (or any text sharing website)"

-
Image
  In this tutorial, we'll learn about the system design for PasteBin. If you don't know what Pastebin is here goes its definition. Pastebin is a type of online content hosting service where users can store texts to share with others over the internet.  Pastebin system design requirements  Functional Basic - User should be able to paste the text, generate the link and share the link with other users. There should be the max size of the text that our system will support (say 10Mb) Users should be able to provide the custom URL path for the link created. Like if a user wants to create the links  www.pb.com/letlearn, he can give this URL and pastebin should create a link using this URL to share with others. The link or paste expiry feature should be there for each stored text. There can be default expiry time say 1 year but this should be configurable. Personalization means the user should be able to login to track all the pastes created by him and manage them. Non-Functional Durab
Read more

Lets learn "How to add tables on blogger post"

-
Image
  In this article, we'll learn how to create a table on a blogger post. At first, I was surprised to see the blogger editor doesn't have support to add the tables in the blog post. Googling a bit mentioned to add a table change the View to "HTML View" and write HTML code to add a table. But what about if you don't know HTML which most of the bloggers don't ;) Below is an image highlighting the button to switch the view. Now let's learn how to insert a table on Compose View. There can be multiple ways to do that but the easiest method I found is using the google docs. Let's go through step by step on how to do that. 1. Open google docs from here . 2. Create a blank document.  3. Click the insert button. 4. Select "Table" and select the size of the table. 5. On selecting the table, a table will be generated on the doc. 6. This table is resizable, you can resize if you want. 7. Select and Copy the table 8. Paste the table on the blogger "Co
Read more