K
Q

kube-router assigning ClusterIPs outside of default subnet

1/5/2021

I have in my home lab a default installation of Kubernetes with kube-router as the network provider. kube-router is, as default, set as the service proxy. I have not set an explicit service-cluster-ip-network in my kube-controller-manager, so kube-router should be assigning service cluster IPs only within the default 10.96.x.x/16 subnet. However, I am regularly getting service cluster IPs anywhere within the larger 10.x.x.x./8 subnet. I am at a loss where/why it's not remaining within 10.96.x.x. Ideas? Thanks!

-- Scott Balmos
kubernetes
networking

Similar Questions

How can I control which data gets reported to the Istio Mixer by the side car?
egress policy not getting applied with kubernetes 1.9
kubectl command to get list of pods running on k8s master node
Kubernetes node can't reach external Kafka Cluster
Kubernetes v1.8.12 can't list *extensions.Deployment

2 Answers

12/18/2021

For this issue, I was able to find the file /etc/kubernetes/manifests/kube-apiserver.yaml on the master node and correct the service-cluster-ip-range line.

In my case, it appears that a kubeadm command to install the descheduler modified the line.

-- Jeff Voight
Source: StackOverflow
1/18/2021

TL;DR

Your Kubernetes cluster is behaving correctly.

By default (if not specified otherwise) using kubeadm to provision your cluster, the --service-cidr is set to 10.96.0.0/12.

ClusterIP address like 10.110.15.13 would be included in the above mentioned network (10.96.0.0/12).

I've provided more explanation below:

Subnetting

If you use one of the available online IP calculators you will be seeing exact same situation like the one included below:

CIDR10.96.0.0/12
Subnet mask255.240.0.0
Network address (first)10.96.0.0
Broadcast address (last)10.111.255.255
First useable address10.96.0.1
Last useable address10.111.255.254
Number of hosts allocatable1048574

By above diagram you can see that the Service IP range would be following:

  • 10.96.0.1-10.111.255.254

This would make IP's like: 10.104.5.2, 10.110.15.13 be in range of above network.

Kubernetes --service-cidr

As said earlier if you don't specify the --service-cidr when using $ kubeadm init it will be set to default 10.96.0.0/12.

Following the official documentation of kubeadm: 

--service-cidr string     Default: "10.96.0.0/12"
Use alternative range of IP address for service VIPs.

-- Kubernetes.io: Docs: Reference: Setup tools: Kubeadm: Kubeadm init: Options

If you provision the cluster without this parameter, you will be able to see it configured in the:

  • kube-apiserver: 
$ kubectl get pods -n kube-system   kube-apiserver-kubernetes-NODE_NAME -o yaml | grep "service-cluster-ip-range" 
    - --service-cluster-ip-range=10.96.0.0/12
  • kube-controller-manager
$ kubectl get pods -n kube-system   kube-controller-manager-kubernetes-NODE_NAME -o yaml | grep "service-cluster-ip-range" 
    - --service-cluster-ip-range=10.96.0.0/12

Kube-router

It's also explicitly stated in the kube-router's source code: 

func NewKubeRouterConfig() *KubeRouterConfig {
	return &KubeRouterConfig{
		// SKIPPED 
		ClusterIPCIDR:                  "10.96.0.0/12",
       // SKIPPED
	}
}

-- Github.com: Cloudnativelabds: Kube-router: Pkg: Options: Options.go: Line 73

	fs.StringVar(&s.ClusterIPCIDR, "service-cluster-ip-range", s.ClusterIPCIDR,
		"CIDR value from which service cluster IPs are assigned. Default: 10.96.0.0/12")

-- Github.com: Cloudnativelabds: Kube-router: Pkg: Options: Options.go: Line 187

It's also referenced in the user guide.

-- Dawid Kruk
Source: StackOverflow