freeleaps-ops/cluster/ansible

Freeleaps Cluster Maintain Ansible

This folder used to Freeleaps Kubernetes Cluster maintaining.

Pre-settings for each node

Enable br_netfilter module for Kernel

We need enable br_netfilter module for each node's kernel.

Using command modprobe br_netfilter on each nodes to enable it.

Enable HugePage on node's kernel

# Check nodes HugePage params if not 1024
sudo grep HugePages /proc/meminfo

# Add HugePage params into kernel settings
echo 1024 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
echo vm.nr_hugepages = 1024 | sudo tee -a /etc/sysctl.conf

Change Kernel Params

You need ensure kernel params (/etc/sysctl.conf) like belows:

net.ipv4.ip_forward=1
kernel.keys.root_maxbytes=25000000
kernel.keys.root_maxkeys=1000000
kernel.panic=10
kernel.panic_on_oops=1
vm.overcommit_memory=1
vm.panic_on_oom=0
net.ipv4.ip_local_reserved_ports=30000-32767
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-arptables=1
net.bridge.bridge-nf-call-ip6tables=1
vm.nr_hugepages = 1024

Using sysctl -p to make kernel params effect.

How to use ?

Cluster Scaling

If you want to scale nodes in cluster, you can execute script upgrade-cluster.sh after environment initialized.

Tips - ANONYMOUS AUTH ENABLING BEFORE SCALE

kubeadm using bootstrap token to join newly added nodes into cluster, which means we need enable anonymous authentication for kube-apiserver temporally.

You need edit /etc/kubernetes/manifests/kube-apiserver.yaml on master-01 node and set --anonymous-auth=true and you must recovery it to false when cluster has been scaled.