Kubernetes/Kubelet - Revision history

Pio2pio: /* Troubleshooting */

2019-10-10T07:26:00Z

Troubleshooting

← Older revision		Revision as of 09:26, 10 October 2019
Line 54:		Line 54:
	= Troubleshooting =		= Troubleshooting =
	<source lang=bash>		<source lang=bash>
			sudo journalctl -u kubelet
			sudo more syslog \| tail -120 \| grep kubelet

			# restart service
	sudo systemctl enable kubelet && systemctl start kubelet		sudo systemctl enable kubelet && systemctl start kubelet

Line 60:		Line 64:
	swapoff -a && sed -i '/ swap / s/^/#/' /etc/fstab		swapoff -a && sed -i '/ swap / s/^/#/' /etc/fstab
	</source>		</source>


	=References=		=References=
	*[https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ Kubelet] K8s docs		*[https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ Kubelet] K8s docs

Pio2pio: /* References */

2019-10-10T07:16:08Z

References

@@ Line 52: / Line 52: @@
    --key  /var/lib/minikube/certs/apiserver-kubelet-client.key
 </source>
 =References=
 *[https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ Kubelet] K8s docs

Pio2pio at 06:36, 29 August 2019

2019-08-29T06:36:15Z

← Older revision		Revision as of 08:36, 29 August 2019
Line 1:		Line 1:
	The kubelet is the primary “node agent” that runs on each node. The kubelet takes a set of PodSpecs that are provided through various mechanisms (primarily through the apiserver) and ensures that the containers described in those PodSpecs are running and healthy.		The kubelet is the primary “node agent” that runs on each node. The kubelet takes a set of PodSpecs that are provided through various mechanisms (primarily through the apiserver) and ensures that the containers described in those PodSpecs are running and healthy.


			Kubelet provides also interface to gether [https://kubernetes.io/docs/tasks/debug-application-cluster/resource-usage-monitoring/ resource metrics or full metrics pipelines] to collect monitoring statistics - used by autoscaling, monitoring and health checks solutions.

Pio2pio at 22:13, 28 August 2019

2019-08-28T22:13:31Z

← Older revision		Revision as of 00:13, 29 August 2019
Line 1:		Line 1:
	The kubelet is the primary “node agent” that runs on each node. The kubelet takes a set of PodSpecs that are provided through various mechanisms (primarily through the apiserver) and ensures that the containers described in those PodSpecs are running and healthy.		The kubelet is the primary “node agent” that runs on each node. The kubelet takes a set of PodSpecs that are provided through various mechanisms (primarily through the apiserver) and ensures that the containers described in those PodSpecs are running and healthy.


	A few interesting options:		A few interesting options:
Line 6:		Line 7:
	* <code>--healthz-port int32</code> :- the port of the localhost healthz endpoint (set to 0 to disable) (default 10248)		* <code>--healthz-port int32</code> :- the port of the localhost healthz endpoint (set to 0 to disable) (default 10248)
	* <code>--max-pods int32</code> :- number of Pods that can run on this Kubelet. (default 110)		* <code>--max-pods int32</code> :- number of Pods that can run on this Kubelet. (default 110)


	Check what options are applied, ssh to one of the nodes and check kubelet process		Check what options are applied, ssh to one of the nodes and check kubelet process
Line 36:		Line 38:
	"usageCoreNanoSeconds": 383367236057		"usageCoreNanoSeconds": 383367236057
	},		},
	~~"memory": {~~
	~~"time": "2019-08-28T21:32:41Z",~~
	~~"usageBytes": 530628608,~~
	~~"workingSetBytes": 269578240,~~
	~~"rssBytes": 94576640,~~
	~~"pageFaults": 3165428,~~
	~~"majorPageFaults": 0~~
	},
	~~"userDefinedMetrics": null~~
	},
	{
	~~"name": "pods",~~
	~~"startTime": "2019-08-28T21:13:59Z",~~
	~~"cpu": {~~
	~~"time": "2019-08-28T21:32:38Z",~~
	~~"usageNanoCores": 169576204,~~
	~~"usageCoreNanoSeconds": 565856265044~~
	...		...

	# ~~Https~~		# Minikube calling kubelet on https
			sudo curl https://localhost:10250/stats/summary -k \
			--cert /var/lib/minikube/certs/apiserver-kubelet-client.crt \
			--key /var/lib/minikube/certs/apiserver-kubelet-client.key

			sudo curl https://${HOSTNAME}:10250/stats/summary \
			--cert /var/lib/minikube/certs/apiserver-kubelet-client.crt \
			--key /var/lib/minikube/certs/apiserver-kubelet-client.key
	</source>		</source>
	=References=		=References=
	*[https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ Kubelet] K8s docs		*[https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ Kubelet] K8s docs

Pio2pio: Created page with "The kubelet is the primary “node agent” that runs on each node. The kubelet takes a set of PodSpecs that are provided through various mechanisms (primarily through the api..."

2019-08-28T21:44:38Z

Created page with "The kubelet is the primary “node agent” that runs on each node. The kubelet takes a set of PodSpecs that are provided through various mechanisms (primarily through the api..."

New page

The kubelet is the primary “node agent” that runs on each node. The kubelet takes a set of PodSpecs that are provided through various mechanisms (primarily through the apiserver) and ensures that the containers described in those PodSpecs are running and healthy.

A few interesting options:
* <code>--port int32</code> :- the port for the Kubelet to serve on. (default 10250)
* <code>--read-only-port int32</code> :- the read-only port for the Kubelet to serve on with no authentication/authorization (set to 0 to disable) (default 10255)
* <code>--healthz-port int32</code> :- the port of the localhost healthz endpoint (set to 0 to disable) (default 10248)
* <code>--max-pods int32</code> :- number of Pods that can run on this Kubelet. (default 110)

Check what options are applied, ssh to one of the nodes and check kubelet process
<source lang=bash>
kubectl get nodes -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
minikube Ready master 45m v1.15.2 10.0.2.15 <none> Buildroot 2018.05.3 4.15.0 docker://18.9.8

ps -aux | grep kubelet
root 3409 8.3 4.8 1353596 96020 ? Ssl 20:54 2:40 /usr/bin/kubelet --authorization-mode=Webhook --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroup-driver=cgroupfs --client-ca-file=/var/lib/minikube/certs/ca.crt --cluster-dns=10.96.0.10 --cluster-domain=cluster.local --container-runtime=docker --fail-swap-on=false --hostname-override=minikube --kubeconfig=/etc/kubernetes/kubelet.conf --pod-manifest-path=/etc/kubernetes/manifests
root 3819 8.5 14.2 471736 284396 ? Ssl 20:54 2:43 kube-apiserver --advertise-address=192.168.99.104 --allow-privileged=true --authorization-mode=Node,RBAC --client-ca-file=/var/lib/minikube/certs/ca.crt --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota --enable-bootstrap-token-auth=true --etcd-cafile=/var/lib/minikube/certs/etcd/ca.crt --etcd-certfile=/var/lib/minikube/certs/apiserver-etcd-client.crt --etcd-keyfile=/var/lib/minikube/certs/apiserver-etcd-client.key --etcd-servers=https://127.0.0.1:2379 --insecure-port=0 --kubelet-client-certificate=/var/lib/minikube/certs/apiserver-kubelet-client.crt --kubelet-client-key=/var/lib/minikube/certs/apiserver-kubelet-client.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --proxy-client-cert-file=/var/lib/minikube/certs/front-proxy-client.crt --proxy-client-key-file=/var/lib/minikube/certs/front-proxy-client.key --requestheader-allowed-names=front-proxy-client --requestheader-client-ca-file=/var/lib/minikube/certs/front-proxy-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=8443 --service-account-key-file=/var/lib/minikube/certs/sa.pub --service-cluster-ip-range=10.96.0.0/12 --tls-cert-file=/var/lib/minikube/certs/apiserver.crt --tls-private-key-file=/var/lib/minikube/certs/apiserver.key
docker 18542 0.0 0.0 9240 476 pts/0 S+ 21:26 0:00 grep kubelet
</source>

Check check ports
<source lang=bash>
# Http
curl http://localhost:10255/stats/summary
{
"node": {
"nodeName": "minikube",
"systemContainers": [
{
"name": "runtime",
"startTime": "2019-08-28T21:14:07Z",
"cpu": {
"time": "2019-08-28T21:32:41Z",
"usageNanoCores": 80916024,
"usageCoreNanoSeconds": 383367236057
},
"memory": {
"time": "2019-08-28T21:32:41Z",
"usageBytes": 530628608,
"workingSetBytes": 269578240,
"rssBytes": 94576640,
"pageFaults": 3165428,
"majorPageFaults": 0
},
"userDefinedMetrics": null
},
{
"name": "pods",
"startTime": "2019-08-28T21:13:59Z",
"cpu": {
"time": "2019-08-28T21:32:38Z",
"usageNanoCores": 169576204,
"usageCoreNanoSeconds": 565856265044
...

# Https

</source>
=References=
*[https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ Kubelet] K8s docs