Profile Applicability: Level 1
Do not disable timeouts on streaming connections.
Setting idle timeouts ensures that you are protected against Denial-of-Service attacks,
inactive connections and running out of ephemeral ports.
NoteBy default,
--streaming-connection-idle-timeout is set to 4 hours which might be too high for your environment. Setting this as appropriate
would additionally ensure that such streaming connections are timed out after serving
legitimate use cases. |
NoteSee the Azure AKS documentation for the default value.
|
Impact
Long-lived connections could be interrupted.
Audit
Audit Method 1:
-
SSH into the relevant node and run the following command on each node to find the running kubelet process:
ps -ef | grep kubelet
- If the command line for the process includes the argument
streaming-connection-idle-timeoutverify that it is not set to 0. -
If the
streaming-connection-idle-timeoutargument is not present in the output of the above command, refer instead to theconfigargument that specifies the location of the Kubelet config file e.g.--config /etc/kubernetes/kubelet/kubelet-config.json.
- If the command line for the process includes the argument
-
Open the Kubelet config file:
cat /etc/kubernetes/kubelet/kubelet-config.json
Verify that thestreamingConnectionIdleTimeoutargument is not set to"0".
Audit Method 2:
If using the API configz endpoint, consider searching for the status of
"streamingConnectionIdleTimeout":"4h0m0s" by extracting the live configuration from the nodes running Kubelet.-
Set the local proxy port and the following variables, and provide the proxy port number and node name:
HOSTNAME_PORT="localhost-and-port-number"NODE_NAME="The-Name-Of-Node-To-Extract-Configuration" from the output of "kubectl get nodes"
kubectl proxy --port=8001 & export HOSTNAME_PORT=localhost:8001 (example host and port number) export NODE_NAME=ip-192.168.31.226.aks.internal (example node name from "kubectl get nodes") curl -sSL "http://${HOSTNAME_PORT}/api/v1/nodes/${NODE_NAME}/proxy/configz"
Remediation
Remediation Method 1:
If modifying the Kubelet config file, edit
/etc/kubernetes/kubelet/kubelet-config.json and set the below parameter to a non-zero value in the format of #h#m#s:"streamingConnectionIdleTimeout": "4h0m0s"
Ensure that the kubelet service file
/etc/systemd/system/kubelet.service.d/10-kubelet-args.conf does not specify a --streaming-connection-idle-timeout argument because it would override the Kubelet config file.Remediation Method 2:
If using executable arguments, edit the Kubelet service file
/etc/systemd/system/kubelet.service.d/10-kubelet-args.conf on each worker node and add the below parameter at the end of the KUBELET_ARGS variable string:--streaming-connection-idle-timeout=4h0m0s
Remediation Method 3:
If using the API configz endpoint, consider searching for the status of
"streamingConnectionIdleTimeout": by extracting the live configuration from the nodes running Kubelet.See step-by-step configmap procedures in the Kubernetes documentation, and rerun the curl statement from the audit process to check for kubelet configuration
changes:
kubectl proxy --port=8001 &
export HOSTNAME_PORT=localhost:8001 (example host and port number)
export NODE_NAME=ip-192.168.31.226.aks.internal (example node name from
"kubectl get nodes")
curl -sSL "http://${HOSTNAME_PORT}/api/v1/nodes/${NODE_NAME}/proxy/configz"
For all remediations:
Restart the
kubelet service and verify the status: systemctl daemon-reload
systemctl restart kubelet.service
systemctl status kubelet -l
