Kubernetes
WIP
Kubernetes
It’s orchestration engine and gives basic primitives to orchestrate application deployments on a low level -such as pods, jobs, deployments, services, ingress, persisten volumes, volume claims, secrets, configmaps, daemon, and statefulsets etc..
Basics
If you like cars, before diving into engine specs and horsepower, you first need to know how to drive. In this blog is no different—so before we get under the hood with intermediate concepts, let’s briefly touch on the basics to make sure we’re all on the same page. Promise we’ll keep it quick!
Setup
Verify
kubectl config view # tells which cluster it talks to
kubectl config view --raw #
kubectl config get-contexts $(kubectl config current-context) # details
kubectl config use-context #
kubectl api-resources --api-group=apps --namespaced=false
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJkekNDQVIyZ0F3SUJBZ0lCQURBS0JnZ3Foa2pPUFFRREFqQWpNU0V3SHdZRFZRUUREQmhyTTNNdGMyVnkKZG1WeUxXTmhRREUzT0RRNE16Y3dNalV3SGhjTk1qWXdOekl6TWpBd016UTFXaGNOTXpZd056SXdNakF3TXpRMQpXakFqTVNFd0h3WURWUVFEREJock0zTXRjMlZ5ZG1WeUxXTmhRREUzT0RRNE16Y3dNalV3V1RBVEJnY3Foa2pPClBRSUJCZ2dxaGtqT1BRTUJCd05DQUFRZGxhNnpjaFljbklkVHN3M2U2NmNocG1kL2dEdWdJMkx5UEY3L1BTOXcKWDNsUVRyWnNmaCtsNDZ0eHdiekxyQmVyR3pLL3lDak5tS05PWXFQMlhHK1FvMEl3UURBT0JnTlZIUThCQWY4RQpCQU1DQXFRd0R3WURWUjBUQVFIL0JBVXdBd0VCL3pBZEJnTlZIUTRFRmdRVUo3LzlwZmE1NStkS1ZxZXZTdjRaCm9GaURyZmt3Q2dZSUtvWkl6ajBFQXdJRFNBQXdSUUloQUpjWkRzUzJuUzdOcG9XWHNPUkN0RGdUSER6YS9FdkEKVm1vTG5HSlA0MjhkQWlBcVlNajUyNXhiUFlNVE90WmlsVnZrd1U4bUI4OUtJVkF2VSthZUhBSERlUT09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
server: https://127.0.0.1:6443
name: default
contexts:
- context:
cluster: default
user: default
name: default
current-context: default
kind: Config
users:
- name: default
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJrVENDQVRlZ0F3SUJBZ0lJSGs0NGVNd2lGU013Q2dZSUtvWkl6ajBFQXdJd0l6RWhNQjhHQTFVRUF3d1kKYXpOekxXTnNhV1Z1ZEMxallVQXhOemcwT0RNM01ESTFNQjRYRFRJMk1EY3lNekl3TURNME5Wb1hEVEkzTURjeQpNekl3TURNME5Wb3dNREVYTUJVR0ExVUVDaE1PYzNsemRHVnRPbTFoYzNSbGNuTXhGVEFUQmdOVkJBTVRESE41CmMzUmxiVHBoWkcxcGJqQlpNQk1HQnlxR1NNNDlBZ0VHQ0NxR1NNNDlBd0VIQTBJQUJFRHZzNVFOL0cwTHRVVGIKc3p4WW43aXYyeTZSY1drakdmc3lVSjdzaVYrYlVBcXpLNXNKa3RIWWNpcUJDMGRndU5UekwxeDYvNjNsTWVUWQpwYk5pclB1alNEQkdNQTRHQTFVZER3RUIvd1FFQXdJRm9EQVRCZ05WSFNVRUREQUtCZ2dyQmdFRkJRY0RBakFmCkJnTlZIU01FR0RBV2dCU2NhOTk3MWVkMmhBUHJNOUlTMnRsbmtkc1dxVEFLQmdncWhrak9QUVFEQWdOSUFEQkYKQWlCZXBXT2hHSmtvdWk1RVdvbzMxZWFpSDF1MGROYzlrQkdIZmI1ejQ3czlId0loQUtLUTZsM3JWVFhLK1hTdQpuWkUzbUJ6U2Vvd1R1bENQdDBUd0lvY1lmODJnCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0KLS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJlRENDQVIyZ0F3SUJBZ0lCQURBS0JnZ3Foa2pPUFFRREFqQWpNU0V3SHdZRFZRUUREQmhyTTNNdFkyeHAKWlc1MExXTmhRREUzT0RRNE16Y3dNalV3SGhjTk1qWXdOekl6TWpBd016UTFXaGNOTXpZd056SXdNakF3TXpRMQpXakFqTVNFd0h3WURWUVFEREJock0zTXRZMnhwWlc1MExXTmhRREUzT0RRNE16Y3dNalV3V1RBVEJnY3Foa2pPClBRSUJCZ2dxaGtqT1BRTUJCd05DQUFUQnlFd2pKVDF6bVN0OVBuR1MranZGNkN0NklTVkdnTTg0bkNVWGxRZUIKNHVGT3FlZUxheDA4eENRTGozZXpBKzJCNloxaWp2dXpGSkl3QXpoeEJVTm5vMEl3UURBT0JnTlZIUThCQWY4RQpCQU1DQXFRd0R3WURWUjBUQVFIL0JBVXdBd0VCL3pBZEJnTlZIUTRFRmdRVW5HdmZlOVhuZG9RRDZ6UFNFdHJaClo1SGJGcWt3Q2dZSUtvWkl6ajBFQXdJRFNRQXdSZ0loQUxvbnhMeWVpWVYzbXd4M2JoY25mVFZnd0RQbkhxOXkKY2pHaU9PVEpTQ3JpQWlFQTduZ0NGNC9CZ3JJQzhTaDJGaEgvd0V2ZHFSTXZ5czdRd3RWQjFCWWdOUFU9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
client-key-data: LS0tLS1CRUdJTiBFQyBQUklWQVRFIEtFWS0tLS0tCk1IY0NBUUVFSUlJcWJXd2FiemFwWGxYRVBwMGswa21RditqWjNpYkxVOGNvd09jaTBHUVJvQW9HQ0NxR1NNNDkKQXdFSG9VUURRZ0FFUU8remxBMzhiUXUxUk51elBGaWZ1Sy9iTHBGeGFTTVorekpRbnV5Slg1dFFDck1ybXdtUwowZGh5S29FTFIyQzQxUE12WEhyL3JlVXg1TmlsczJLcyt3PT0KLS0tLS1FTkQgRUMgUFJJVkFURSBLRVktLS0tLQo=
kubectl api-resources
NAME SHORTNAMES APIVERSION NAMESPACED KIND
bindings v1 true Binding
componentstatuses cs v1 false ComponentStatus
configmaps cm v1 true ConfigMap
endpoints ep v1 true Endpoints
events ev v1 true Event
limitranges limits v1 true LimitRange
namespaces ns v1 false Namespace
nodes no v1 false Node
persistentvolumeclaims pvc v1 true PersistentVolumeClaim
persistentvolumes pv v1 false PersistentVolume
pods po v1 true Pod
podtemplates v1 true PodTemplate
replicationcontrollers rc v1 true ReplicationController
resourcequotas quota v1 true ResourceQuota
secrets v1 true Secret
serviceaccounts sa v1 true ServiceAccount
services svc v1 true Service
mutatingwebhookconfigurations admissionregistration.k8s.io/v1 false MutatingWebhookConfiguration
validatingadmissionpolicies admissionregistration.k8s.io/v1 false ValidatingAdmissionPolicy
validatingadmissionpolicybindings admissionregistration.k8s.io/v1 false ValidatingAdmissionPolicyBinding
validatingwebhookconfigurations admissionregistration.k8s.io/v1 false ValidatingWebhookConfiguration
customresourcedefinitions crd,crds apiextensions.k8s.io/v1 false CustomResourceDefinition
apiservices apiregistration.k8s.io/v1 false APIService
controllerrevisions apps/v1 true ControllerRevision
daemonsets ds apps/v1 true DaemonSet
deployments deploy apps/v1 true Deployment
replicasets rs apps/v1 true ReplicaSet
statefulsets sts apps/v1 true StatefulSet
selfsubjectreviews authentication.k8s.io/v1 false SelfSubjectReview
tokenreviews authentication.k8s.io/v1 false TokenReview
localsubjectaccessreviews authorization.k8s.io/v1 true LocalSubjectAccessReview
selfsubjectaccessreviews authorization.k8s.io/v1 false SelfSubjectAccessReview
selfsubjectrulesreviews authorization.k8s.io/v1 false SelfSubjectRulesReview
subjectaccessreviews authorization.k8s.io/v1 false SubjectAccessReview
horizontalpodautoscalers hpa autoscaling/v2 true HorizontalPodAutoscaler
cronjobs cj batch/v1 true CronJob
jobs batch/v1 true Job
certificatesigningrequests csr certificates.k8s.io/v1 false CertificateSigningRequest
leases coordination.k8s.io/v1 true Lease
endpointslices discovery.k8s.io/v1 true EndpointSlice
events ev events.k8s.io/v1 true Event
flowschemas flowcontrol.apiserver.k8s.io/v1 false FlowSchema
prioritylevelconfigurations flowcontrol.apiserver.k8s.io/v1 false PriorityLevelConfiguration
gatewayclasses gc gateway.networking.k8s.io/v1 false GatewayClass
gateways gtw gateway.networking.k8s.io/v1 true Gateway
grpcroutes gateway.networking.k8s.io/v1 true GRPCRoute
httproutes gateway.networking.k8s.io/v1 true HTTPRoute
referencegrants refgrant gateway.networking.k8s.io/v1beta1 true ReferenceGrant
helmchartconfigs helm.cattle.io/v1 true HelmChartConfig
helmcharts helm.cattle.io/v1 true HelmChart
accesscontrolpolicies hub.traefik.io/v1alpha1 false AccessControlPolicy
aiservices hub.traefik.io/v1alpha1 true AIService
apiaccesses hub.traefik.io/v1alpha1 true APIAccess
apibundles hub.traefik.io/v1alpha1 true APIBundle
apicatalogitems hub.traefik.io/v1alpha1 true APICatalogItem
apiplans hub.traefik.io/v1alpha1 true APIPlan
apiportals hub.traefik.io/v1alpha1 true APIPortal
apiratelimits hub.traefik.io/v1alpha1 true APIRateLimit
apis hub.traefik.io/v1alpha1 true API
apiversions hub.traefik.io/v1alpha1 true APIVersion
managedsubscriptions hub.traefik.io/v1alpha1 true ManagedSubscription
addons k3s.cattle.io/v1 true Addon
etcdsnapshotfiles k3s.cattle.io/v1 false ETCDSnapshotFile
nodes metrics.k8s.io/v1beta1 false NodeMetrics
pods metrics.k8s.io/v1beta1 true PodMetrics
ingressclasses networking.k8s.io/v1 false IngressClass
ingresses ing networking.k8s.io/v1 true Ingress
ipaddresses ip networking.k8s.io/v1 false IPAddress
networkpolicies netpol networking.k8s.io/v1 true NetworkPolicy
servicecidrs networking.k8s.io/v1 false ServiceCIDR
runtimeclasses node.k8s.io/v1 false RuntimeClass
poddisruptionbudgets pdb policy/v1 true PodDisruptionBudget
clusterrolebindings rbac.authorization.k8s.io/v1 false ClusterRoleBinding
clusterroles rbac.authorization.k8s.io/v1 false ClusterRole
rolebindings rbac.authorization.k8s.io/v1 true RoleBinding
roles rbac.authorization.k8s.io/v1 true Role
deviceclasses resource.k8s.io/v1 false DeviceClass
resourceclaims resource.k8s.io/v1 true ResourceClaim
resourceclaimtemplates resource.k8s.io/v1 true ResourceClaimTemplate
resourceslices resource.k8s.io/v1 false ResourceSlice
priorityclasses pc scheduling.k8s.io/v1 false PriorityClass
csidrivers storage.k8s.io/v1 false CSIDriver
csinodes storage.k8s.io/v1 false CSINode
csistoragecapacities storage.k8s.io/v1 true CSIStorageCapacity
storageclasses sc storage.k8s.io/v1 false StorageClass
volumeattachments storage.k8s.io/v1 false VolumeAttachment
volumeattributesclasses vac storage.k8s.io/v1 false VolumeAttributesClass
ingressroutes traefik.io/v1alpha1 true IngressRoute
ingressroutetcps traefik.io/v1alpha1 true IngressRouteTCP
ingressrouteudps traefik.io/v1alpha1 true IngressRouteUDP
middlewares traefik.io/v1alpha1 true Middleware
middlewaretcps traefik.io/v1alpha1 true MiddlewareTCP
serverstransports traefik.io/v1alpha1 true ServersTransport
serverstransporttcps traefik.io/v1alpha1 true ServersTransportTCP
tlsoptions traefik.io/v1alpha1 true TLSOption
tlsstores traefik.io/v1alpha1 true TLSStore
traefikservices traefik.io/v1alpha1 true TraefikService
K8S 101
- Container
A sealed application package (Docker)
- Pod
The smallest and simplest Kubernetes object. It represents a single instance of a running process in your cluster. pods are one or more containers
kubectl run web-pod --image=gcr.io/google-samples/kubernetes-bootcamp:v1 --dry-run=client -o yml > pod.yml
- Qos: Settings
requestsandlimitsinfluence thekubeletto make decision as to which pod to evicit first in the event of resource starvationrequests:: min amount of resource that pod needs.limits:: define the max amount of resources that you need to supply for a given pod.- Guaranteed:: Every container must have both CPU and memory limits and requests defined.
requests == limits - Burstable:: At least one container has a CPU or memory request that does not equal its limit.
requests defined - Best Effort:: No containers have any CPU or memory requests or limits defined.
1st one get killed if resource starvation
- Guaranteed:: Every container must have both CPU and memory limits and requests defined.
- Pod Priority and preemption: Now, what happens if high priority comes when there are no nodes with respective pods? The scheduler will remove low-priority pods
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: high-priority-apps
value: 1000000
# preemptionPolicy: Never # Non-preempting PriorityClass
globalDefault: false
description: "This priority class is reserved for mission-critical core applications."
---
apiVersion: v1
kind: Pod
metadata:
name: critical-api-pod
labels:
app: api-service
spec:
priorityClassName: high-priority-apps # Linking the pod-priority
containers:
- name: web-app
image: nginx:latest
# defining resource and limits
resources:
requests:
memory: "256Mi"
cpu: "500m"
limits:
memory: "512Mi"
cpu: "1000m"
- Sharable Resources that can be shared among different consumers limited when required. CPU is sharable resources, it pod wants more than limit, it will not get terminated. Kubelet throttles the container, leads to negative performance.
-
Non sharable resources that cannot be shared by nature, Memory. when container exceeds it will get killed OOMKilled
- Health Checks:
Liveness Probe,Readiness ProbeandStartup Probe
Application Running
│
▼
Liveness Probe
│
Healthy?
/ \
Yes No
│ │
│ Restart Container
▼
Continue Running
|
Application Starting
│
▼
Readiness Probe
│
Ready?
/ \
Yes No
│ │
│ Don't send traffic
▼
Receive traffic
|
- Labels
Identify metadata attached to objects. use to determin which objects to apply an operation to primitive objects
apiVersion: v1
kind: Pod
metadata:
name: dev-fe
labels:
app: nify
phase: dev
role: fe
spec:
containers:
- name: nginx
image: nginx:alpine
---
apiVersion: v1
kind: Pod
metadata:
name: dev-be
labels:
app: nify
phase: dev
role: be
spec:
containers:
- name: python
image: python:3.12-alpine
command: ["python", "-m", "http.server", "8000"]
---
apiVersion: v1
kind: Pod
metadata:
name: test-fe
labels:
app: nify
phase: test
role: fe
spec:
containers:
- name: httpd
image: httpd:alpine
---
apiVersion: v1
kind: Pod
metadata:
name: test-be
labels:
app: nify
phase: test
role: be
spec:
containers:
- name: golang
image: golang:1.24-alpine
command:
- sh
- -c
- |
cat <<'EOF' > /tmp/server.go
package main
import (
"fmt"
"net/http"
)
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "Hello from Go!")
})
http.ListenAndServe(":8080", nil)
}
EOF
go run /tmp/server.go
- Selector
Query against labels, producing a set result
apiVersion: v1
kind: Service
metadata:
name: nify-service
spec:
selector:
app: nify
ports:
- port: 80
targetPort: 80
type: ClusterIP
- Query
App=Nifty
apiVersion: v1
kind: Service
metadata:
name: nify-fe-service
spec:
selector:
app: nify
role: fe
ports:
- port: 80
targetPort: 80
type: ClusterIP
- Query
App=NiftyandRole=FE
apiVersion: v1
kind: Service
metadata:
name: nify-dev-service
spec:
selector:
app: nify
phase: dev
ports:
- port: 80
targetPort: 80
type: ClusterIP
- Query
App=NiftyandPhase=Dev
To test the connection
kubectl port-forward service/SERVICE_NAME 8080:80
- Controller
A reconcilation loop that drives current state towards desired state
while(true) {
desired = API Server
actual = Cluster
if desired != actual {
reconcile()
}
}
- Replica Set
- Deployment
Grp of pods of the same type together to achieve load balancing. Greate for stateless workload, where exact copies of app runs and destory, maintain desired number of apps.
- Rollout
- Service
A set of pods that work together in deployment and Service helps expose your deployment. This exposure can be to other deployments and/or to the outside world.
- Types of Services
- ClusterIP
- NodePort
- LoadBalancer
- ConfigMap
- Secrets
- Namespace
It provide a mechanism for isolating groups of resources within a single cluster.
kubectl create ns NAMESPACE
kubectl config set-context --current --namespace=NAMESPACE
- StatefulSets
Similar to deployments but used for applications where copies of same application must coordianate with each other to maintain state. It manage the lifecycle of unique copies of pods. make sure networking & storage are reused if unhealthy pod need to be replaced.
- Volumes
Kubernets Components
- API Server: responsible for communication with kubelet on the worker nodes.
Authentication,Authorizationof the requestor - Etcd: key-value store of the critical state of system. Distributed Core Logic snapsotting the status of the k8s cluster
- Kube Controller Manager: responsible for monitoring the shared state of cluster through apiserver and making it to desired state
- Kube Scheduler: responsbile for select the worker node for a POD, and provision on target node according to resource specification.
