Kubernetes Best Practices for Production Environments

When deploying Kubernetes in production environments, following best practices is crucial for ensuring security, reliability, and operational efficiency. In this article, I’ll share key strategies based on my experience working with enterprise Kubernetes deployments.

1. Resource Management

Properly managing resources is essential for stable Kubernetes clusters.

Set Resource Requests and Limits

Always define resource requests and limits for your containers:

apiVersion: v1
kind: Pod
metadata:
  name: frontend
spec:
  containers:
  - name: app
    image: app:v1
    resources:
      requests:
        memory: "128Mi"
        cpu: "250m"
      limits:
        memory: "256Mi"
        cpu: "500m"

This helps the scheduler make better decisions and prevents resource contention.

Implement Resource Quotas

Use namespace resource quotas to prevent teams from consuming excessive cluster resources:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: team-quota
  namespace: team-a
spec:
  hard:
    pods: "10"
    requests.cpu: "4"
    requests.memory: 8Gi
    limits.cpu: "8"
    limits.memory: 16Gi

2. Security Best Practices

Use Network Policies

Network policies act as a firewall for pod communication. Always implement the principle of least privilege:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-allow
spec:
  podSelector:
    matchLabels:
      app: api
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: frontend
    ports:
    - protocol: TCP
      port: 8080

Enable RBAC with Least Privilege

Role-Based Access Control (RBAC) should be configured with minimal permissions:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

Use Pod Security Policies

Enforce security standards with Pod Security Policies:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restricted
spec:
  privileged: false
  allowPrivilegeEscalation: false
  requiredDropCapabilities:
    - ALL
  # ... more restrictions

3. High Availability Configuration

Deploy Multiple Control Plane Nodes

For production, always use at least three control plane nodes spread across availability zones.

Configure Pod Disruption Budgets

Protect your applications during voluntary disruptions:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: api-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: api

4. Monitoring and Observability

Implement Comprehensive Monitoring

Use Prometheus and Grafana for monitoring cluster and application metrics:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: api-service-monitor
spec:
  selector:
    matchLabels:
      app: api
  endpoints:
  - port: metrics
    interval: 15s

Set Up Proper Logging

Centralize logs using solutions like ELK Stack or Loki:

apiVersion: v1
kind: Pod
metadata:
  name: counter
spec:
  containers:
  - name: count
    image: busybox
    args:
    - /bin/sh
    - -c
    - >
      i=0;
      while true;
      do
        echo "$i: $(date)" >> /var/log/app.log;
        i=$((i+1));
        sleep 1;
      done
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  volumes:
  - name: varlog
    emptyDir: {}

5. Upgrade Strategy

Plan Cluster Upgrades Carefully

Always test upgrades in a staging environment first and have a rollback plan.

Use Rolling Updates for Applications

Configure deployments to use rolling updates:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  # ... rest of deployment

Conclusion

Implementing these best practices will help you build a more reliable, secure, and maintainable Kubernetes environment. Remember that Kubernetes is complex, and production readiness requires attention to many details beyond what’s covered here.

What Kubernetes best practices have you found most valuable in your production environments? Share your experiences in the comments below!

This article was written by Dijeesh Padinharethil, AWS Container Hero and Cloud Infrastructure Consultant. If you need help optimizing your Kubernetes environment, schedule a consultation.