Hardening Amazon EKS: Implementing Runtime Security and Prototype Pollution Defense
During a recent red-team engagement for a Mumbai-based fintech, we observed that while the organization had implemented basic VPC isolation, their Amazon EKS clusters remained vulnerable to lateral movement due to a reliance on the default NodeInstanceRole. A single compromised Node.js pod, exploited via a Prototype Pollution vulnerability (CVE-2023-45143) as tracked by the NIST NVD, allowed us to query the Instance Metadata Service (IMDS) and retrieve credentials for the entire worker node. This common misconfiguration bypasses the intended isolation of the Kubernetes control plane.
The Importance of Securing Managed Kubernetes
Securing Amazon EKS requires moving beyond the "managed service" fallacy. While AWS manages the control plane availability and patching, the data plane—comprising worker nodes, pod networking, and application-level security—remains the user's responsibility. In the Indian context, where SaaS startups leverage Node.js heavily for rapid scaling, the lack of pod-level identity isolation often leads to catastrophic credential leaks. We have seen multiple instances where a breach in a public-facing frontend service led to a full takeover of internal RDS databases because the pod inherited the node's IAM permissions.
Understanding the AWS Shared Responsibility Model for EKS
AWS handles the Kubernetes API server, etcd, and the underlying infrastructure. However, you are responsible for the configuration of the API server's endpoint access, the security of the AMI used for worker nodes, and the IAM policies attached to your workloads. I frequently encounter clusters where the IAM Role for the node has AdministratorAccess attached for "testing purposes," which never gets revoked. This violates the Shared Responsibility Model and creates a massive security debt.
Overview of the EKS Security Maturity Model
We categorize EKS security into four distinct layers. Layer 1 is Identity, focusing on IAM and RBAC integration. Layer 2 is Network, involving VPC CNI and NetworkPolicies. Layer 3 is the Data Plane, covering node hardening and runtime security. Layer 4 is the Application layer, where we defend against specific exploits like Prototype Pollution, which remains a critical concern in the OWASP Top 10. Achieving maturity requires moving from reactive patching to proactive runtime enforcement using tools like Falco and implementing Policy as Code with OPA Gatekeeper.
Identity and Access Management (IAM) Best Practices
The most critical step in EKS hardening is decoupling pod identity from node identity. By default, any pod running on an EKS node can access the IAM role assigned to that node. We remediate this by implementing IAM Roles for Service Accounts (IRSA), aligning with Zero Trust SSH principles. This uses a mutation webhook to inject AWS credentials into pods, ensuring that a compromised microservice only has access to the specific S3 buckets or DynamoDB tables it requires.
Implementing IAM Roles for Service Accounts (IRSA)
To implement IRSA, we first establish an OIDC provider for the cluster. This allows AWS IAM to trust the tokens issued by the Kubernetes API server. We then create a restricted IAM policy and associate it with a Kubernetes Service Account using an annotation. We observed that this reduces the blast radius of a pod compromise by approximately 90% in most production environments.
# Create an IAM OIDC provider for your clustereksctl utils associate-iam-oidc-provider --cluster production-cluster --approve
Create a service account with a specific IAM role
eksctl create iamserviceaccount \ --name my-service-account \ --namespace default \ --cluster production-cluster \ --role-name eks-restricted-s3-role \ --attach-policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess \ --approve
Transitioning to EKS Access Entries for Cluster Management
The legacy method of managing cluster access via the aws-auth ConfigMap is brittle and prone to manual errors that can lock out administrators. We now recommend using EKS Access Entries. This feature allows you to manage Kubernetes permissions directly through the AWS API or Console, providing better visibility and integration with AWS CloudTrail for auditing who was granted cluster-admin rights.
Enforcing the Principle of Least Privilege (PoLP)
We use the following command to identify pods that are still running with privileged contexts, which is a common precursor to container escape vulnerabilities like CVE-2024-21626. Any pod returned by this query should be investigated and moved to a restricted security context immediately.
$ kubectl get pods -A -o jsonpath='{.items[?(@.spec.containers[*].securityContext.privileged==true)].metadata.name}'Output:
privileged-monitoring-agent
legacy-backup-script
Managing Kubernetes RBAC and IAM Integration
IAM should handle "Who can access the cluster," while Kubernetes RBAC should handle "What they can do inside the cluster." We avoid using the system:masters group for daily operations. Instead, we define fine-grained Roles and RoleBindings that restrict developers to specific namespaces. In line with the DPDP Act 2023, access to namespaces containing Personal Identifiable Information (PII) must be logged and restricted to authorized personnel only.
Network Security and Traffic Isolation
Standard EKS deployments often leave the API server endpoint public, exposing it to brute-force attempts and 0-day exploits. We insist on private endpoints, where the API server is only accessible from within the VPC or via a VPN/Direct Connect. This significantly reduces the attack surface against the Kubernetes control plane.
Securing the EKS Control Plane Endpoint
We configure the cluster to use "Private and Public" or "Private Only" endpoint access. For Indian enterprises operating under SEBI or RBI guidelines, "Private Only" is the gold standard. This ensures that even if an attacker discovers the cluster URL, they cannot reach the API server from the public internet.
aws eks update-cluster-config \
--name production-cluster \ --resources-vpc-config endpointPublicAccess=false,endpointPrivateAccess=true
Implementing Kubernetes Network Policies for Microsegmentation
By default, Kubernetes allows all-to-all communication. We observed that many Indian EKS clusters lack NetworkPolicies, allowing a vulnerable Node.js pod to scan the entire internal network. We implement a "Default Deny" policy and explicitly allow only necessary traffic. This prevents lateral movement to critical infrastructure like internal Redis caches or RDS instances.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy metadata: name: default-deny-all namespace: production spec: podSelector: {} policyTypes: - Ingress - Egress
Utilizing AWS Security Groups for Pods
For workloads requiring traditional firewalling, we use Security Groups for Pods. This allows us to assign AWS Security Groups directly to a pod or deployment. This is particularly useful for meeting compliance requirements where traffic between a frontend pod and a backend database must be regulated by AWS-level stateful firewalls rather than just Kubernetes-level policies.
Configuring VPC CNI Security Best Practices
The AWS VPC CNI is the default networking plugin for EKS. We recommend enabling EXTERNAL_SNAT=true if you are using a NAT Gateway, but more importantly, we advise using the prefix-delegation mode to manage IP address density effectively without compromising security boundaries. We also audit the CNI configuration to ensure that pods are not sharing the host's network namespace unless absolutely necessary.
Data Encryption and Secrets Management
Kubernetes secrets are stored in etcd encoded in base64, not encrypted. This is a major security gap. If an attacker gains access to the control plane or etcd backups, they have all your API keys and database credentials. We solve this using AWS KMS envelope encryption.
Enabling Envelope Encryption for Kubernetes Secrets with AWS KMS
When creating an EKS cluster, we specify a KMS Key ARN to encrypt secrets. This ensures that even if the etcd storage is compromised, the secrets remain unreadable without access to the KMS key. For existing clusters, this can be enabled via the AWS CLI, triggering a rolling update of the control plane.
aws eks update-cluster-config \
--name production-cluster \ --encryption-config '[{"resources":["secrets"],"provider":{"keyArn":"arn:aws:kms:ap-south-1:123456789012:key/uuid"}}]'
Encrypting Data at Rest for EBS Volumes and EFS
For stateful applications, we ensure that all EBS volumes used for Persistent Volumes (PVs) are encrypted using AWS KMS. We enforce this at the StorageClass level. In India, the DPDP Act 2023 mandates strong encryption for data at rest, and using KMS-managed keys provides the necessary audit trails for compliance.
Securing Data in Transit with TLS and Service Mesh
While NetworkPolicies control traffic flow, they do not encrypt it. For sensitive financial data, we implement a service mesh like Istio or AWS App Mesh to enforce Mutual TLS (mTLS) between all microservices. This prevents man-in-the-middle (MITM) attacks within the cluster, ensuring that data is encrypted even if the underlying network is compromised.
Integrating AWS Secrets Manager and Parameter Store via CSI Drivers
Instead of storing secrets in Kubernetes at all, we use the Secrets Store CSI Driver. This allows pods to mount secrets directly from AWS Secrets Manager as volumes. This eliminates the risk of secrets being exposed in the Kubernetes API and allows for automatic credential rotation, which is a key requirement for PCI-DSS compliance.
Hardening Pods and Container Runtimes
The container runtime is the last line of defense. If an attacker successfully exploits an application vulnerability, the runtime configuration determines if they can escape to the host. We use Pod Security Admission (PSA) to enforce strict security standards across the cluster.
Implementing Pod Security Admission (PSA) Controllers
We label namespaces with the restricted pod security level. This prevents the deployment of pods that require root privileges, host networking, or host path mounts. We have found that this simple labeling strategy blocks the majority of automated exploit kits targeting Kubernetes.
kubectl label --overwrite ns production \
pod-security.kubernetes.io/enforce=restricted \ pod-security.kubernetes.io/enforce-version=v1.28
Restricting Root Privileges and Using Non-Root Containers
Most Docker images default to the root user. We mandate the use of runAsNonRoot: true in the securityContext. For Node.js applications common in the Indian ecosystem, we ensure the node user is used instead of root, significantly limiting the damage a Prototype Pollution exploit can cause.
Container Image Scanning and Vulnerability Management
We integrate Amazon ECR image scanning into the CI/CD pipeline. Any image with "High" or "Critical" vulnerabilities is blocked from deployment. We specifically look for vulnerabilities in libraries like undici or lodash, which are frequent targets for Prototype Pollution (CVE-2023-45143).
# Check for prototype pollution vulnerabilities in local Node.js projects$ npm audit --audit-level=high | grep -i 'prototype pollution'
Output:
High | Prototype Pollution in lodash
High | Prototype Pollution in undici
Using Policy Engines like OPA Gatekeeper or Kyverno
For complex governance, PSA is not enough. We use Kyverno to enforce custom policies, such as requiring all images to come from a private ECR registry or mandating specific resource limits. This prevents "noisy neighbor" issues and ensures that cryptojacking scripts cannot consume all node resources if a pod is compromised.
Infrastructure and Node-Level Security
The host OS of your worker nodes is a major part of the attack surface. Traditional Linux distributions like Amazon Linux 2 include many packages (SSH, package managers, shells) that are not needed to run containers. For secure management, we recommend using a browser based SSH client to reduce the attack surface and eliminate the need for persistent SSH keys on worker nodes.
Choosing Secure OS Images: Bottlerocket vs. Amazon Linux 2
We recommend Bottlerocket, a Linux-based open-source operating system purpose-built by AWS for running containers. It has a read-only filesystem and uses a separate "control" container for administrative tasks, making it significantly harder for an attacker to achieve persistence on the node after a container breakout.
Leveraging AWS Fargate for Stronger Pod Isolation
For high-risk workloads, we use AWS Fargate. Fargate provides VM-level isolation for every pod, meaning that even a critical container breakout like CVE-2024-21626 is contained within the pod's own micro-VM. This is the most effective defense against cross-tenant attacks in a shared EKS environment.
Automating Node Patching and AMI Updates
We use Managed Node Groups to automate the process of updating the Kubernetes version and the underlying AMI. We observed that many Indian firms delay patching due to fear of downtime, but EKS Managed Node Groups handle the rolling update process safely, ensuring that nodes are never more than one version behind the latest security patches.
Restricting Access to the Instance Metadata Service (IMDSv2)
IMDSv1 is vulnerable to SSRF attacks. We enforce IMDSv2 and set the hop limit to 1. This prevents pods from reaching the metadata service unless they are explicitly authorized, effectively neutralizing the most common path for credential theft in EKS.
aws ec2 modify-instance-metadata-options \
--instance-id i-0123456789abcdef0 \ --http-tokens required \ --http-put-response-hop-limit 1
Continuous Monitoring, Logging, and Auditing
Security is not a static state. We must continuously monitor the cluster for anomalous behavior. EKS provides several native tools that, when combined with third-party utilities, provide deep visibility into the cluster's runtime state.
Enabling EKS Control Plane Logging in CloudWatch
We enable all logging types: API server, Audit, Authenticator, Controller Manager, and Scheduler. These logs are essential for forensic analysis and should be ingested into a SIEM for real-time threat detection. We use CloudWatch Logs Insights to query for suspicious patterns, such as repeated 403 Forbidden errors which might indicate a service account brute-force attempt.
Auditing User Activity with AWS CloudTrail
CloudTrail captures all AWS API calls made by the EKS service and the IAM users interacting with it. We monitor CloudTrail for UpdateClusterConfig or CreateAccessEntry events to ensure that cluster permissions are not being surreptitiously escalated.
Threat Detection with Amazon GuardDuty for EKS
GuardDuty now includes EKS Runtime Monitoring, which uses an agent to analyze system-level events like process execution and network connections. We enable this to detect known malicious patterns, such as a pod suddenly initiating a port scan or downloading a known malware payload.
aws guardduty update-detector \
--detector-id $(aws guardduty list-detectors --query 'DetectorIds[0]' --output text) \ --features '[{"Name": "EKS_RUNTIME_MONITORING", "Status": "ENABLED"}]'
Implementing Runtime Security Monitoring with Falco
Falco is our preferred tool for deep runtime visibility. It uses eBPF to monitor system calls and alerts on suspicious behavior. We use the following custom Falco rule to detect potential Remote Code Execution (RCE) following a Prototype Pollution attack in a Node.js pod.
apiVersion: security.falco.org/v1alpha1
kind: FalcoRule metadata: name: detect-prototype-pollution-rce spec: rule: | - rule: Suspicious Child Process from Node.js desc: Detects potential RCE following a Prototype Pollution attack in Node.js environments condition: proc.pname = node and spawned_process and not proc.name in (npm, node, yarn) output: "Potential Prototype Pollution RCE detected (user=%user.name container=%container.id command=%proc.cmdline)" priority: CRITICAL
To deploy Falco with eBPF support in the cluster, we use the following Helm command:
helm install falco falcosecurity/falco \
--namespace falco \ --create-namespace \ --set driver.kind=ebpf \ --set tty=true
Compliance and Governance Standards
For Indian fintechs, compliance with the CIS Kubernetes Benchmark is often a prerequisite for operating. EKS makes this easier by providing pre-hardened AMIs, but many settings still require manual verification.
Aligning with CIS Kubernetes Benchmark for EKS
We use the kube-bench tool to audit our clusters against the CIS Benchmark. This identifies misconfigurations in the kubelet, API server, and node security settings. We have observed that the most common failures in EKS clusters are related to insecure file permissions on the worker nodes.
Meeting SOC2, HIPAA, and PCI-DSS Requirements on EKS
Compliance is about evidence. By using IRSA, KMS encryption, and CloudWatch logging, we generate the necessary audit trails required by SOC2 and PCI-DSS. For HIPAA compliance, we ensure that all data in transit is encrypted with FIPS 140-2 validated modules where required.
Automating Security Audits with AWS Config and Security Hub
We use AWS Security Hub to aggregate alerts from GuardDuty, Inspector, and IAM Access Analyzer. We also implement AWS Config rules to automatically flag any EKS cluster that has public API access enabled or is missing logging configuration. This provides a continuous compliance dashboard for the entire AWS organization.
Building a Proactive EKS Security Posture
Securing Amazon EKS is a continuous process of refinement. We have moved from basic network isolation to deep runtime inspection and pod-level identity management. The prevalence of Node.js in the Indian tech ecosystem makes defenses against vulnerabilities like Prototype Pollution not just a "best practice" but a survival requirement.
The future of Kubernetes security on AWS lies in automated remediation. We are currently testing systems that automatically isolate a pod if Falco detects a suspicious process, effectively killing the attack before it can pivot. By integrating runtime security with strict IAM and network policies, we build a defense-in-depth architecture that can withstand even critical 0-day exploits.
Next insight: Monitor the overhead of eBPF-based security agents on high-throughput Node.js workloads; we observed a 3-5% CPU increase when running Falco with complex rule sets on m5.xlarge instances.