Opscraft
Jenkins · Lesson 2 of 11 · 5 min read

Agents in depth

Ways agents connect

Type How it connects Typical use
SSH agent Controller SSHes to the machine and launches agent.jar Static Linux VMs
Inbound agent (formerly JNLP) Agent connects to the controller over TCP port 50000 or WebSocket via the HTTP port Agents behind NAT/firewalls, containers, Windows
Docker agent agent { docker { image 'node:20' } } runs steps inside a container on a Docker-capable agent Per-job toolchains on static hosts
Kubernetes plugin Creates a pod per build from a pod template; the jnlp container connects inbound Ephemeral, autoscaling agents on EKS
Cloud plugins (EC2, EC2 Fleet, Azure VM) Provision VMs on demand, terminate when idle Heavy builds needing full VMs (e.g. Docker-in-VM, large compiles)

WebSocket inbound connections avoid opening an extra TCP port through load balancers and ingresses — the preferred mode on Kubernetes.

Enlarge diagramcontrollerstatic VMcontrollerpod / VM behindNATSSH agentcontroller SSHes in, starts agent.jarInbound agentagent dials out: WebSocket on :443
Figure 1. The two ways an agent connects. Inbound over WebSocket is the usual choice for containers and Kubernetes.
Swipe sideways to see all of it, or tap to open it full screen.

Ephemeral agents with the Kubernetes plugin

The modern pattern: Jenkins controller on EKS, and every build gets a fresh pod.

pipeline {
  agent {
    kubernetes {
      yaml '''
apiVersion: v1
kind: Pod
spec:
  serviceAccountName: jenkins-build
  containers:
  - name: maven
    image: maven:3.9-eclipse-temurin-21
    command: ["sleep"]
    args: ["infinity"]
    resources:
      requests: { cpu: "1", memory: "2Gi" }
      limits:   { memory: "3Gi" }
  - name: kaniko
    image: gcr.io/kaniko-project/executor:debug
    command: ["sleep"]
    args: ["infinity"]
'''
    }
  }
  stages {
    stage('Build') { steps { container('maven') { sh 'mvn -B package' } } }
  }
}

Benefits: clean workspace every time (no "works on agent 3 only"), per-build tool images, autoscaling via Cluster Autoscaler/Karpenter, cost tied to usage, and isolation between teams (namespaces, service accounts with IRSA for AWS access).

Things to get right:

  • Resource requests on build containers, or pods get evicted / starve.
  • Image builds without Docker-in-Docker: Kaniko, BuildKit rootless, or Buildah — mounting the node's Docker/containerd socket gives builds root on the node.
  • Caching: ephemeral pods start cold. Use a dependency proxy (Nexus/Artifactory/CodeArtifact), a PVC or S3-backed cache, and registry-based Docker layer cache.
  • Pod startup latency: pre-pull images on nodes, keep images small, consider a warm node pool.
Enlarge diagramBUILD POD (SERVICEACCOUNT JENKINS-BUILD)controllerjnlpagent.jarmavenmvn packagekanikoimage buildshared workspace volumeall containers see the checked-out code
Figure 2. A pod per build: the jnlp container talks to the controller; tool containers share the workspace volume; the pod is deleted afterwards.
Swipe sideways to see all of it, or tap to open it full screen.

Sizing executors

For static agents, roughly one executor per CPU core is a starting point, lower for memory-heavy builds. Too many executors on one VM means builds fight for disk I/O and memory; too few means queueing. Ephemeral pod agents typically use one executor per pod.

"Waiting for next available executor"

Causes, in order of likelihood: no online agent has that label (typo in label, agent offline); all matching executors busy; with Kubernetes agents, pods can't be created or scheduled (check the cloud configuration, namespace quotas, kubectl get pods -n jenkins for Pending pods and their events, node capacity, image pull errors); the concurrency limit on the Kubernetes cloud config is reached.

Enlarge diagramLabeldoes any agent haveit? typo?Onlineare matching agentsconnected?Busyall executors in use?PodsPending: quota,scheduling, image pull1234
Figure 3. "Waiting for next available executor": check these in order.
Swipe sideways to see all of it, or tap to open it full screen.