Interview questions and model answers
"Walk me through what happens when you run a command in the shell."
The shell parses the line, expands variables and globs, and looks up the command — alias, builtin, function, then a search through $PATH. For an external program it forks a child, the child execs the binary (the kernel loads it, maps libraries via the dynamic linker), inherits the shell's environment and FDs 0/1/2 (rewired by any redirections), and runs. The parent shell waits and collects the exit status into $?. If I add &, the shell doesn't wait. If the program is killed by a signal, the status is 128 + the signal number.
"A disk is 100% full on a production server. What do you do?"
First, protect the service: find what's consuming space quickly — df -h to confirm which filesystem, du -xh --max-depth=1 | sort -rh to drill down. If du doesn't add up, check lsof +L1 for deleted-but-open files and truncate or restart that process. Also check df -i in case it's inodes. Free space safely — truncate (not delete) active logs, clean old rotated logs, docker system prune / crictl rmi --prune on container hosts, journald vacuum. If needed, grow the EBS volume online with growpart and xfs_growfs/resize2fs. Then prevent recurrence: logrotate or log shipping with retention, disk alerts at 80%, and fix whatever produced the growth.
"What's the difference between a process and a thread? Why does it matter for containers?"
A process has its own address space, FDs and PID; threads share their process's memory and FDs and are scheduled independently. In containers it matters because limits apply per cgroup: a JVM with 200 threads all count toward the same CPU quota, so a CPU limit of 1 can throttle a multi-threaded app heavily during GC. PID limits also count threads.
"Explain load average. Is a load of 8 bad?"
It depends on the CPU count and what the load is. Load is the average number of tasks running, waiting for CPU, or in uninterruptible I/O sleep over 1/5/15 minutes. Load 8 on 16 vCPUs is fine; on 2 vCPUs it's saturation. I'd look at the CPU breakdown — high us/sy means CPU-bound, high wa with idle CPU means I/O-bound, high st means the hypervisor or CPU credits.
"What is a zombie process and how do you get rid of it?"
A zombie has exited but its parent hasn't read its exit status with wait(). It holds only a PID slot. Killing it does nothing. I fix the parent — signal it, restart it, or if it dies the zombie is adopted by PID 1 and reaped. In containers I make sure PID 1 reaps children by using --init/tini.
"How does the OOM killer decide what to kill, and how would you investigate an OOM?"
Each process has an oom_score largely proportional to memory use, adjusted by oom_score_adj; the highest score is killed with SIGKILL. In containers the cgroup limit triggers it for that cgroup. I confirm with dmesg/journalctl -k, check memory graphs to tell a leak from a spike, compare usage to limits/requests, and for JVMs check heap vs container memory. Fix with right-sized limits, fixing leaks, and alerting on memory approaching limits.
"SSH to a server hangs. How do you debug?"
ssh -vvv to see where it stops. If it hangs before "Connection established", it's network: security group / NACL / route / host firewall — test nc -zv host 22 and check the SG. If it connects but auth fails, it's keys, permissions on ~/.ssh, user, or sshd_config. If it authenticates then hangs, it may be a slow login script, DNS reverse lookup (UseDNS no), or a full disk. On AWS I'd fall back to SSM Session Manager or the serial console.
"How do you find which process is listening on port 8080, and why can't another host reach it?"
ss -tlnp | grep 8080 (or lsof -i :8080). If it's bound to 127.0.0.1 it's local-only and needs 0.0.0.0. Then I'd check host firewall rules, the security group and NACLs, and test with nc -zv from the client side and tcpdump on the server to see whether SYNs arrive.
"What does set -euo pipefail do?"
-e exits on errors, -u treats unset variables as errors, pipefail makes a pipeline fail if any stage fails. Together they stop scripts from continuing silently after failures — critical in CI and deploy scripts. I pair it with trap for cleanup and explicit handling where failure is acceptable (cmd || true).
"How do containers isolate processes on Linux?"
Namespaces give each container its own view — PID tree, network stack, mounts, hostname, users. cgroups limit and account for CPU, memory, I/O and PIDs. On top of that, capabilities are dropped, seccomp filters syscalls, and AppArmor/SELinux confine access. It's still the same kernel, which is why container escapes are possible and why we avoid --privileged and run as non-root.