Operating and troubleshooting TLS
The essential commands
# Full handshake view: protocol, cipher, chain, verification result
openssl s_client -connect shop.example.com:443 -servername shop.example.com -showcerts </dev/null
# Just the expiry dates
echo | openssl s_client -connect shop.example.com:443 -servername shop.example.com 2>/dev/null \
| openssl x509 -noout -dates -subject -issuer
# SANs on the served certificate
echo | openssl s_client -connect shop.example.com:443 -servername shop.example.com 2>/dev/null \
| openssl x509 -noout -ext subjectAltName
# Test protocol support
openssl s_client -connect shop.example.com:443 -tls1_1 </dev/null # should FAIL on a modern config
openssl s_client -connect shop.example.com:443 -tls1_3 </dev/null
# Verbose curl
curl -vI https://shop.example.com
curl --cacert internal-ca.pem https://api.internal:8443/health
# Verify a chain file offline
openssl verify -CAfile root.pem -untrusted intermediate.pem server.pem
External scanners (SSL Labs, testssl.sh) grade configurations and catch chain issues.
Common errors
| Error | Cause | Fix |
|---|---|---|
certificate has expired |
Renewal failed or wasn't automated | Renew/replace and reload now; then automate and alert |
unable to get local issuer certificate / unable to verify the first certificate |
Server doesn't send intermediates, or the client lacks the root (private CA, old image) | Serve fullchain; add the CA to the client trust store |
certificate verify failed: Hostname mismatch / SSL: no alternative certificate subject name matches |
Hostname not in SANs; wrong cert served (missing SNI → default cert) | Correct SANs; send SNI (-servername) |
self signed certificate in certificate chain |
Corporate TLS-intercepting proxy, or a private CA unknown to the client | Install the corporate/private CA in the trust store (image, JVM cacerts) |
wrong version number |
Speaking TLS to a plain HTTP port (or vice versa) | Correct port/protocol (https:// vs http://) |
handshake failure / no shared cipher / protocol version |
Client and server share no protocol or cipher (old client vs TLS1.2+ policy; FIPS settings) | Upgrade client, or adjust policy deliberately |
| Works in browser, fails in curl/Java | Browser fetched or cached the missing intermediate | Serve the full chain |
| Fails only in containers | Minimal/distroless image without ca-certificates, or old CA bundle |
Install/update CA bundle in the image |
| Intermittent cert errors | One of several backends/LB nodes has an old cert | Check each backend individually |
certificate is not yet valid |
Client/server clock skew | Fix NTP/chrony |
Preventing expiry outages
An expired certificate is a full outage that's 100% preventable:
- Automate issuance and renewal — ACM with DNS validation, cert-manager, Let's Encrypt clients.
- Inventory every certificate: ACM, imported certs, Kubernetes Secrets, certificates baked into appliances, client certificates used by apps, internal CAs.
- Monitor and alert at 30 days, escalate at 14 and 7: ACM
DaysToExpiryCloudWatch metric and EventBridge expiry events, cert-manager Prometheus metrics, blackbox exporter probes (probe_ssl_earliest_cert_expiry) against every public endpoint, synthetic checks. - Rotate without downtime — deploy the new certificate alongside (ALB listeners accept multiple certificates; ACM swaps in place), then reload processes gracefully (
nginx -s reload, rolling restarts). - Rehearse rotation of internal CAs and client certificates, which are often forgotten.
At 2 AM with an expired cert: replace/renew immediately (ACM re-issue, cert-manager cmctl renew, manual upload), reload, verify with openssl s_client from outside, then write the postmortem: why automation didn't renew (e.g. the DNS validation CNAME was deleted, ACME DNS permissions broke, imported cert with no owner) and add alerts.
A leaked private key
Treat it as an incident: revoke the certificate with the CA, generate a new key and certificate, deploy everywhere the old one was used, check CT logs for unexpected issuance, rotate any secrets that might have been exposed with it, and investigate how it leaked (keys in Git, in images, in shared drives). Prefer services where keys never leave a managed boundary (ACM, KMS, HSMs).
Certificate pinning
Mobile apps sometimes pin a certificate or public key to resist interception. Rotation then breaks old app versions unless you pin to a stable intermediate/root or a backup key, plan rotations with app releases, and keep pin sets updated. Many teams now prefer CT monitoring over pinning.
Performance
TLS cost today is small, but at scale: use TLS 1.3, ECDSA certificates (cheaper signatures), session resumption, keep-alive and connection pooling (avoid handshakes per request), HTTP/2, and offload to managed load balancers and CDNs. If an ALB/proxy shows high TLS negotiation counts, look for clients that don't reuse connections.