DevOps Interview Prep — Sikander Kumbhar

70+

Questions

Topics

Code Labs

🚀

Interview Ready

⚙️

Coderbyte Assessment Tasks

Python S3 · Terraform ECS · Docker · SQL

6 tasks

▶

TASK 1 Python S3 — Read contents of a public S3 bucket file with a __cb__ prefix

Python boto3 AWS S3

import boto3
from botocore.config import Config
from botocore import UNSIGNED

# Use UNSIGNED for public bucket — no credentials needed
s3 = boto3.client('s3', config=Config(signature_version=UNSIGNED))

response = s3.list_objects_v2(
    Bucket='coderbytechallengesandbox',
    Prefix='__cb__'
)

file_key = response['Contents'][0]['Key']
obj = s3.get_object(Bucket='coderbytechallengesandbox', Key=file_key)
contents = obj['Body'].read().decode('utf-8')
print(contents)
python

💡 Key Points

UNSIGNED — skip credentials for public buckets
list_objects_v2() — list files by prefix
get_object()['Body'].read().decode('utf-8') — read file contents

TASK 2 Terraform — AWS VPC + ECS Cluster + terraform init && terraform graph

Terraform AWS

main.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
  shared_config_files      = ["$HOME/.aws/config"]
  shared_credentials_files = ["$HOME/.aws/credentials"]

  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true
  access_key = "mock"
  secret_key = "mock"
}

resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
  tags = { Name = "aws-vpc" }
}

resource "aws_ecs_cluster" "main" {
  name = "aws-ecs-cluster"
}
hcl

main.sh

#!/bin/bash
terraform init
terraform graph
bash

💡 Trick: Mock credentials

Use skip_credentials_validation = true + access_key = "mock" in challenge environments so terraform init doesn't fail.

TASK 3 Docker — Install Node.js app (node:18-alpine, yarn install --production)

Docker Node.js

#!/bin/bash

cat > Dockerfile << 'EOF'
FROM node:18-alpine
WORKDIR /app
RUN yarn install --production
CMD ["node", "src/index.js"]
EXPOSE 3000
EOF

cat Dockerfile
bash

TASK 4 Docker — Custom Nginx with HTML files and custom config

Docker Nginx

#!/bin/bash

cat > Dockerfile << 'EOF'
FROM nginx:latest
EXPOSE 80
COPY index.html /usr/share/nginx/html/
COPY error.html /usr/share/nginx/html/
COPY nginx.conf /etc/nginx/conf.d/default.conf
ENV NGINX_ENV=custom
EOF

cat Dockerfile
bash

TASK 5 SQL — Month-to-month change in user signups using LAG()

SQL Window Functions

Primary Solution:

SELECT 
    Month,
    (cnt - prev_cnt) AS MonthToMonthChange
FROM (
    SELECT 
        DATE_FORMAT(DateJoined, '%M') AS Month,
        COUNT(*) AS cnt,
        LAG(COUNT(*)) OVER (ORDER BY MIN(DateJoined)) AS prev_cnt
    FROM maintable_FWCAX
    GROUP BY DATE_FORMAT(DateJoined, '%Y-%m')
) AS t
WHERE prev_cnt IS NOT NULL;
sql

Alternative (CTE):

WITH monthly_counts AS (
    SELECT 
        DATE_FORMAT(DateJoined, '%Y-%m') AS year_month,
        DATE_FORMAT(DateJoined, '%M')    AS Month,
        COUNT(*) AS cnt
    FROM maintable_FWCAX
    GROUP BY 1, 2
)
SELECT 
    Month,
    cnt - LAG(cnt) OVER (ORDER BY year_month) AS MonthToMonthChange
FROM monthly_counts
WHERE LAG(cnt) OVER (ORDER BY year_month) IS NOT NULL;
sql

💡 SQL Tips

LAG() — compare with previous row
DATE_FORMAT(date, '%M') — month name (January, February…)
DATE_FORMAT(date, '%Y-%m') — for chronological ordering
Filter WHERE prev_cnt IS NOT NULL to skip first row

SCENARIO Debugging Next.js 500 errors in EKS/ECS production deployments

Check logs: kubectl logs <pod> --previous or ECS task logs
Env vars: NEXT_PUBLIC_ for client-side, standard for server-side
Node version: Match Dockerfile Node version to local dev
Standalone mode: Add output: 'standalone' in next.config.js
Port binding: Container must expose port 3000 and ALB health check must match

📋

Ansible

Playbooks · Handlers · Vault · Idempotency

9 Q&A

▶

⭐ Core Concept: Idempotency

Running the same playbook multiple times results in the same system state — no errors, no duplicates. Ansible modules check current state before acting.

Q1What is the purpose of handlers in Ansible?

Answer: Handlers run tasks that are triggered only when notified by other tasks. They execute at the end of a play and only if the notifying task reported a changed state.

📌 Classic Example

Update an Nginx config → notify: restart nginx → handler restarts Nginx only if the config actually changed.

Q2Which Ansible module manages packages using apt?

Answer: The apt module — designed specifically for Debian/Ubuntu package management.

- name: Install nginx
  apt:
    name: nginx
    state: present

Q3What are Ansible facts?

Answer: Facts are system information automatically gathered from remote hosts via the setup module. They include OS details, IP addresses, memory, CPU, etc.

Access them as: ansible_facts['os_family'] or ansible_os_family

Q4How does Ansible determine task execution order?

Answer: Tasks execute sequentially, top to bottom as listed in the playbook. There is no parallel execution within a single play unless using async or strategy: free.

Q5What is the default inventory file name?

Answer: hosts (located at /etc/ansible/hosts) or a custom file specified with -i inventory.ini.

Q6Which directive includes variables from a file?

Answer: include_vars loads variables from an external file at runtime.

- name: Load vars
  include_vars: vars/secrets.yml

Q7Default roles directory?

Answer: /etc/ansible/roles or ./roles (relative to the playbook). Defined in ansible.cfg via roles_path.

Q8Purpose of the when statement?

Answer: Conditional execution — a task only runs when the condition evaluates to true.

- name: Restart on RedHat only
  service:
    name: httpd
    state: restarted
  when: ansible_os_family == "RedHat"

Q9What is Ansible Vault and how do you encrypt a file?

Answer: Vault encrypts sensitive data (passwords, keys) at rest. Encrypted files can be safely committed to version control.

ansible-vault encrypt secrets.yml        # Encrypt
ansible-vault decrypt secrets.yml        # Decrypt
ansible-vault view secrets.yml           # View encrypted
ansible-playbook site.yml --ask-vault-pass  # Run playbook

☁️

AWS Cloud & CloudWatch

EC2 · S3 · RDS · Aurora · ALB · CloudWatch · Secrets Manager

14 Q&A

▶

Service	Use Case
DynamoDB	Session state, high-availability NoSQL
ElastiCache	Session state, low-latency caching
Aurora	HA relational DB, up to 15 read replicas, auto-scaling storage
EBS io1/io2	High IOPS storage (up to 64,000 IOPS)
Secrets Manager	Secure password/key storage, auto-rotation
S3 Versioning	Protect against accidental deletes

Q10EC2 is running but cannot SSH — what do you check?

Security Group: Port 22 open to your public IP
Route Table: Subnet has route 0.0.0.0/0 → Internet Gateway (IGW)
Instance Status Checks: Check AWS Console (failed = OS frozen)
Fallback: Use SSM Session Manager (no SSH needed)

Q11ALB returning 502 Bad Gateway — most common cause?

Answer: Keep-Alive timeout mismatch — backend closes TCP connection before ALB expects it.

⚠️ Fix

Backend Keep-Alive timeout (e.g., 65s) must be higher than ALB idle timeout (default 60s). Set Nginx keepalive_timeout 65;

Q12How do you securely pass DB password to EC2/ECS?

❌ Never do

Hardcode passwords in code, Dockerfiles, or environment variable configs committed to Git.

✅ Correct approach

Store in AWS Secrets Manager
Grant IAM Role (EC2/ECS Task Role) permission to read the secret
Application fetches secret at runtime using SDK

Q13What is the maximum CloudWatch Logs retention period?

Answer: Indefinitely — with manual configuration (no expiry). Default is "Never Expire" unless you set a retention policy (1 day to 10 years).

Q14How do you create a custom CloudWatch metric?

Answer: Publish data points using the AWS CLI or SDK.

aws cloudwatch put-metric-data \
  --namespace "MyApp" \
  --metric-name "RequestLatency" \
  --value 150 \
  --unit Milliseconds

Q15CloudWatch detailed monitoring — how frequent?

Answer: 1-minute intervals (vs. default 5-minute basic monitoring). Costs extra per metric.

Q16Which service to use for long-term CloudWatch Log storage?

Answer: Export to Amazon S3 for cost-effective long-term storage. Use CloudWatch Logs subscriptions → Kinesis Firehose → S3.

Q17What is CloudWatch Logs Insights?

Answer: A query service to query and analyze log data in real-time using a SQL-like syntax. Much faster than grep for large log volumes.

fields @timestamp, @message
| filter @message like /ERROR/
| sort @timestamp desc
| limit 20

Q18CloudWatch alarm — "Datapoints to Alarm" parameter?

Answer: Defines how many periods the condition must be met before triggering the alarm. Prevents false alarms from single spikes. (e.g., "3 out of 5 periods" = more reliable)

Q19Which TWO services store session state for stateless web servers?

Answer: DynamoDB (persistent, scalable NoSQL) and ElastiCache (Redis/Memcached for fast in-memory access).

Q20Protect S3 documents from accidental deletion?

Answer: Enable S3 Versioning. Deleted objects become "delete markers" — previous versions are preserved and can be restored.

Q21DB needs 8TB, grows 8GB/day, 8+ read replicas needed?

Answer: Amazon Aurora — auto-scaling storage (up to 128TB), up to 15 read replicas, multi-AZ, and 5x faster than standard MySQL.

Q22EBS volume for up to 16,000 IOPS?

Answer: EBS Provisioned IOPS SSD (io1/io2) — up to 64,000 IOPS per volume. General Purpose (gp2/gp3) tops out at ~16,000 IOPS.

Q23Recommended practice for reducing CloudWatch costs?

Answer: Use metric filters to extract data from logs instead of sending all raw logs, and aggregate logs before forwarding. Also: export old logs to S3, delete unnecessary log groups.

⚡

GitHub Actions

Workflows · Matrix · Artifacts · Caching · Secrets

10 Q&A

▶

Q24How do you create a reusable workflow?

Answer: Define a workflow with the workflow_call event trigger in .github/workflows/.

on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string

Q25How do you conditionally run a step?

Answer: Use the if conditional in the step definition.

- name: Deploy to prod
  if: github.ref == 'refs/heads/main'
  run: ./deploy.sh

Q26What does strategy.fail-fast do in a matrix job?

Answer: When set to true (default), it cancels all running matrix jobs as soon as any one job fails. Set to false to let all matrix jobs complete regardless.

Q27How do you cache dependencies?

Answer: Use actions/cache with a cache key.

- uses: actions/cache@v3
  with:
    path: ~/.npm
    key: ${{ runner.os }}-node-${{ hashFiles('package-lock.json') }}

Q28What does the needs keyword do?

Answer: Defines job dependencies — a job won't start until all jobs listed in needs have completed successfully.

deploy:
  needs: [build, test]
  runs-on: ubuntu-latest

Q29Purpose of jobs.<id>.outputs?

Answer: Allows one job to pass data to another job downstream.

jobs:
  build:
    outputs:
      image-tag: ${{ steps.tag.outputs.tag }}
  deploy:
    needs: build
    env:
      TAG: ${{ needs.build.outputs.image-tag }}

Q30How do you define environment variables for a job?

Answer: Use the env keyword at job or step level.

jobs:
  build:
    env:
      NODE_ENV: production
      API_URL: https://api.example.com

Q31How do you upload an artifact?

Answer: Use actions/upload-artifact.

- uses: actions/upload-artifact@v3
  with:
    name: build-output
    path: dist/

Q32What does jobs.<id>.container do?

Answer: Runs the job inside a Docker container instead of the host runner.

jobs:
  test:
    container:
      image: node:18-alpine
      env:
        NODE_ENV: test

Q33How do you persist data between jobs?

Answer: Use actions/upload-artifact to save, then actions/download-artifact in the next job to retrieve. Jobs run on isolated runners, so filesystem state is NOT shared.

🐧

Linux / Unix

Commands · Permissions · Troubleshooting · File System

8 Q&A

▶

📌 Permission Cheatsheet

r=4, w=2, x=1 | Owner/Group/Others | 640 = rw-/r--/--- | 755 = rwx/r-x/r-x

Q34Find files in a directory based on name or size?

Answer: find

find /var/log -name "*.log"          # by name
find /home -type f -size +100M      # files > 100MB
find /tmp -mtime +7 -delete         # files older than 7 days

Q35Give owner rw, group r, others no access?

Answer: chmod 640 filename

Owner: r+w = 4+2 = 6
Group: r   = 4
Others:    = 0
Result: 640

Q36Wildcard to match any single character in a filename?

Answer: ? (question mark)
* = match any number of chars | ? = match exactly one char

Example: ls file?.txt matches file1.txt, fileA.txt but not file10.txt

Q37Primary purpose of /etc directory?

Answer: Store system configuration files. Examples: /etc/nginx/nginx.conf, /etc/hosts, /etc/passwd, /etc/fstab.

Q38Display dynamic real-time view of running processes?

Answer: top (or htop for a better UI). Shows CPU%, MEM%, PID, process name, load averages.

Q39Detailed info about file permissions, ownership, size?

Answer: ls -l or ls -la (includes hidden files)

-rw-r--r-- 1 sikander staff 1024 Sep 9 10:00 file.txt
 ↑ permissions  ↑ owner  ↑ size ↑ date

Q40Securely copy a local file to a remote server?

Answer: scp (Secure Copy over SSH)

scp localfile.txt user@remotehost:/path/to/destination/
scp -r ./folder/ user@remotehost:/opt/app/   # recursive
scp -i ~/.ssh/key.pem file.txt ec2-user@1.2.3.4:/home/ec2-user/

SCENARIOProduction Linux server is extremely slow — debug steps?

top or htop — find high CPU/memory PID
df -h — check disk is not 100% full
dmesg -T | grep -i oom — check OOM killer logs
ss -tulpn — check unexpected network connections
iostat -x 1 — check disk I/O bottleneck
free -h — check available memory/swap

🏗️

Terraform (IaC)

State Management · Multi-Env · Drift · Modules

3 scenarios

▶

S1How do you manage Terraform state in a team environment?

Answer: Use a remote backend with state locking.

✅ Best Practice

S3 — stores state file (encrypted, versioned)
DynamoDB — provides state locking (prevents concurrent applies)

terraform {
  backend "s3" {
    bucket         = "my-terraform-state"
    key            = "prod/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-locks"
    encrypt        = true
  }
}

S2terraform plan shows unexpected destroy/change — what happened?

Answer: State drift — someone made manual changes in the AWS Console.

⚠️ Resolution Options

terraform apply — revert AWS back to code-defined state
Update Terraform code to match manual changes, then apply
Use terraform import to bring manual resources under IaC control

S3How do you structure Terraform for Dev/Staging/Prod?

Answer: Use a directory-based structure with version-pinned modules.

environments/
├── dev/
│   ├── main.tf
│   └── terraform.tfvars
├── staging/
│   ├── main.tf
│   └── terraform.tfvars
└── prod/
    ├── main.tf
    └── terraform.tfvars
modules/
└── vpc/
    └── main.tf    # version-pinned: ref=v1.2.0

💡 Why not Workspaces?

Workspaces share state backend config — a mistake in one workspace can affect others. Directories are safer for production isolation.

🐳

Docker Best Practices

Dockerfiles · Multi-stage · Optimization

Quick Ref

▶

Practice	Why
Pin image versions (`node:18-alpine`)	Reproducible builds, no surprise updates
Use `alpine` base images	Minimal attack surface, smaller size
Use heredoc `<< 'EOF'`	Multi-line file creation in scripts
`output: 'standalone'` in Next.js	Smaller Docker image, all deps bundled
Non-root user	Security — never run as root in containers
`.dockerignore`	Exclude node_modules, .git, etc. from build context
Multi-stage builds	Separate build environment from runtime image

📌 Always print at end

In Coderbyte challenges, always end scripts with cat Dockerfile to display the file contents.

🗄️

SQL Quick Reference

Window Functions · Aggregations · Date Functions

Cheatsheet

▶

Function	Purpose	Example
`LAG(col)`	Previous row value	Month-over-month change
`LEAD(col)`	Next row value	Future projections
`ROW_NUMBER()`	Unique row rank	Pagination, deduplication
`RANK()`	Rank with gaps on ties	Leaderboards
`DATE_FORMAT(d, '%M')`	Month name	January, February…
`DATE_FORMAT(d, '%Y-%m')`	Year-month for ordering	2026-01, 2026-02…
`COUNT(*)`	Count all rows	Total signups per month

🎭

Behavioral & Professional Q&A

STAR Method · Goals · Team Scenarios

7 answers

▶

Situation

Set the context and background

Task

What was your responsibility?

Action

What specific steps did YOU take?

Result

Quantify the outcome (%, time, $)

B1Example of effective teamwork you initiated?

"Initiated a cross-team collaboration to implement a GitOps workflow using ArgoCD. Organized daily standups between Dev, Ops, and QA to align on CI/CD pipeline improvements. Result: Reduced deployment time by 60% and achieved zero-downtime releases."

B2Helped a colleague outside your responsibility?

"A developer struggled with Docker container networking in their local environment. Though not my responsibility, I spent 2 hours debugging and created a comprehensive dev environment setup guide. Result: Helped the entire team reduce onboarding time by 40%."

B3Resolved a conflict within your team?

"Two team members disagreed on IaC tool selection (Terraform vs Pulumi). I facilitated a proof-of-concept evaluation where both presented their cases. We agreed on Terraform for consistency, with Pulumi for specific use cases. Result: Both felt heard and we moved forward unified."

B4Short-term professional goals?

"Deepen expertise in cloud-native technologies — Kubernetes and AWS. Plan: earn AWS Solutions Architect certification, contribute to open-source DevOps projects, and implement advanced CI/CD pipelines with GitOps practices in production environments."

B5What inspires you to learn something new?

"Solving real-world problems. When I encounter a deployment bottleneck or infrastructure limitation, I'm driven to research new tools that can solve it efficiently. The DevOps field evolves rapidly — staying curious keeps me relevant and effective."

B6Professional achievement you're proud of?

"Automated a manual deployment process from 4 hours down to 10 minutes using Jenkins pipelines and Docker. Reduced human errors, improved team productivity, and gave developers more time to focus on features rather than deployments."

B7New skill you're interested in learning?

"GitOps practices with ArgoCD and Flux, and deepening knowledge of service mesh technologies like Istio for microservices architecture."

🧩

Logic & Aptitude Puzzles

Reasoning · Mathematical · Sequence

3 puzzles

▶

P17 people at round table, each says "I sit between two Tricksters". How many Tricksters?

Answer: 4 Tricksters

💡 Logic

If a Truth Teller said it, they'd actually be between two Tricksters. If a Trickster said it, they're lying — so they're NOT between two Tricksters. Arrangement: T-L-L-T-L-L-T... (T=Truth, L=Liar) with 4 Tricksters (Liars) and 3 Truth Tellers.

P2Tom+Jon=35, Jon+Paul=36, Paul+Tom=37. How old is the oldest?

Answer: Paul is 19 (oldest), Tom is 18, Jon is 17

💡 Math

Sum all three: 2(T+J+P) = 108, so T+J+P = 54. Paul = 54-35 = 19. Tom = 54-36 = 18. Jon = 54-37 = 17.

P3Sequence: _ _ D _ A _ with rules for B, C, E, F?

Answer: B-F-D-C-A-E

💡 Rules Applied

E must be last ✓ → position 6
A is at position 5 (given)
D is at position 3 (given)
C must be before A but not before D → position 4
B can be before D or A → position 1
F fills remaining → position 2

🔥

Linux Tricky & Debugging Questions

Real scenarios from public interviews · SRE/DevOps focused

37 Q&A

▶

📌 Source Note

These questions are compiled from publicly shared DevOps/SRE interview experiences on platforms like Reddit r/devops, r/sysadmin, Glassdoor, Blind, GitHub Discussions, and engineering blogs from Netflix, Cloudflare, HashiCorp, and ByteByteGo.

🧠 Process & Memory

L1A process is stuck in D state (uninterruptible sleep). How do you diagnose and resolve it? (Seen at Google, Cloudflare SRE interviews)

⚠️ Tricky Part

You CANNOT kill a D-state process with kill -9 — it ignores all signals while waiting for I/O.

Diagnose:

ps aux | grep " D "               # find D-state processes
cat /proc/<PID>/status | grep State
cat /proc/<PID>/wchan              # what kernel function it's waiting on
dmesg -T | tail -30                # check for NFS/disk errors
iostat -x 1 5                      # check disk I/O saturation

Common Causes & Fixes:

NFS mount hung → umount -f -l /mnt/nfs (force + lazy)
Disk I/O frozen → check dmesg for hardware errors, may need reboot
SCSI/disk error → smartctl -a /dev/sda to check disk health

💡 Interview Answer

"D-state means the process is waiting for a kernel I/O operation to complete. Since it ignores signals, the only real fix is to resolve the underlying resource (unmount hung NFS, fix disk) — or reboot as last resort."

L2Your Linux server has plenty of free RAM but applications are running out of memory and getting OOM-killed. Why? (Netflix SRE interview)

Answer: Memory fragmentation or NUMA imbalance — not actual shortage.

free -h                            # shows "free" but misleading
cat /proc/meminfo | grep -E "MemFree|Cached|Buffers|Slab|HugePages"
numactl --hardware                 # check NUMA node distribution
cat /proc/buddyinfo                # check memory fragmentation
echo 1 > /proc/sys/vm/drop_caches # drop page cache (test only!)

💡 Key Insight

"Free" RAM in Linux includes page cache. Actual usable = MemFree + Buffers + Cached. Also check: huge pages exhausted, cgroup memory limits, or NUMA node imbalance where app is bound to a node with less free mem.

L3How do you find which process has a file open that prevents unmounting a filesystem? (Very common SRE question)

lsof /mount/path                   # list open files on that mount
fuser -m /mount/path               # show PIDs using that mount
fuser -km /mount/path              # KILL those PIDs (careful!)
lsof +D /mount/path                # recursive open files

💡 Tip

Sometimes a deleted file is still held open. Check: lsof | grep deleted — these hold space until the process closes the file descriptor.

L4A cron job runs fine manually but fails when run via cron. What are the common causes? (Asked at HashiCorp, Datadog)

Classic tricky question — cron has a minimal, non-interactive environment.

PATH is different: Cron's PATH is only /usr/bin:/bin. Always use absolute paths in cron scripts.
No shell sourcing: .bashrc, .profile are NOT loaded. Source manually: source /etc/profile
Env vars missing: Variables set in .bashrc don't exist in cron context
Working directory: Cron starts in user's home ($HOME), not the script directory
Output not captured: Add 2>&1 | tee /tmp/cron.log to see errors

# In crontab — always use full paths
* * * * * /usr/bin/python3 /opt/scripts/myscript.py >> /var/log/cron.log 2>&1
# Or source environment
* * * * * source /etc/profile && /opt/script.sh >> /tmp/out.log 2>&1

🌐 Networking

L5How do you check if a port is open without using netstat, nmap, or curl? (Cloudflare interview)

💡 Tricky — tests knowledge of built-in tools

# Method 1: /dev/tcp (bash built-in)
timeout 3 bash -c "echo >/dev/tcp/hostname/443" && echo "open" || echo "closed"

# Method 2: ss (modern replacement for netstat)
ss -tulpn | grep :80

# Method 3: nc (netcat)
nc -zv hostname 443 2>&1

# Method 4: Python one-liner
python3 -c "import socket; s=socket.socket(); s.settimeout(2); print(s.connect_ex(('host',443)))"

L6You run ping google.com successfully but curl https://google.com fails with "connection refused". What do you debug? (Common in interviews)

ping uses ICMP — proves DNS works and host is reachable at layer 3
Check port 443 specifically: telnet google.com 443 or /dev/tcp
Check proxy/firewall: env | grep -i proxy — maybe HTTP_PROXY set
Check SSL: curl -v https://google.com 2>&1 | head -40 — see TLS handshake
Check local firewall: iptables -L -n or ufw status
Try without SSL: curl http://google.com

L7What does TIME_WAIT mean in ss/netstat output, and how can too many cause issues? (Asked at Stripe, Pinterest SRE)

Answer: TIME_WAIT is a TCP state — after connection closes, the kernel waits 2×MSL (~60–120 seconds) before releasing the port to prevent stale packets from old connections.

⚠️ Problem

Thousands of TIME_WAIT sockets exhaust ephemeral ports (32768–60999), causing new connection failures: "Cannot assign requested address".

ss -s                              # summary including TIME_WAIT count
cat /proc/sys/net/ipv4/ip_local_port_range   # check port range
# Fix: enable tcp_tw_reuse (safe) — NOT tcp_tw_recycle (dangerous)
echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse
# Permanent: add to /etc/sysctl.conf
net.ipv4.tcp_tw_reuse = 1

L8Explain what happens step by step when you run curl https://sikanderkumbhar.com (Full stack networking question — asked at FAANG)

DNS resolution: Check /etc/hosts → systemd-resolved → recursive DNS → authoritative NS
TCP 3-way handshake: SYN → SYN-ACK → ACK to port 443
TLS handshake: Client Hello → Server Hello → cert exchange → session key
HTTP/2 request: GET / with Host header
Server processes: Nginx/proxy → backend app
Response: 200 OK with HTML body
TCP FIN: Connection teardown → TIME_WAIT

💾 Filesystem & Disk

L9df -h shows disk is 100% full, but du -sh /* doesn't account for the usage. Why? (Very tricky — seen at multiple companies)

Answer: Deleted files still held open by running processes.

💡 Explanation

When a process opens a file and it gets deleted, the inode and blocks are NOT freed until the process closes its file descriptor. du only sees existing files. df sees actual block usage.

lsof | grep deleted | awk '{print $7, $9}' | sort -rh | head -20
# Then either restart the process holding the file
# or: kill -HUP <PID> to force log rotation

⚠️ Also Check

Inode exhaustion: df -i — disk can be "full" in inodes even with free space.

L10How do you find the top 10 largest files recursively in a directory without using any GUI tools?

# Method 1: find + sort
find /var -type f -printf '%s %p\n' 2>/dev/null | sort -rn | head -10

# Method 2: du
du -ah /var 2>/dev/null | sort -rh | head -10

# Method 3: find with ls
find /var -type f -exec ls -s {} \; 2>/dev/null | sort -rn | head -10

# Find files larger than 1GB
find / -type f -size +1G -ls 2>/dev/null

L11What is the difference between a hard link and a soft (symbolic) link? When would you use each?

	Hard Link	Soft Link
Points to	Inode directly	File path (filename)
Works across filesystems?	❌ No	✅ Yes
If original deleted?	✅ Data survives	❌ Broken link
Can link directories?	❌ No (usually)	✅ Yes
Check with	`ls -li` (same inode)	`ls -la` (shows arrow)

ln file.txt hardlink.txt           # hard link (same inode)
ln -s /path/to/file symlink.txt    # soft link

🔒 Permissions & Security

L12What is the sticky bit, setuid, and setgid? Give a real-world example of each. (Asked at LinkedIn SRE)

Bit	On Files	On Directories	Example
`setuid (4xxx)`	Run as file OWNER	-	`/usr/bin/passwd` runs as root
`setgid (2xxx)`	Run as file GROUP	New files inherit dir group	Shared project dirs
`sticky (1xxx)`	-	Only owner can delete their files	`/tmp` directory

ls -la /usr/bin/passwd             # shows -rwsr-xr-x (s = setuid)
ls -la /tmp                        # shows drwxrwxrwt (t = sticky)
chmod 4755 script.sh               # setuid
chmod 1777 /shared                 # sticky

L13How do you find all SUID/SGID files on a system? Why is this a security concern?

# Find all SUID files (potential privilege escalation)
find / -type f -perm -4000 -ls 2>/dev/null

# Find all SGID files
find / -type f -perm -2000 -ls 2>/dev/null

# Find both SUID and SGID
find / -type f \( -perm -4000 -o -perm -2000 \) -ls 2>/dev/null

🔒 Security Risk

SUID files run with root privileges. An attacker who exploits a vulnerability in a SUID binary gets root access. This is the basis of many Linux privilege escalation exploits.

⚡ Performance & Advanced Debugging

L14CPU is at 100% but top shows no single process taking more than 5%. What could be the issue? (Asked at Meta, Twitter SRE)

Answer: Many small processes collectively consuming CPU — or kernel/IRQ overhead.

top -H                             # show individual threads
sar -u 1 5                         # show user/sys/iowait/irq breakdown
mpstat -P ALL 1                    # per-CPU breakdown
cat /proc/interrupts               # check IRQ counts
perf top                           # live kernel/userspace profiling
strace -p <PID> -c                 # syscall profile of a process

💡 Key

Check %si (software interrupts) and %hi (hardware interrupts) in top. High %sy (kernel) often means context switching or system calls are the bottleneck.

L15How do you trace which system calls a misbehaving process is making? (Senior Linux interview essential)

# strace: trace system calls
strace -p 1234                     # attach to running process
strace -p 1234 -e trace=network    # only network syscalls
strace -p 1234 -o /tmp/trace.log   # output to file
strace -p 1234 -c                  # summary/count syscalls

# ltrace: trace library calls
ltrace -p 1234

# perf: performance counters
perf record -p 1234 sleep 10
perf report

# /proc filesystem
cat /proc/1234/fd/                 # open file descriptors
cat /proc/1234/net/tcp             # network connections

L16What is load average and what does it mean when load average is higher than number of CPU cores?

Answer: Load average = number of processes in runnable (R state) OR uninterruptible wait (D state) at a given time, averaged over 1, 5, and 15 minutes.

uptime
# output: load average: 2.50, 1.80, 1.20
#                       ↑1min  ↑5min ↑15min

nproc                              # number of CPU cores
# If load > nproc → system is overloaded (queue building up)

💡 Key Rule

Load average of 1.0 per core = 100% utilization. If you have 4 cores and load is 4.0 → fully loaded. If load is 8.0 → 2x overloaded, things are queuing. High load from D-state processes = I/O bottleneck.

L17How do you check and analyze kernel messages for hardware errors in production? (SRE interview — AWS, GCP)

dmesg -T                           # kernel ring buffer with timestamps
dmesg -T | grep -iE "error|fail|warn|oom|killed|segfault"
journalctl -k                      # kernel messages via journald
journalctl -k --since "1 hour ago"
journalctl -p err -b               # errors since last boot
# Hardware errors
mcelog                             # machine check exceptions (CPU errors)
cat /sys/class/net/eth0/statistics/rx_errors  # NIC errors

🧩 Tricky Command Outputs & Edge Cases

L18What is the output of echo $? after running a command in a pipeline like false | true? (Shell scripting gotcha)

Answer: 0 (success) — because $? captures the exit code of the last command in the pipeline (true), not the overall pipeline.

⚠️ Use PIPESTATUS to catch pipeline failures

false | true
echo $?           # 0  ← misleading!
echo ${PIPESTATUS[@]}   # "1 0" ← shows each command's exit code

# In bash, use pipefail option
set -o pipefail
false | true
echo $?           # 1  ← now correctly fails

L19What does 2>&1 vs &>file vs 2>file 1>file do? What's the difference? (Shell redirect tricky question)

cmd > file 2>&1       # stdout to file, then stderr to where stdout is (file)
cmd &> file           # shorthand for above (bash only)
cmd 2>err.log 1>out.log  # stdout and stderr to SEPARATE files

# WRONG order (common mistake):
cmd 2>&1 > file       # stderr goes to terminal (old stdout), stdout to file!

⚠️ Order matters!

2>&1 > file does NOT send stderr to file. The redirect is evaluated left-to-right: stderr is redirected to stdout (terminal) FIRST, then stdout is redirected to file.

L20How do you run a command that persists after you close your SSH session? What are the differences between the methods?

Method	Persists?	Output	Best For
`nohup cmd &`	✅	nohup.out	Quick one-off
`screen`	✅	Interactive TTY	Interactive sessions
`tmux`	✅	Interactive TTY	Modern, recommended
`disown %1`	✅	Terminal	Already running jobs
systemd service	✅ + restart	journald	Production daemons

nohup ./long-script.sh &> /tmp/out.log &
tmux new -s mysession
# Detach: Ctrl+B, D   |   Reattach: tmux attach -t mysession

L21What is the difference between kill, kill -9, and kill -15? When should you NOT use kill -9?

Signal	Number	Name	Behavior
`kill <PID>`	15	SIGTERM	Graceful shutdown — process can clean up
`kill -9 <PID>`	9	SIGKILL	Immediate kill — OS level, no cleanup
`kill -1 <PID>`	1	SIGHUP	Reload config (for daemons)
`kill -2 <PID>`	2	SIGINT	Ctrl+C equivalent

⚠️ Don't use kill -9 when

Process has open DB transactions, is writing files (data corruption risk), or holds locks. Always try SIGTERM first and give it 5–10 seconds.

L22You see a file named -rf in your directory. How do you delete it without accidentally running rm -rf?

Answer: Use -- to signal end of options, or use ./ prefix.

rm -- -rf           # -- tells rm: no more flags after this
rm ./-rf            # ./ makes it a path, not a flag
unlink ./-rf        # another safe way

💡 Same pattern for other dangerous filenames

ls -- -la, cat -- -file. Always use ./filename or -- for files starting with -.

L23How do you check listening ports without root privileges on Linux? (Trick: some tools need root)

# As non-root (no PIDs shown for other users' processes)
ss -tulpn                          # shows most info
ss -tlnp | grep LISTEN

# With root (shows PIDs and process names)
sudo ss -tulpn
sudo netstat -tulpn                # older systems

# Check if specific port is in use
ss -tlnp | grep ':8080'

# Proc filesystem (always accessible)
cat /proc/net/tcp                  # hex format: local_address
cat /proc/net/tcp6                 # IPv6

L24How do you run a command every N seconds without cron? (Asked to test knowledge of built-in tools)

# Method 1: watch (runs every 2 seconds by default)
watch -n 5 "df -h"               # every 5 seconds
watch -n 1 "cat /proc/loadavg"   # every 1 second

# Method 2: while loop
while true; do
  df -h
  sleep 5
done

# Method 3: for production — use systemd timers (not cron)
# Create: /etc/systemd/system/myjob.timer
# and:    /etc/systemd/system/myjob.service

L25What is the difference between /dev/null, /dev/zero, and /dev/urandom? Give a use case for each.

Device	What it does	Real Use Case
`/dev/null`	Discards everything written; returns EOF	`cmd > /dev/null 2>&1` suppress all output
`/dev/zero`	Returns infinite null bytes (0x00)	`dd if=/dev/zero of=file bs=1M count=100` — create empty file
`/dev/random`	Cryptographically secure random (blocks when entropy low)	SSL key generation
`/dev/urandom`	Fast random (doesn't block — uses CSPRNG)	Password generation: `cat /dev/urandom \| tr -dc A-Za-z0-9 \| head -c 32`

BONUSOne-liner cheatsheet: Grep, Awk, Sed — most common interview patterns

# ── grep ──────────────────────────────────────────────
grep -r "pattern" /var/log/       # recursive search
grep -v "pattern" file            # invert match (exclude)
grep -c "pattern" file            # count matches
grep -n "pattern" file            # show line numbers
grep -E "err|warn|crit" file      # extended regex (OR)

# ── awk ──────────────────────────────────────────────
awk '{print $1, $3}' file         # print columns 1 and 3
awk -F: '{print $1}' /etc/passwd  # custom delimiter, print users
awk 'NR==5' file                  # print line 5
awk '/pattern/ {print NR, $0}' f  # print matching lines with numbers
awk '{sum+=$1} END{print sum}' f  # sum column 1

# ── sed ──────────────────────────────────────────────
sed 's/old/new/g' file            # replace all occurrences
sed -i 's/old/new/g' file         # in-place edit
sed -n '5,10p' file               # print lines 5-10
sed '/pattern/d' file             # delete matching lines
sed 's/^/PREFIX: /' file          # add prefix to every line

# ── Practical combinations ────────────────────────────
# Top 5 IPs hitting your Nginx logs
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -5

# Extract all unique HTTP status codes
awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn

# Find failed SSH attempts
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -rn | head -10

⚔️ Command Comparisons — "Which & When?"

C1rsync vs scp vs sftp — differences and when to use each? (Asked at every DevOps interview)

Tool	Resumes?	Delta sync?	Best For
`scp`	❌ No	❌ No	Quick one-off file copy (being deprecated)
`rsync`	✅ Yes	✅ Yes (only diffs)	Backups, large transfers, mirroring
`sftp`	❌ No	❌ No	Interactive browser/upload session

✅ Winner: rsync (almost always)

Only transfers changed blocks — massive speed gain on re-syncs
Resumes interrupted transfers with --partial
Preserves all metadata with -a (archive mode)
Mirror with deletion: --delete

rsync -avz --progress file.txt user@host:/dest/       # basic
rsync -avz --partial --progress bigfile user@host:/dest/  # resumable
rsync -avz --delete /src/ user@host:/dest/             # mirror
rsync -avzn /src/ /dest/                               # dry-run first!

scp file.txt user@host:/dest/                          # simple copy
scp -r ./dir/ user@host:/dest/                         # recursive

sftp user@host       # interactive: ls, get file, put file, mkdir

⚠️ scp is being deprecated

OpenSSH 8.8+ deprecated the old scp protocol. Prefer rsync or sftp in new scripts.

C2locate vs find — which is faster, which is accurate, when to use each?

	`find`	`locate`
Speed	🐢 Real-time disk scan	⚡ Instant (queries DB)
Accuracy	✅ Always current	❌ Stale until `updatedb`
Filters (size/date/perms)	✅ Full	❌ Name only
Works on new files?	✅ Immediately	❌ Only after `updatedb`

locate nginx.conf                         # instant
sudo updatedb                             # refresh locate database
locate -i "*.log"                         # case-insensitive

find / -name "nginx.conf" 2>/dev/null     # real-time, always accurate
find /var -name "*.log" -mtime -1         # modified in last 24h
find /home -name "*.sh" -perm /111        # executable scripts
find /tmp -type f -size +10M -delete      # find AND delete

💡 Rule of thumb

Use locate for quick "where is this file". Use find in scripts or when filtering by size/date/perms, or when file was just created.

C3wget vs curl — when to prefer each?

	`curl`	`wget`
Protocols	20+ (HTTP, HTTPS, SMTP, LDAP…)	HTTP, HTTPS, FTP
Recursive download	❌	✅ `-r`
REST API calls	✅ Excellent	❌ Poor
Custom headers/POST	✅ Full control	⚠️ Limited

# curl — DevOps/API testing choice
curl -X POST https://api.example.com/data \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer TOKEN" \
  -d '{"key":"value"}'
curl -s -o /dev/null -w "%{http_code}" https://site.com  # just status code
curl -C - -O https://example.com/big.iso  # resume download

# wget — simple downloads, recursive mirror
wget https://example.com/file.tar.gz
wget -c https://example.com/big.iso       # resume
wget -r -np https://example.com/docs/     # recursive scrape

C4ps aux vs top vs htop vs pgrep — what is each best for?

Tool	View	Best For
`ps aux`	Snapshot	Scripting, grep for specific process
`top`	Real-time	Quick CPU/MEM check, always available
`htop`	Real-time	Visual, scrollable, kill interactively
`pgrep`	Snapshot	Get PIDs by name in scripts

ps aux | grep nginx                        # find process
ps aux --sort=-%cpu | head -10             # top 10 by CPU
ps aux --sort=-%mem | head -10             # top 10 by memory

pgrep -a nginx                             # PID + command
pgrep -u www-data                          # all PIDs by user
pkill nginx                                # kill by name

# In scripts — always pgrep, not ps | grep
if pgrep -x "nginx" > /dev/null; then echo "running"; fi

C5tar vs zip vs gzip vs bzip2 vs xz — compression comparison?

Tool	Packs dirs?	Speed	Ratio	Best For
`gzip`	❌	⚡ Fast	Medium	Logs, pipelines
`bzip2`	❌	Medium	Good	Better ratio than gzip
`xz`	❌	🐢 Slow	Best	Releases, smallest size
`zip`	✅	⚡ Fast	Medium	Windows compatibility
`tar`	✅	N/A	N/A	Archive, combine with compressor

tar -czf archive.tar.gz  dir/    # gzip (fast)
tar -cjf archive.tar.bz2 dir/    # bzip2 (better)
tar -cJf archive.tar.xz  dir/    # xz (best ratio)
tar -xzf archive.tar.gz          # extract
tar -tf  archive.tar.gz          # list without extracting
# Memory trick: c=Create x=eXtract f=File z=gZip j=bzip2 J=xz

C6iptables vs ufw vs firewalld vs nftables — Linux firewall comparison?

Tool	Distro	Complexity	Modern?
`iptables`	Universal	⚠️ Complex	Legacy (but everywhere)
`ufw`	Ubuntu/Debian	✅ Simple	Frontend for iptables
`firewalld`	RHEL/CentOS	⚠️ Medium	Frontend for nftables
`nftables`	Modern Linux	⚠️ Complex	✅ Replaces iptables

# ufw — simple and human-readable (Ubuntu)
sudo ufw allow 22/tcp && sudo ufw allow 443/tcp
sudo ufw deny 8080
sudo ufw status verbose
sudo ufw enable

# iptables — universal, scriptable
sudo iptables -L -n -v
sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT
sudo iptables -A INPUT -j DROP
sudo iptables-save > /etc/iptables/rules.v4

C7systemctl vs service vs init.d — what changed and why does it matter?

	`systemctl` (systemd)	`service` (SysVinit)
Modern?	✅ Yes (2015+)	Legacy
Parallel boot	✅	❌ Sequential
Logs	`journalctl`	`/var/log/syslog`
Dependency mgmt	✅ Built-in	❌ Manual

sudo systemctl start|stop|restart|reload nginx
sudo systemctl enable nginx           # start on boot
sudo systemctl status nginx           # detailed status + recent logs
sudo systemctl is-active nginx        # for scripts (returns 0 if running)

journalctl -u nginx -f                # follow live logs
journalctl -u nginx --since "1 hour ago"
journalctl -p err -b                  # errors since last boot

C8df vs du vs lsblk vs fdisk -l — storage command comparison?

Command	Shows	Best For
`df -h`	Filesystem usage (mount level)	Check if disk is full
`du -sh`	Directory/file actual disk size	Find what's eating space
`lsblk`	Block device tree (disk → partition)	See disk layout
`fdisk -l`	Partition table details	Partition management
`df -i`	Inode usage	Inode exhaustion debug

df -h && df -i                         # space + inodes
du -sh /var/log/ && du -ah /home | sort -rh | head -10
lsblk -f                               # filesystem type + UUID
sudo fdisk -l                          # partition tables

C9cron vs systemd timers vs at vs anacron — scheduling comparison?

Tool	Missed jobs?	Best For
`cron`	❌ Skipped if server off	Regular periodic tasks
`anacron`	✅ Runs on next boot	Laptops/servers that may power off
`at`	N/A (one-time)	Run once at a specific time
systemd timers	✅ `OnBootSec` catch-up	Production servers, modern

# cron — crontab -e
*/5 * * * * /opt/check.sh              # every 5 min
0 2 * * 0 /opt/backup.sh               # Sunday 2 AM
# MIN HOUR DOM MON DOW CMD

# at — one-time
echo "/opt/deploy.sh" | at 23:00 today
at -l && atrm 3                        # list / remove

systemctl list-timers --all            # see all systemd timers

C10grep vs egrep vs fgrep vs ripgrep (rg) — which and when?

Tool	Regex?	Speed	Notes
`grep`	Basic (BRE)	Medium	Standard, always available
`egrep` / `grep -E`	Extended (ERE)	Medium	Supports `+`, `?`, `\|`, `()`
`fgrep` / `grep -F`	None (literal)	⚡ Fastest	Special chars, IPs, exact strings
`rg` (ripgrep)	Full (RE2)	🚀 Blazing	.gitignore aware, parallel, modern

grep -E "error|warn|crit" /var/log/syslog    # extended regex OR
grep -F "192.168.1.1" access.log              # literal string (fastest)
grep -rn "TODO" /opt/app/                     # recursive + line numbers

rg "pattern" /var/log/                        # recursive, fast
rg -i "error" .                               # case insensitive
rg -l "TODO" .                                # filenames only

C11HARD: cp vs rsync vs dd — the surprising differences that trip people up? (Senior-level)

Command	Metadata?	Sparse files?	Progress?	Best For
`cp`	⚠️ Partial (`-a`)	✅	❌	Simple local copies
`rsync -a`	✅ Full	✅	✅	Backups, sync
`dd`	✅ Byte-for-byte	❌ Fills zeros	⚠️ status=progress	Disk imaging, block copy

cp -a src/ dest/                           # best metadata preservation
rsync -aHAX --progress src/ dest/          # preserve hard links, ACLs, xattrs

dd if=/dev/sda of=/backup/disk.img bs=4M status=progress  # disk image
dd if=/dev/zero of=testfile bs=1M count=100               # create blank file

⚠️ dd = "disk destroyer" mnemonic

Always verify if= (input) and of= (output). Swapping them overwrites your source. Always do a dry-run or double-check before running.

C12HARD: ln -s vs mount --bind vs overlayfs — what are the deep differences? (Container/SRE interview)

Method	Kernel level?	Cross-FS?	Best For
`ln -s` (symlink)	❌ Filesystem path	✅	Simple redirects, config files
`mount --bind`	✅ VFS level	✅	Expose dir at another path, containers
`overlayfs`	✅ VFS layer	N/A	Docker layers, live CDs, copy-on-write

# bind mount — expose /data at /mnt/data (kernel level, no copy)
sudo mount --bind /data /mnt/data
# In /etc/fstab:  /data /mnt/data none bind 0 0

# overlayfs — Docker's storage driver
# lowerdir = read-only base | upperdir = writable layer | workdir = temp
sudo mount -t overlay overlay \
  -o lowerdir=/base,upperdir=/changes,workdir=/tmp/work \
  /merged

💡 Docker connection

Every Docker container uses OverlayFS — each layer is a lowerdir, and your container's writable layer is upperdir. Understanding this explains why docker diff shows changed files.

🚀

Pro Tips & Interview Strategy

30-min Panel · AI DevOps · Questions to Ask

Must Read

▶

✅ Admit What You Don't Know

"I haven't encountered that specific edge case, but my approach would be to check the official documentation, review the logs, and test in a staging environment first."

⚡ AI Agentics — Your Unique Edge

"I'm exploring AI agentic workflows to reduce DevOps toil. Instead of manually parsing CloudWatch logs, AI agents automatically summarize error spikes, suggest root causes, and draft Terraform fixes or Kubernetes rollback commands."

❓ Questions to Ask Them

"What does day-to-day collaboration look like between DevOps and software developers at eOcean?"
"What is the biggest infrastructure or deployment challenge the team is currently solving?"

KEY LEARNINGS SUMMARY

Domain	Key Takeaway
Python/boto3	`UNSIGNED` for public S3, decode with `.read().decode('utf-8')`
Terraform	Remote state (S3+DynamoDB), directory-based multi-env
Docker	Heredoc for scripts, pin versions, `cat Dockerfile` at end
SQL	`LAG()` for row comparison, filter `NULL` from first row
Behavioral	STAR format, measurable outcomes, concise (3-4 sentences)
AWS	SSM Session Manager as SSH fallback, Secrets Manager for creds
Ansible	Handlers for conditional restarts, Vault for secrets
Linux	`find` + `top` + `df -h` + `dmesg` for troubleshooting

🌐 Live Scenarios Platform

Visit interview.naveedkumbhar.com — 957 live DevOps/Cloud/SRE scenarios for additional practice.