No jargon, no complexity, real solutions on the command line.
Whether you keep systems running, manage deployments, or develop backend services, these 10 commands will help you during 3 AM outages.
1. iostat
Simple and practical description to check disk performance in real time and detect which storage device is the bottleneck.
Usage example iostat -x 1
Key indicators: %util if above 80% indicates a possible bottleneck; await shows the average I/O wait time in milliseconds; iowait indicates CPU time waiting for disk operations.
2. vmstat
Quick view of memory, CPU, and I/O activity to identify memory pressure or processes waiting for CPU.
Usage example vmstat 1
Key indicators: si and so show swap in and swap out; wa indicates percentage of I/O wait; r shows ready processes waiting for CPU.
3. lsof
Lists all open files, sockets, and network connections and shows which process owns them. Useful for port conflicts and descriptor leaks.
Practical examples lsof -i :8080 and to find handle users lsof | awk {print $2} | sort | uniq -c | sort -nr | head -10
Use cases: detect port conflicts before starting an app; locate descriptor leaks with lsof -p; view network connections with lsof -i.
4. sar
Collects historical performance data to understand patterns over time, ideal for investigating nighttime spikes.
Usage example sar -u 1 10 and to review history sar -f /var/log/sysstat/sa...
Useful metrics: sar -u for CPU, sar -r for memory, and sar -n DEV for network statistics.
5. iotop
Shows which processes are generating disk I/O in real time, sorted by actual usage.
Usage example iotop -o
Practical uses: identify processes hogging I/O, view reads and writes per process, and detect out-of-control processes affecting disks. The -o flag filters processes that actually perform I/O.
6. strace
Shows the system calls a process makes. Ideal for debugging file and socket access and understanding why a process fails or hangs.
Useful commands strace -f -e trace=file for file calls only, strace -e trace=network for network, and strace -p -f -o /tmp/trace.log to attach to a process and save the trace.
Tip: use -T to see the time consumed by each call.
7. ss
Modern replacement for netstat to view sockets and connection states with process information and per-socket usage.
Usage example ss -tulpn
Diagnostic commands: ss -o state established for established connections, ss -m for per-socket memory, and ss -s for socket state summary.
8. dstat
Combined view of CPU, disk, network, and memory on a clear screen, useful for quick inspections.
Usage example dstat -cdngy and to view top processes dstat --top-cpu --top-io --top-mem 5
Common flags: -c CPU, -d disk, -n network, -g pages, and -y system statistics. Ideal for quick and visual diagnostics.
9. pidstat
Monitors resource usage per process over time, showing trends and allowing per-thread analysis.
Usage example pidstat -u -r -d 1 and for specific processes pidstat -p
Advantages: shows trends instead of snapshots, provides per-thread statistics with -t, and integrates well with sar for histories.
10. perf
Advanced tool for deep CPU analysis: cache misses, branch prediction failures, and information on instruction efficiency.
Basic commands perf top, to profile perf record -g and then perf report, and for full system metrics perf stat -a sleep 10
Useful for identifying CPU bottlenecks and low-level performance issues when simple approaches are not enough.
Q2BSTUDIO is a custom software and application development company specializing in practical, no-nonsense solutions. We offer custom software, custom applications, artificial intelligence and AI projects for businesses, integrated AI agents, and cybersecurity services. We also provide AWS and Azure cloud services, business intelligence services, and Power BI dashboard development to turn data into decisions. Our approach is clear: effective tools and commands to resolve incidents now, and architecture and automation so they don't happen again.
If you need support with Linux observability, system optimization, security, or developing solutions with artificial intelligence and Power BI, Q2BSTUDIO can help you with custom projects and specialized consulting.
Thanks for reading and remember to keep these commands handy the next time the phone rings at 3 AM.
