Linux Hardening for Security Engineers

TL;DR: Linux hardening reduces your attack surface by removing unnecessary services, enforcing least privilege, securing SSH, and implementing proper logging. These fundamentals are essential for security engineering interviews and real-world defensive operations.

Table of Contents

Quick Reference

Essential Hardening Commands

Key Configuration Files

Introduction to Linux Hardening

What is Linux Hardening?

Linux hardening is the process of securing a Linux system by reducing its attack surface. This involves:

• Removing unnecessary software and services
• Configuring secure defaults
• Implementing access controls
• Enabling comprehensive logging
• Applying security patches

Why Security Engineers Must Understand Hardening

As a security engineer, you’ll be expected to:

1. Assess - Evaluate Linux systems for security weaknesses
2. Recommend - Provide hardening guidance to operations teams
3. Validate - Verify security controls are properly implemented
4. Respond - Investigate incidents on Linux systems
5. Automate - Build security automation for Linux environments

Interview Insight: Interviewers often ask “How would you secure a fresh Linux server?” This question tests your systematic approach to hardening.

The Principle of Least Privilege

Every hardening decision should follow this principle:

Grant only the minimum permissions necessary for a user,
process, or system to perform its intended function.

This applies to:

• User accounts and groups
• File and directory permissions
• Network services and ports
• Sudo privileges
• Application capabilities

Threat Modeling for Linux

Before hardening, understand what you’re protecting against.

Common Linux Attack Vectors

Threat Model Questions

Ask yourself:

1. What data exists on this system? (sensitivity level)
2. Who needs access? (users, services, external systems)
3. What services must run? (minimize attack surface)
4. Where does it sit in the network? (exposure level)
5. What compliance requirements apply? (PCI, HIPAA, etc.)

Attack Surface Categories

┌─────────────────────────────────────────────────────────────┐
│                    LINUX ATTACK SURFACE                      │
├─────────────────┬─────────────────┬─────────────────────────┤
│   NETWORK       │   LOCAL         │   PHYSICAL              │
├─────────────────┼─────────────────┼─────────────────────────┤
│ • Open ports    │ • User accounts │ • Console access        │
│ • Running svcs  │ • SUID binaries │ • USB devices           │
│ • SSH config    │ • Cron jobs     │ • Boot loader           │
│ • Firewall      │ • File perms    │ • BIOS/UEFI             │
│ • DNS/NTP       │ • Kernel params │ • Single user mode      │
└─────────────────┴─────────────────┴─────────────────────────┘

User and Group Management

User Account Security

Audit Existing Users

# List all users
cat /etc/passwd

# List users with login shells
grep -v '/nologin\|/false' /etc/passwd

# Find users with UID 0 (root privileges)
awk -F: '($3 == 0) {print}' /etc/passwd

# Check for empty passwords
sudo awk -F: '($2 == "") {print $1}' /etc/shadow

# Find users with no password expiration
sudo chage -l username

Secure User Creation

# Create user with specific home directory and shell
useradd -m -d /home/appuser -s /bin/bash appuser

# Create system user (no login, no home)
useradd -r -s /usr/sbin/nologin serviceaccount

# Set password with expiration
passwd appuser
chage -M 90 -W 14 appuser  # 90 day max, 14 day warning

Remove/Lock Unnecessary Accounts

# Lock user account
usermod -L username

# Disable login shell
usermod -s /usr/sbin/nologin username

# Delete user and home directory
userdel -r username

# Check locked accounts
passwd -S username  # Look for 'L' status

Group Management

Security-Sensitive Groups

# List group members
getent group sudo

# Remove user from group
gpasswd -d username groupname

# Audit all group memberships
for user in $(cut -d: -f1 /etc/passwd); do
  groups $user 2>/dev/null
done

Sudo Configuration

Secure sudoers Best Practices

# Always use visudo to edit
sudo visudo

# Or edit drop-in files
sudo visudo -f /etc/sudoers.d/custom

Secure sudoers example:

# /etc/sudoers.d/secure-defaults

# Require password for sudo
Defaults    authenticate

# Log sudo commands
Defaults    logfile="/var/log/sudo.log"

# Timeout after 5 minutes
Defaults    timestamp_timeout=5

# Don't allow sudo -i or sudo su
Defaults    requiretty

# Restrict specific user to specific commands
webadmin    ALL=(ALL) /usr/bin/systemctl restart nginx, /usr/bin/systemctl status nginx

File Permissions and Common Mistakes

Understanding Linux Permissions

Permission Format: -rwxrwxrwx
                   │└┬┘└┬┘└┬┘
                   │ │  │  └── Others (o)
                   │ │  └───── Group (g)
                   │ └──────── User/Owner (u)
                   └────────── File type (- = file, d = directory)

Numeric: rwx = 4+2+1 = 7
         rw- = 4+2+0 = 6
         r-- = 4+0+0 = 4
         --- = 0+0+0 = 0

Critical Permission Settings

Common Permission Mistakes

Mistake 1: World-Readable Sensitive Files

# BAD: Anyone can read
-rw-r--r-- 1 root root config.yml  # Contains passwords!

# FIX: Restrict access
chmod 600 config.yml
chown root:root config.yml

Mistake 2: World-Writable Directories

# Find world-writable directories (excluding /tmp, /var/tmp)
find / -type d -perm -0002 ! -path "/tmp*" ! -path "/var/tmp*" 2>/dev/null

# FIX: Remove world-write
chmod o-w /path/to/directory

Mistake 3: Missing Sticky Bit on Shared Directories

# Check if sticky bit is set
ls -ld /tmp
# Should show 't' at end: drwxrwxrwt

# Set sticky bit
chmod +t /shared/directory

SUID/SGID Binaries - The Privilege Escalation Goldmine

SUID (Set User ID): Executes with owner’s privileges SGID (Set Group ID): Executes with group’s privileges

# Find all SUID binaries
find / -perm -4000 -type f 2>/dev/null

# Find all SGID binaries
find / -perm -2000 -type f 2>/dev/null

# Find both
find / -perm /6000 -type f 2>/dev/null

Dangerous SUID Binaries

# Remove SUID bit
chmod u-s /path/to/binary

# Compare against known-good baseline
# Ubuntu default SUID binaries:
/usr/bin/passwd
/usr/bin/sudo
/usr/bin/chfn
/usr/bin/chsh
/usr/bin/newgrp
/usr/bin/gpasswd

SSH Hardening Best Practices

SSH is often the primary remote access method and a prime target.

SSH Configuration Hardening

Edit /etc/ssh/sshd_config:

# Disable root login
PermitRootLogin no

# Use SSH Protocol 2 only (default in modern OpenSSH)
Protocol 2

# Disable password authentication (use keys only)
PasswordAuthentication no
PermitEmptyPasswords no

# Limit authentication attempts
MaxAuthTries 3

# Set login grace time
LoginGraceTime 60

# Disable X11 forwarding (if not needed)
X11Forwarding no

# Disable TCP forwarding (if not needed)
AllowTcpForwarding no

# Limit users who can SSH
AllowUsers deployer admin
# OR limit by group
AllowGroups sshusers

# Use strong ciphers and MACs
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com
KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org

# Enable strict mode
StrictModes yes

# Log level for auditing
LogLevel VERBOSE

# Idle timeout (disconnect after 5 min idle)
ClientAliveInterval 300
ClientAliveCountMax 0

After changes:

# Validate configuration
sshd -t

# Restart SSH
systemctl restart sshd

SSH Key Management

# Generate strong key (Ed25519 preferred)
ssh-keygen -t ed25519 -C "user@hostname"

# Or RSA with 4096 bits minimum
ssh-keygen -t rsa -b 4096 -C "user@hostname"

# Set proper permissions
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub
chmod 600 ~/.ssh/authorized_keys

Fail2Ban Configuration

Install and configure Fail2Ban to block brute-force attempts:

# Install
apt install fail2ban  # Debian/Ubuntu
yum install fail2ban  # RHEL/CentOS

# Create local config
cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Edit /etc/fail2ban/jail.local:

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
findtime = 600

# Start and enable
systemctl enable --now fail2ban

# Check banned IPs
fail2ban-client status sshd

SSH Hardening Checklist

• Disable root login
• Use key-based authentication only
• Disable password authentication
• Use strong ciphers and MACs
• Limit SSH users/groups
• Set idle timeout
• Configure Fail2Ban
• Change default port (security through obscurity, optional)
• Enable verbose logging

Service Minimization

Principle: If It’s Not Needed, Remove It

Every running service is a potential vulnerability.

Audit Running Services

# List all running services
systemctl list-units --type=service --state=running

# List enabled services (start at boot)
systemctl list-unit-files --type=service --state=enabled

# List listening ports and their services
ss -tulnp
# or
netstat -tulnp

Common Unnecessary Services

Disable Unnecessary Services

# Stop and disable a service
systemctl stop cups
systemctl disable cups

# Or in one command
systemctl disable --now cups

# Mask service (prevent starting even manually)
systemctl mask cups

# Check service status
systemctl status cups

Firewall Configuration

Use firewalld (RHEL/CentOS) or ufw (Ubuntu):

UFW (Ubuntu):

# Enable UFW
ufw enable

# Default deny incoming
ufw default deny incoming

# Allow specific services
ufw allow ssh
ufw allow 443/tcp

# Allow from specific IP
ufw allow from 192.168.1.0/24 to any port 22

# Check status
ufw status verbose

Firewalld (RHEL):

# Start and enable
systemctl enable --now firewalld

# Add service
firewall-cmd --permanent --add-service=ssh

# Remove service
firewall-cmd --permanent --remove-service=cockpit

# Reload
firewall-cmd --reload

# List active rules
firewall-cmd --list-all

Service Minimization Checklist

• List all running services
• Identify business-critical services
• Disable non-essential services
• Mask services that should never run
• Configure firewall to allow only needed ports
• Document approved services baseline

Logging and Monitoring Basics

Why Logging Matters

Logs are your forensic evidence. Without logs:

• You can’t detect intrusions
• You can’t investigate incidents
• You can’t prove compliance

Essential Log Files

Configure rsyslog for Remote Logging

Edit /etc/rsyslog.conf:

# Send all logs to remote SIEM
*.* @@siem.company.com:514

# Or specific facilities
auth,authpriv.* @@siem.company.com:514

Auditd - Linux Audit Framework

Auditd provides detailed system call auditing.

# Install
apt install auditd audispd-plugins  # Debian/Ubuntu
yum install audit                    # RHEL/CentOS

# Enable and start
systemctl enable --now auditd

Example audit rules (/etc/audit/rules.d/security.rules):

# Monitor password file changes
-w /etc/passwd -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/group -p wa -k identity
-w /etc/sudoers -p wa -k sudoers

# Monitor SSH config
-w /etc/ssh/sshd_config -p wa -k sshd_config

# Monitor privileged commands
-a always,exit -F path=/usr/bin/sudo -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged

# Monitor file deletions
-a always,exit -F arch=b64 -S unlink -S unlinkat -S rename -S renameat -F auid>=1000 -F auid!=4294967295 -k delete

# Load rules
augenrules --load

# Search audit logs
ausearch -k identity
ausearch -k sudoers

# Generate reports
aureport --summary
aureport --auth

Log Retention and Protection

# Configure logrotate for security logs
# /etc/logrotate.d/security

/var/log/auth.log {
    weekly
    rotate 52
    compress
    delaycompress
    notifempty
    create 640 root adm
}

Protect logs from tampering:

# Make log files append-only
chattr +a /var/log/auth.log

# Check attribute
lsattr /var/log/auth.log

Quick Hardening Checklist

Initial System Hardening

## User & Access
- [ ] Remove/disable unnecessary user accounts
- [ ] Enforce password policy (length, complexity, expiration)
- [ ] Configure sudo with least privilege
- [ ] Disable root login (SSH and console where possible)
- [ ] Review group memberships

## SSH
- [ ] Disable password authentication
- [ ] Disable root login
- [ ] Use strong ciphers
- [ ] Configure Fail2Ban
- [ ] Set idle timeout

## File System
- [ ] Set proper permissions on sensitive files
- [ ] Audit and minimize SUID/SGID binaries
- [ ] Set sticky bit on shared directories
- [ ] Configure umask (027 or 077)

## Services & Network
- [ ] Disable unnecessary services
- [ ] Configure host-based firewall
- [ ] Limit listening ports to required only
- [ ] Disable IPv6 if not used

## Logging & Monitoring
- [ ] Enable auditd
- [ ] Configure remote logging
- [ ] Protect log files
- [ ] Set up log rotation

## Patching
- [ ] Enable automatic security updates
- [ ] Subscribe to security mailing lists
- [ ] Regular patch review process

## Kernel Hardening (sysctl)
- [ ] Disable IP forwarding (if not a router)
- [ ] Enable SYN cookies
- [ ] Disable ICMP redirects
- [ ] Enable address space layout randomization (ASLR)

Sysctl Hardening

Add to /etc/sysctl.d/99-security.conf:

# Disable IP forwarding
net.ipv4.ip_forward = 0
net.ipv6.conf.all.forwarding = 0

# Enable SYN flood protection
net.ipv4.tcp_syncookies = 1

# Disable ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0

# Don't send ICMP redirects
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0

# Enable ASLR
kernel.randomize_va_space = 2

# Restrict kernel pointer exposure
kernel.kptr_restrict = 2

# Restrict dmesg access
kernel.dmesg_restrict = 1

# Apply changes
sysctl -p /etc/sysctl.d/99-security.conf

Hands-On Lab

Lab: Harden a Vulnerable Linux VM

Objective: Take a deliberately misconfigured Linux VM and apply hardening measures.

Setup:

1. Use a fresh Ubuntu Server or CentOS VM
2. Or use a vulnerable VM like Metasploitable

Lab Tasks:

Task 1: User Audit

# Find all users with shells
grep -v '/nologin\|/false' /etc/passwd

# Find users with UID 0
awk -F: '($3 == 0) {print}' /etc/passwd

# Check for empty passwords
sudo awk -F: '($2 == "") {print $1}' /etc/shadow

# ACTION: Lock unnecessary accounts

Task 2: SSH Hardening

# Backup original config
sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak

# Apply hardening (create key first!)
ssh-keygen -t ed25519

# Edit sshd_config:
# - PermitRootLogin no
# - PasswordAuthentication no
# - MaxAuthTries 3

# Test and restart
sudo sshd -t && sudo systemctl restart sshd

Task 3: Find and Fix Dangerous SUID Binaries

# Find SUID binaries
find / -perm -4000 -type f 2>/dev/null > /tmp/suid_binaries.txt

# Compare against baseline
# Remove SUID from non-essential binaries

Task 4: Service Minimization

# List running services
systemctl list-units --type=service --state=running

# Identify and disable unnecessary services
# Example: cups, avahi-daemon, etc.

# Configure firewall
sudo ufw enable
sudo ufw default deny incoming
sudo ufw allow ssh

Task 5: Enable Audit Logging

# Install and configure auditd
sudo apt install auditd

# Add rules for sensitive files
# Start auditd
sudo systemctl enable --now auditd

Verification:

• Run a vulnerability scanner (OpenVAS, Nessus)
• Compare before/after security posture
• Document all changes made

Interview Questions & Answers

Basic Questions

Q1: How would you secure a fresh Linux server?

Strong Answer: “I follow a systematic approach:

1. Update the system - Apply all security patches
2. User management - Disable root login, create admin user with sudo, enforce password policy
3. SSH hardening - Key-based auth only, disable root login, configure fail2ban
4. Service minimization - Disable unnecessary services, configure firewall for only required ports
5. File permissions - Audit SUID binaries, secure sensitive files
6. Logging - Enable auditd, configure remote logging to SIEM
7. Kernel hardening - Apply sysctl security parameters

I’d also baseline the system and set up file integrity monitoring.”

Q2: Explain Linux file permissions.

Strong Answer: “Linux uses a permission model with three access types: read (r=4), write (w=2), and execute (x=1) for three categories: owner, group, and others.

For example, chmod 750 means:

• Owner: rwx (7 = 4+2+1)
• Group: r-x (5 = 4+0+1)
• Others: --- (0)

Special permissions include SUID (runs as file owner), SGID (runs as group), and sticky bit (only owner can delete in shared directories).

From a security perspective, I always check for world-writable files, overly permissive configs, and unnecessary SUID binaries which are common privilege escalation vectors.”

Q3: How would you harden SSH?

Strong Answer: “Key SSH hardening measures include:

1. Disable password auth - Use key-based authentication only
2. Disable root login - Set PermitRootLogin no
3. Use strong ciphers - Remove weak algorithms
4. Limit users - Use AllowUsers or AllowGroups
5. Rate limiting - Configure MaxAuthTries and use Fail2Ban
6. Idle timeout - Set ClientAliveInterval
7. Logging - Enable LogLevel VERBOSE for audit trail

I’d also consider port knocking or VPN for additional protection on sensitive systems.”

Intermediate Questions

Q4: What’s the difference between disabling and masking a service?

Strong Answer: “Disabling a service (systemctl disable) prevents it from starting automatically at boot, but it can still be started manually. Masking (systemctl mask) creates a symlink to /dev/null, preventing the service from being started by any means, even manually.

For security-critical situations where a service should never run, masking is preferred because it provides stronger protection against accidental or malicious starts.”

Q5: How do you find and assess SUID binaries?

Strong Answer: “I use find / -perm -4000 -type f 2>/dev/null to locate SUID binaries. Then I:

1. Compare against baseline - Known defaults for the OS
2. Check GTFOBins - For privilege escalation potential
3. Assess necessity - Does this binary need SUID?
4. Remove if unnecessary - chmod u-s /path/to/binary

Common dangerous SUID binaries include vim, find, nmap, and interpreters like python/perl because they can spawn shells or execute arbitrary code with elevated privileges.”

Q6: How would you detect if someone is trying to brute-force SSH?

Strong Answer: “Several detection methods:

1. Log analysis - Check /var/log/auth.log for repeated ‘Failed password’ entries from same IP
2. Real-time monitoring - Use journalctl -fu sshd to watch live
3. Fail2Ban status - Check fail2ban-client status sshd for banned IPs
4. Auditd - Configure rules to alert on authentication events
5. SIEM alerts - Correlate failed login events across time windows

Prevention includes key-based auth, fail2ban, and geographic IP restrictions if applicable.”

Advanced Questions

Q7: Explain the Linux audit framework and how you’d use it for security monitoring.

Strong Answer: “The Linux Audit Framework (auditd) provides system call-level auditing. It consists of:

• auditd - The daemon that writes logs
• auditctl - Manages rules dynamically
• ausearch/aureport - Query and report on logs

For security, I’d configure rules to monitor:

• Changes to /etc/passwd, /etc/shadow, /etc/sudoers
• Privileged command execution (sudo, su)
• File deletions and modifications in sensitive directories
• System call filtering for specific operations

The logs feed into our SIEM for correlation with other security events. This provides the detailed forensic trail needed for incident response and compliance.”

Q8: How would you respond if you discovered an unauthorized SUID binary?

Strong Answer: “This is a potential indicator of compromise, so I’d:

1. Don’t modify it yet - Preserve evidence
2. Document - File hash, permissions, timestamps, location
3. Check timestamps - When was it created/modified?
4. Analyze the binary - What does it do? (strings, file, strace)
5. Check for related IOCs - Other suspicious files, processes, connections
6. Escalate - Follow incident response procedures
7. After investigation - Remove SUID, potentially isolate system
8. Hunt - Check other systems for similar artifacts

The key is treating it as a potential incident, not just a misconfiguration.”

Glossary

What’s Next

Now that you understand Linux hardening fundamentals, continue with:

1. Bash and PowerShell for Security Automation - Automate hardening tasks
2. Linux Privilege Escalation - Understand what you’re defending against
3. Detection Engineering Basics - Build detections for Linux attacks

Questions or feedback? Open an issue on GitHub.