Security
Security Essentials
A defensive baseline for Linux hosts — MFA, least privilege, SSH hardening, TLS, secret rotation, patching, audit trails, and tested backups, each with a one-line way to verify it holds.
Quick reference — reach for these first
ss -tlnpList every listening TCP socket with its process — anything unexpected is attack surface to close.find / -xdev -perm -4000 -type f 2>/dev/nullEnumerate setuid binaries; an unknown one is a classic privilege-escalation foothold.openssl s_client -connect scm.cc:443 -tls1_2Inspect the served cert and negotiated protocol; check expiry, chain, and that TLS<1.2 is refused.ssh-keygen -lf ~/.ssh/authorized_keysShow fingerprint, bits, and type for each trusted key — spot weak or unrecognized keys.sudo lastb -a | headRead /var/log/btmp for bad login attempts; a flood from one IP signals brute force.Defense in depth: layered controls around the asset
Text equivalent: Five concentric rings of defense wrap a central data/asset core. From outermost to innermost: (1) Network — default-deny firewall, only ports 22 and 443 open; (2) Identity — MFA and least-privilege sudo; (3) Access — SSH keys only, no root login, tight file permissions; (4) Data — TLS 1.2+ with HSTS and rotated short-lived secrets; (5) the Asset core. A continuous audit-and-backup band runs around every ring: central append-only logs plus tested restores, because if one layer fails the next still holds and the audit trail records the breach.
Common tasks
Risk · Control · How to verify
| Risk | Control | How to verify |
|---|---|---|
| Weak / single-factor auth | Enroll TOTP MFA (pam_google_authenticator or an IdP); require it for SSH + admin UIs | grep -R 'pam_google_authenticator' /etc/pam.d/ && test a login — a correct password alone is rejected |
| Over-broad access (standing admin) | Scope sudo to named commands; drop shared root; grant least privilege per role | sudo -l -U USER lists only intended commands; getent group sudo wheel shows a short, known membership |
| SSH exposed to password + root | Set PasswordAuthentication no, PermitRootLogin no, PubkeyAuthentication yes; reload sshd | sshd -T | grep -E 'passwordauthentication|permitrootlogin' shows 'no'; ssh -o PreferredAuthentications=password fails |
| Loose Linux file permissions | chmod 600 secrets/keys, 700 ~/.ssh; audit world-writable + SUID files | find / -xdev -perm -0002 -type f 2>/dev/null and find / -xdev -perm -4000 both return only expected paths |
| Open network / unneeded ports | Default-deny firewall; allow only needed services (e.g. 22,443) | sudo firewall-cmd --list-all (or sudo ufw status verbose) shows deny-by-default + the intended allow-list; ss -tlnp agrees |
| Long-lived secrets & tokens | Rotate on a schedule and on exposure; prefer short-lived tokens from a vault | Secret manager / git log shows rotation within policy window; the prior credential now returns 401/403 |
| Unpatched OS & services | Apply security updates; enable unattended-upgrades for critical patches | apt list --upgradable (or dnf updateinfo list security) is empty; needrestart reports no stale services |
| Vulnerable dependencies | Scan builds in CI: npm audit / pip-audit / trivy fs . | CI gate fails on High/Critical; latest scan report shows 0 unfixed High+ findings |
| Weak / expiring TLS | Serve TLS 1.2+ only, modern ciphers, valid chain; send HSTS header | openssl s_client -connect scm.cc:443 -tls1_1 is refused; curl -sI https://scm.cc | grep -i strict-transport-security |
| No audit trail | Ship auth, sudo, and app logs to a central, append-only store; enable auditd | Central index receives today's events; auditctl -l shows active rules; local tampering doesn't erase the remote copy |
| Unproven backups | Automate backups AND run periodic restore drills to a scratch host | Last drill log shows a successful restore + checksum match within RPO; restore time meets the RTO target |
| Shared / orphaned accounts | One identity per human; disable departed users; expire inactive keys | lastlog flags never/stale logins; passwd -S lists no unexpected active accounts; no shared logins in auth logs |
Containing a leaked GitHub token (app.shipreadymetrics.com)
incident
curl -sS -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $LEAKED_TOKEN" https://api.github.com/user
# 200 -> token is VALID and active; treat as a live incident# Revoke in the GitHub UI / API, then re-test:
curl -sS -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $LEAKED_TOKEN" https://api.github.com/user
# 401 -> old token is dead# GitHub Settings > Security log, and per-repo:
gh api /repos/OWNER/REPO/events --paginate | jq '.[] | {actor:.actor.login, type, created_at}'
# Look for pushes, deploy-key adds, or workflow changes you did not makegh secret set DEPLOY_TOKEN --repo OWNER/REPO --body "$NEW_TOKEN"
# scrub history if the value was committed:
git filter-repo --replace-text <(echo "$LEAKED_TOKEN==>REDACTED")
# CI now authenticates with the new secret; old value no longer appears in git log -pcurl -sS -o /dev/null -w 'new=%{http_code}\n' -H "Authorization: Bearer $NEW_TOKEN" https://api.github.com/user
gh api /repos/OWNER/REPO/keys | jq '.[].title'
# new=200, old=401, key list contains only expected deploy keysTroubleshooting — symptom → first check → next check
| Symptom | First check | Establishes | Next check |
|---|---|---|---|
| Burst of failed SSH logins from one or many IPs | sudo lastb -a | head and journalctl -u ssh -S -1h | grep -i 'failed' | Confirms brute-force attempts and whether any succeeded (watch for an 'Accepted' line after the flood) | Ensure password auth is off and keys-only; add fail2ban / rate limits; firewall-restrict source ranges; rotate any account that shows a success |
| A port is open that no service should be exposing | ss -tlnp | grep :PORT then map the PID to its unit | Identifies the listening process and whether it is expected or a backdoor/misconfig | Stop/disable the service if unneeded, bind it to localhost, and tighten the firewall default-deny; re-scan to confirm it's gone |
| Browser or client rejects the site's TLS certificate | openssl s_client -connect scm.cc:443 2>/dev/null | openssl x509 -noout -dates -issuer | Shows notBefore/notAfter and issuer — distinguishes expiry from a broken chain or wrong hostname | Renew/reissue (ACME or CA), deploy the full chain, reload the server; verify with a fresh s_client and an external checker |
| A secret (key/token) was committed to git | git log -p -S 'SECRET_SUBSTRING' -- PATH and probe whether the credential still authenticates | Confirms exposure and whether the leaked credential is currently live | Revoke/rotate first, then scrub history (filter-repo/BFG) and force-update; treat as compromised even if the repo is private |
| Backup restore drill fails or data is unreadable | Restore the latest backup to a scratch host and checksum against source | Proves whether backups are actually recoverable vs. silently corrupt/incomplete | Fix the backup job (encryption keys, retention, integrity checks), re-run the drill, and record a passing restore within RPO/RTO |
| sudo grants more than a user should have | sudo -l -U USER and grep -R '' /etc/sudoers /etc/sudoers.d | Reveals NOPASSWD/ALL grants and over-broad group membership | Replace blanket ALL with named commands, require a password, remove stale sudoers.d drop-ins; re-run sudo -l to confirm |
| Local logs look edited or gaps appear | auditctl -l and compare local auth logs against the central/append-only store | Shows whether auditing is active and whether an attacker truncated local evidence | Rely on the immutable central copy, enable auditd + log shipping if missing, and investigate the gap as potential tampering |
| Dependency scanner reports High/Critical CVEs | trivy fs . (or npm audit / pip-audit) and read the fixed-version column | Confirms which packages are vulnerable and whether a fix is available | Bump to patched versions, rebuild, re-scan to zero; gate the CI pipeline so new High+ findings block merge |
References
- OpenSSH sshd_config manualman.openbsd.org
- OpenSSL s_client commanddocs.openssl.org
- Linux auditd (audit.rules) manualman7.org
- MDN: HTTP Strict-Transport-Securitydeveloper.mozilla.org
- CISA — Rotate exposed credentials / secrets guidancecisa.gov
Linux host (systemd, iproute2), OpenSSH 8+, OpenSSL ≥3, firewalld or ufw; commands run as root or via sudo.