ITGOIT - Wiki

Ubuntu server

/dev/sda: clean. files. blocks

sudo apt-get purge nvidia*
sudo ubuntu-drivers autoinstall

Automount a device

Locate the device with lsblk -f and save the UUID

lsblk -f
sudo nano /etc/fstab
#PARTITIONLABLEHERE (dev/PARTITIONNAMEHERE)
UUID=INSERTUUIDHERE       /media/disk/hdd01       ext4    defaults        0       0
Filesystem Type Description Supported Platforms Use Case
ext4 Fourth extended filesystem, a widely used filesystem for Linux, providing journaling and large volume support. Linux Best for Linux OS installations and general use on native Linux disks.
ext3 Older version of ext4 with less performance and features but still stable. Linux Older Linux systems, though less efficient compared to ext4.
ext2 Lacks journaling, making it faster but less safe during crashes. Linux Suitable for small partitions like /boot or USB drives where journaling isn’t necessary.
xfs High-performance journaling filesystem, commonly used in large-scale enterprise systems. Linux Suitable for large file systems, high-performance storage, and server applications.
btrfs Advanced filesystem with features like snapshots and built-in RAID support. Linux Great for advanced use cases like versioning, snapshots, and RAID.
ntfs-3g Userspace driver to mount NTFS with full read/write support on Linux. Linux, Windows For full read/write support of NTFS filesystems on Linux.
ntfs Default Windows filesystem; supports large files and partitions but with limited write support on Linux. Windows, Linux (read-only by default) Use for dual-boot systems or external drives shared between Windows and Linux (with ntfs-3g).
fat32 Older filesystem with wide compatibility but limited to 4GB file size. Linux, Windows, macOS Best for USB drives and external storage with maximum compatibility across platforms.
exfat Supports larger file sizes than FAT32 and has broad compatibility. Linux, Windows, macOS Preferred for external storage (USB, SD cards) shared between multiple OSes.
vfat Another name for FAT32, commonly used when mounting FAT32 on Linux systems. Linux, Windows, macOS Similar to FAT32, used mainly for older external drives or small partitions.
hfs+ Apple’s older hierarchical filesystem, used before APFS. Linux, macOS Suitable for legacy macOS systems; use hfsplus for mounting on Linux.
zfs Advanced filesystem and volume manager known for data integrity, snapshots, and high scalability. Linux, FreeBSD, Solaris Great for large-scale storage, particularly where data integrity is critical.
iso9660 Filesystem standard for optical disc media like CDs/DVDs. Linux, Windows, macOS Used for mounting CDs, DVDs, or ISO image files.

Automount vmware shared folder

sudo nano /etc/fstab
vmhgfs-fuse /media/disk/vmshared fuse defaults,allow_other 0 0 

Mount a device

Locate the device with lsblk

lsblk
cd /media/disk
mkdir hdd01
mount -t ntfs /dev/INSERTDISKNAMEHERE1 /media/disk/hdd01

Mount LDM data partition

sudo apt install testdisk
testdisk

Delete Disk partitions and add new

Locate the device with lsblk

sudo fdisk /dev/INSERTDISKNAMEHERE
d
w
n

CTRL + C

mkfs.ext4 /dev/INSERTDISKNAMEHERE1

Unmount a device

umount /dev/INSERTDISKNAMEHERE1

Format Disk to ext4

Locate the device with lsblk

lsblk
sudo mkfs -t ext4 /dev/INSERTDISKNAMEHERE1

Automatic login as root with passwd for SSH

sudo su
passwd

Sign a message with PGP Encryption

sudo apt install gnupg
sudo apt install gnupg

Remove ipv4 address

ip addr show INSERTNETWORKCARDNAME
sudo ip addr del 192.168.1.123/24 dev INSERTNETWORKCARDNAME

DNS Errors

Able to ping local IPv4 addresses but unable to ping external DNS (e.g., 1.1.1.1 or 1.0.0.1), and DNS resolution failures with error messages such as:

Troubleshooting Steps

  1. Check DNS Configuration:

    • Verify the current DNS configuration using:
      resolvectl status
      
    • Ensure the correct DNS servers (e.g., 1.1.1.1 and 1.0.0.1) are set under the correct network interface.
  2. Test Network Connectivity:

    • Confirm that you can ping external DNS servers:
      ping 1.1.1.1
      
    • If ping to 1.1.1.1 works but DNS queries fail, continue to the next step.
  3. Flush DNS Cache:

    • If using systemd-resolved, flush the DNS cache:
      sudo systemd-resolve --flush-caches
      
    • Check the DNS resolution again using:
      dig google.com
      
  4. Restart Network Services:

    • Restart the DNS resolver:
      sudo systemctl restart systemd-resolved
      
  5. Router Issues:

    • If the issue persists, try restarting your router. This can refresh network configurations and solve connection issues.
  6. Resolution:

    • In this case, restarting the router resolved the DNS timeout issue.

General server crashes/freezes

dmesg | grep -iE "error|fail|critical|warn"
sudo journalctl --since "24 hours ago" > all_logs_last_24_hours.txt

Failed to allocate directory watch: Too many open files

echo "fs.inotify.max_user_instances=1280" | sudo tee -a /etc/sysctl.conf
echo "fs.inotify.max_user_watches=655360" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

VSCODE-Server HIGH CPU/Memory on SSH

Avoid Accessing / with VS Code on Root User (Remote SSH)

Opening the entire filesystem at / results in:

Recommendation

Does htop or btop still show high CPU on program: rg? with command .vscode-server?

  1. Open Visual Studio Code
  2. Go to: File --> Preferences --> Settings
  3. Search for: search.followsymlinks
  4. Disable the following setting: Search: Follow Symlinks - Controls whether to follow symlinks while searching.

Running .sh scripts

bad interpreter: No such file or directory

/bin/bash^M: bad interpreter: No such file or directory

Windows ends every line with two invisible characters: Carriage Return + Line Feed (\r\n).
Linux ends every line with just one: Line Feed (\n).

Run the following command to remove the invisible Windows characters:

sed -i 's/\r$//' your_script_name.sh

Here is a complete Wiki.js Markdown article you can copy and paste directly into your Ubuntu Troubleshooting section.


Failed to allocate directory watch (Too many open files)

Overview

When running a high number of Docker containers (80+) or running heavy filesystem-monitoring workloads like Node.js (Nuxt), GitLab, or Docker/containerd, you may encounter the following error across various system services:

Failed to allocate directory watch: Too many open files

Common Symptoms

The Root Cause

Despite the error saying "Too many open files", the issue is usually not the global file descriptor limit. Instead, the system has exhausted its inotify instances.

Linux uses inotify to monitor files and directories for changes. There are two limits at play:

  1. Watches (max_user_watches): How many individual files/directories can be monitored. (Usually defaults to a high number).
  2. Instances (max_user_instances): How many separate "monitoring buckets" can be created. (Ubuntu defaults to 128).

When running many Docker containers, GitLab microservices, and Node.js watchers (like chokidar), the default 128 instances are quickly exhausted. Once the system runs out of instances, it throws the EMFILE (Too many open files) error.

Diagnosis

You can check your current inotify limits by running:

sysctl fs.inotify.max_user_instances
sysctl fs.inotify.max_user_watches

If your max_user_instances is set to 128, it is highly likely the cause of the bottleneck.

The Solution

To fix this, we need to permanently increase the inotify instance limits in the kernel parameters. This uses a microscopic amount of RAM and is considered best practice for Docker hosts and CI/CD runners.

1. Apply the Limits

Run the following commands to append the new, higher limits to your /etc/sysctl.conf file:

# Increase the maximum number of inotify instances
echo "fs.inotify.max_user_instances=8192" | sudo tee -a /etc/sysctl.conf

# Increase the max user watches slightly (optional but recommended for Node.js workloads)
echo "fs.inotify.max_user_watches=524288" | sudo tee -a /etc/sysctl.conf

2. Reload the Kernel Parameters

Apply the changes immediately without needing to reboot the server:

sudo sysctl -p

3. Cleanup & Verification

Because system processes like systemd may have crashed or throttled while the limit was exhausted, it is recommended to reload the systemd daemon:

sudo systemctl daemon-reexec

Finally, verify the fix by running the command that initially failed:

sudo apt update

Tip: If any specific Docker containers are still acting up after applying the fix, simply restart them (docker restart <container_name>) or restart the Docker daemon (sudo systemctl restart docker) to allow them to boot with the newly available inotify resources.