The Linux server's hard disk space is insufficient and cannot be written, but there is still some space left on the hard disk. It turns out that Inodes are to blame!

Ubuntu系统Inodes耗尽，当你遇到硬盘空间不足无法写入的问题

exist Linux On a server, even if the hard disk seems to have enough space, you may still encounterUnable to writeThis is usually due to inodes Exhausted.

When the hard disk cannot be writtenDisk spaceIf there are still some remaining, it is likely that inodes are exhausted. You can clean up small files, adjustFile SystemOr archiving storage and other methods to solve the problem, and cooperate with monitoring and optimization design to prevent similar problems from happening again.

What are inodes?

Inodes are data structures in the file system that store meta-information about files (such as file permissions, owner, creation time, etc.).
Each file occupies an inode, no matter how big the file is. If the inodes are exhausted, even if the hard disk still hasRemaining space, and new files cannot be created.

How to check the usage of hard disk and inodes?

Check Disk Space

df -h

This will show you the disk usage in a human readable format.

Check inodes usage

df -i

Sample output:

Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda1 1000000 999999 1 100% /

IUse% indicates the usage rate of inodes. If IUse% reaches 100%, it means that inodes are exhausted.

• Too many small files will consume a large number of inodes, especially log files and temporary files.
• File system type. Some file systems (such as ext4) will allocate a fixed number of inodes when they are created and cannot be adjusted dynamically.
• Application problem: The application may continuously generate small files, causing inodes to be quickly exhausted.

Solution to inodes occupying 100%

First, delete the system log files

sudo rm -rf /var/log/*.log

After deleting the system log files, it is found that the usage rate of inodes is still 100% or has only decreased a little. At this time, it is necessary to find out which directory files in the system occupy too much space.

Generally in Linux, /home stores website program files, and system program files are generally in the /var directory.

Next, check which directories are taking up too much space, check the size of the folders in the root directory, and sort them by size.

du -h --max-depth=1 / | sort -hr

Further check which directories in the var directory occupy a large amount of files based on the directory size. Follow this method to check other directories.

du -h --max-depth=1 /var | sort -hr

During the investigation, I found that /usr/local/lsws/logs occupied 9.4G. When I entered the directory, I saw an auditmodsec.log file that occupied almost 8G.

auditmodsec.log is the audit log file of ModSecurity, which records the HTTP requests and response data that trigger security rules.

Continuing to investigate, we found that the /var/lib/lsphp/session/lsphp83 directory took up too much space and had too many files.

The /var/lib/lsphp/session/lsphp83 directory is the storage directory related to LiteSpeed PHP (lsphp) sessions. This directory is usually related to LSAPI (LiteSpeed SAPI), which is the API used by LiteSpeed Web Server to handle PHP scripts.

/var/lib/lsphp/session is the directory where LiteSpeed PHP session files are stored. PHP session files are stored here and named according to the session ID.

If there are too many or too large files in the session directory, it may cause insufficient storage space. Clean up invalid session files regularly.

The inodes mentioned above occupy 100%. Most of the time, it is a problem with this directory. Due to the huge number of files in the /var/lib/lsphp/session folder, inodes occupy 100% and the hard disk cannot write files. In addition to the hard disk being full, another reason is that inodes occupy 100%

Both of these problems will cause the hard disk to be unable to write content. When the hard disk cannot write files, the web or database needs to write temporary files during operation. Failure to write temporary files will cause website paralysis, database paralysis and other problems.

Cleaning and managing session files

You can use cron to regularly clean up expired session files to prevent the directory from taking up too much space. Usually, expired session files are automatically cleaned up according to system settings, or you can control it by setting session.gc_maxlifetime.

First enable recycling management in the PHP configuration file.

In PHP, Session Garbage Collection (GC) is a mechanism for cleaning up expired session data. Depending on your environment requirements (such as development environment, high-traffic production server, etc.), you can optimize the configuration of session.gc_probability, session.gc_divisor and session.gc_maxlifetime.

The following are recommendations for optimal settings to enable recycling, depending on your scenario.

Key Parameter Optimization Explanation

session.gc_probability

Purpose: Controls the probability of triggering garbage collection each time a session is initialized.
Recommended settings:
Development environment: 1. It is convenient to frequently clean up session data to reduce debugging interference.
Production environment: 1. Make sure the garbage collection mechanism is turned on.

session.gc_divisor

Function: Together with session.gc_probability, it determines the probability of triggering garbage collection. The probability formula of garbage collection is:
Copy code
Garbage collection probability = gc_probability / gc_divisor
Recommended settings:
Development environment: 100 (i.e., 1/100 probability of triggering garbage collection).
High-traffic production environment: 1000 (i.e., a 1/1000 probability of triggering garbage collection, reducing performance overhead).

session.gc_maxlifetime

Purpose: Set the maximum survival time (in seconds) of session files. Sessions that are not used after this time will be considered "junk".
Recommended settings:
Development environment: Adjust as needed, usually set a shorter time (such as 1440 seconds = 24 minutes).
Production environment: Longer time, set up according to application requirements and user habits.
For example:
Ordinary e-commerce platform: 3600 (1 hour)
Enterprise backend system: 7200 (2 hours) or higher, such as 14400 (4 hours).

; Enable garbage collection mechanism. For production environment, it is recommended to set session.gc_probability = 1 session.gc_divisor = 1000 session.gc_maxlifetime = 14400 ; Set to 4 hours, which can be adjusted according to business needs

Alternatively, use a cron job or other tool to periodically clean up session data instead of relying on PHP's garbage collection mechanism.

编辑 crontab

sudo crontab -e

0 */6 * * * find /var/lib/lsphp/session/lsphp83 -type f -mmin +240 -delete  # 每 6 小时清理 4 小时前文件

解释：0 */6 * * * 表示每 6 小时执行一次；find 命令扫描目录，-type f 只限文件，-mmin +240 过滤 240 分钟（4 小时）前修改的文件，-delete 直接删除。调整 +240 为您的 session 有效期（PHP session.gc_maxlifetime 默认 1440 秒 = 24 分钟）。

保存后验证：

sudo crontab -l

The above command cleans up session files that were last modified more than 240 minutes ago.

PHP 垃圾回收机制：内置与外部结合

PHP 的 session 垃圾回收（GC）是文件级机制，默认 gc_probability=1（1% 几率触发），gc_divisor=100，gc_maxlifetime=1440 秒。但在高并发下，触发率低，文件堆积严重。

配置调整：编辑 php.ini（CyberPanel 面板 > PHP > Edit Configs）：

session.gc_probability = 1
session.gc_divisor = 1000
session.gc_maxlifetime = 14400
session.save_handler = files

重启 PHP-FPM：sudo systemctl restart lsphp83（替换版本）。这提升 GC 效率，但仍依赖随机性。
外部结合：Cron 作为补充，确保 100% 覆盖。自定义 handler 如 Redis（session.save_handler = redis）可完全取代文件存储，减少 I/O：安装 pecl install redis，php.ini session.save_handler = redis; session.save_path = "tcp://127.0.0.1:6379"。
监控：用 find /var/lib/lsphp/session/lsphp83 -type f | wc -l 统计文件数，cron 脚本中添加警报：if [ $(find ... | wc -l) -gt 10000 ]; then mail -s "Session Overflow" [email protected]; fi。

通过 cron 作业的定时清理和 PHP GC 机制的精细调优，您能构建一个高效、自动化的会话管理系统，避免文件堆积带来的性能隐患。在 CyberPanel 等环境中，这种双管齐下策略不仅简单可靠，还能将系统负载降 30%+，确保高并发下稳定运行。定期监控日志和文件数，结合负载测试（如 ab 工具），您的应用将更具弹性。优化会话管理，从现在开始——您的服务器值得更好的表现。

Okay, here comes the Ubuntu systemInsufficient hard disk spaceThe problem of not being able to write has been solved!

附：性能最优设置建议

针对高流量服务器，目标：平衡 GC 频率、减少 I/O、提升回收率 90%+。编辑 php.ini（CyberPanel 面板 > PHP > Edit Configs），重启 PHP-FPM。

基础调优（文件存储模式）：

session.gc_probability = 10  # 提升到 1% 触发率
session.gc_divisor = 100     # 实际率 = 10/100 = 10%，每 10 会话触发一次
session.gc_maxlifetime = 3600  # 1 小时过期，减少文件存活
session.save_handler = files

高级优化：切换外部存储（推荐高并发）： 用 Redis 取代文件，零 I/O 开销：

session.gc_probability = 0  # 禁用文件 GC
session.gc_divisor = 1000
session.gc_maxlifetime = 3600
session.save_handler = redis
session.save_path = "tcp://127.0.0.1:6379?auth=yourpassword"  # Redis 配置

先安装 Redis：sudo apt install redis-server php-redis，配置 /etc/redis/redis.conf（bind 127.0.0.1）。
效果：会话存内存/Redis，回收率 100%，性能提升 5 倍。

Migrating PHP Session Storage to Redis: A Configuration Guide from File System to High-Performance Cache