No space left on device – Could I be running out of Inodes?

By paulg | May 5, 2017 | Category Linux Solutions

Logged onto my dev servers today, and I immediately started having problems started with bash scripts – The server that claimed that claimed “No space left on device”, although partition was not nearly full.

Not sure what to do when running into this trouble i search for a solution. Since df -h showed I had space left I had to look into the next most likely issue, you have exhausted all available Inodes (ie – too many small or zero-sized files on disk). That was the problem and this is how to figure it out and fix it.

First check available disk space to ensure that you still have some

$ df -h

Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/xvda             33030016  10407780  22622236  32% /
tmpfs                   368748         0    368748   0% /lib/init/rw
varrun                  368748        56    368692   1% /var/run
varlock                 368748         0    368748   0% /var/lock
udev                    368748       108    368640   1% /dev
tmpfs                   368748         0    368748   0% /dev/shm

Second check available Inodes

$ df -i

Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/xvda            2080768 2080768       0  100% /
tmpfs                  92187       3   92184    1% /lib/init/rw
varrun                 92187      38   92149    1% /var/run
varlock                92187       4   92183    1% /var/lock
udev                   92187    4404   87783    5% /dev
tmpfs                  92187       1   92186    1% /dev/shm

Thirdly, you need to know how to find all those little files. Start with this command will list directories and number of files in them.

$ for i in /*; do echo $i; find $i |wc -l; done

Once you see a directory with unusually high number of files  repeat the command for that directory to see where exactly the small files are.

$ for i in /putDirHere/*; do echo $i; find $i |wc -l; done

Now that you have zeroed in on the suspected files just delete them.

$ sudo rm -rf /home/bad_user/directory_with_lots_of_empty_files

Your problem is now solved. Check the results with df -i command again. You should see something like this:

Filesystem            Inodes   IUsed   IFree IUse% Mounted on

/dev/xvda              253704    369  253335    1% /dev
tmpfs                  92187       3   92184    1% /lib/init/rw
varrun                 92187      38   92149    1% /var/run
varlock                92187       4   92183    1% /var/lock
tmpfs                  92187       1   92186    1% /dev/shm

Hope this helped you out!!