Error : Too many open files in linux…

Have you ever encountered an error of “Too many open files” while running your programs? I am sure many of you must have. But whats the cause of this error and how to identify the same?

Every OS has  a limit on open file descriptors that a process can have. Whenever that limit exceeds, your process starts encountering error “Too many open files”. File descriptor can be anything, that can be file pointer, that can be socket, pipes (named or unnamed), character or block devices. If we open a file, socket,pipe or device for read or write operation,  the count for file descriptor gets incremented. If we keep on opening descriptors without closing the same, the count reaches the threshold value and hence the error “Too many open files” gets generated. To avoid this error, one must accompany every open call with close call.

To check the number of file descriptors opened by a process currently, we can use following command:

lsof  -p <pid of process>

Eg

[root@linux bin]# lsof -p 22435 |wc -l
210
[root@linux bin]# lsof -p 22435 |wc -l
216

To view which descriptors are opened but not closed by process also, you can use lsof as follows. Check the value under NAME tag.

[root@ussdgw1 bin]# lsof -p 22415
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
mm 23415 root cwd DIR 8,9 4096 18661377 /home/test/mm
mm 23415 root rtd DIR 8,2 4096 2 /
mm 23415 root txt REG 8,9 176353 18661431 /home/test/mm
mm 23415 root mem REG 8,2 102447 1792852 /lib/ld-2.3.4.so
mm 23415 root mem REG 8,2 1454835 1792853 /lib/tls/libc-2.3.4.so
mm 23415 root mem REG 8,2 179139 1792856 /lib/tls/libm-2.3.4.so
mm 23415 root mem REG 8,2 94480 1792854 /lib/tls/libpthread-2.3.4.so
mm 23415 root mem REG 8,7 63528 477859 /usr/lib/libz.so.1.2.1.2
mm 23415 root mem REG 8,7 965536 476793 /usr/lib/libxml2.so.2.6.16
mm 23415 root 0u CHR 136,6 8 /dev/pts/6
mm 23415 root 1u CHR 136,6 8 /dev/pts/6
mm 23415 root 2u CHR 136,6 8 /dev/pts/6
mm 23415 root 3r REG 8,9 978 15089675 /home/mm/mm.cfg
mm 23415 root 4r REG 8,9 2215601 11534369 /home/mm/log/20120904-req.txt
mm 23415 root 5r REG 8,9 2908022 11534372 /home/mm/log/20120904-url-resp.txt
mm 23415 root 6r REG 8,9 2215601 11534369 /home/mm/log/20120904-req.txt
mm 23415 root 7r REG 8,9 2948526 11534370 /home/mm/log/20120904-resp.txt
mm 23415 root 8r REG 8,9 2215601 11534369 /home/mm/log/20120904-req.txt
mm 23415 root 9r REG 8,9 2908022 11534372 /home/mm/log/20120904-url-resp.txt
mm 23415 root 10r REG 8,9 990528 11534371 /home/mm/log/20120904-url-req.txt
mm 23415 root 11r REG 8,9 990528 11534371 /home/mm/log/20120904-url-req.txt
mm 23415 root 12r REG 8,9 2948526 11534370 /home/mm/log/20120904-resp.txt
mm 23415 root 13r REG 8,9 2948526 11534370 /home/mm/log/20120904-resp.txt

Now you have an idea which file descriptors are open, you can take the relevant action.

ADDITIONAL NOTE:

To check the maximum number of open file descriptors your system supports, run following command:

 [meenakshi@meenakshi ~]$ cat /proc/sys/fs/file-max100280

To check the maximum number of file descriptors a process on your system supports, run following command:

[meenakshi@meenakshi ~]$ ulimit -n
1024

To increase the limit of open file descriptors on system level, add following to  /etc/sysctl.conf

# Maximum number of open files permitted
fs.file-max = 65535

To load this new value without rebooting the systen run 

sysctl -p

To increase the number of open file descriptors a process can have, we have following two options:

1) To increase open fd's for current shell only , run following command:

ulimit -n 65535

2) To increase open fd's for a process, system wide and if you want the value to be reatined on reboot, set following values in /etc/security/limits.conf:

*                soft    nofile          65535
*                hard    nofile          65535