Argument list too long?

To explain the problem, let’s consider what the shell does (we won’t get into system calls, to make things simple). The shell (Bourne compatible shells, actually) will first scan the line typed in. One of the first things to do is to expand file globs. For example, the command:

ls a*

will be translated by the shell into:

ls andrew apple alex allen alfred almonzo august axel albert

(and so forth). If the expansion expands beyond the system limitations, then it cannot be processed and the error “argument list too long” results.

The limit could be the number of arguments or the number of characters; however, the fix is the same. The key is to get the list (usually files) out of the argument list and into something else – such as stdout. Once the overlong list is being sent via stdout, the command xargs can be used to place all of the items on the command line, observing all relevant limits as it goes (and as efficient as possible to boot).

There are a variety of quick answers which will all fail, because the argument list would remain too long:

ls a* >savefile.txt
for i in a* ; echo $i ; done
echo a*

However, all is not lost: there are a number of ways to get a long list of files into stdout; the most common is to use find:

find . -name "CACHE.DAT" | xargs ls -ld

This may not be possible, if the desired list of files doesn’t fit neatly into a find command.

Another possibility, related to the previous example, would be this:

ls -1 | sed '/^a/!d' | xargs ls -ld

Yet another possibility might be to use a language like Perl; since it does not scan and process the same way, it would work without limitations:

perl -e 'opendir(DIR, "."); @all = grep /^a/, readdir DIR; closedir DIR; print "@alln";' | xargs ls -ld

I would only recommend using Perl or other such if you are quick and snappy with your knowledge of the language; otherwise, such a long line will have you looking things up repeatedly.

If the arguments are coming from a file, then things become even easier; instead of something like this:

ls -ld $(cat mylist)

You can simply use:

cat mylist | xargs ls -ld

Of course, any binary command can be used with xargs, not just ls.