On Tue, 16 Nov 2010 01:50 -0600, "Mike Miller" <mbmiller+l at gmail.com> wrote: > > > Using these tricks I have reduced the processing time from 158 seconds to > 4.7 seconds. I'll write a script that does this for me. > > So the finding here that might be useful in many situations is that when > searching for a regexp in a big file, you might do much better to filter > lines of the big file with a simpler, more inclusive grep, then do the > regexp search on the stdout from the simple grep. > Mike, My experience is that Perl does text file processing much faster than grep/sed/awk and it saves programming time rather than having to utilize all the neat tricks you've done.