Quoting duncan (duncan at sodatrain.com): > hello- > > Does anyone have any experince with the htdig, mnogo or other search > engine software? Both. Mnogo is better IMHO, htdig is easier to setup. > From what i read, i should be able to config one of them to search for > certain file types... > > I want to spider a list of sites and grab URL's to .iso files and then > crunch them and make a resource like linuxiso.org Mnogo would be better at this and not a problems. Server http://www.mycompany.com/foo/bar Allow Case *.iso -- Minneapolis St. Paul Twin Cities MN | Phone : (952)943-8700 http://www.mn-linux.org Minnesota Linux | Fax : (952)943-8500 Key fingerprint = 6C E9 51 4F D5 3E 4C 66 62 A9 10 E5 35 85 39 D9