# MOMspider System Sites File: Lists IPaddress:port locations we've checked # for a /robots.txt file and followed its directions. # New sites can be added when the program is not running. # The file format is: EntryType IPaddress:Port [ExpireDate] # where EntryType = "Site" # IPaddress = the full hostname or IP address for the site # Port = the numeric TCP port for the site (write 80 for default) # ExpireDate = [*] for never expire (i.e. never check this site) # or [date] (see wwwdates.pl for valid date formats)