Can anyone tell me the best way to be sure that the Yahoo crawler crawls all of the pages in a website. Is a url.txt file necessary? A sitemap.xml file?
Thanks
-- This message may have been cut off and the rest will only be shown to members. To become a member, click here --