About Me
Testimonials
Site History
Sponsorship
Web Design
osCommerce
Icons & Graphics
Logo Design
SEO
Content Writing
Database Design
New Products
Search Store
Specials
TutorialsThe robots.txt file is not just a way to control what search engines can and can’t see on your web site. It has loads of other really importantfunctions too.
Check out this comprehensive article on how to work with robots.txt.
Contact DataMouse
Send me an email and I'll try and get back to you within 24-hours.
Client Login
Enter username and password to view your work
Posts Tagged ‘Crawl’
Working with Robots.txt
Oct 15thGooglebots and Leading the Blind: Dynamic Pages and SEO
Jun 15thDynamic pages are great for web designers – but poor for search engines.
The purpose of this article is to show you how you can index your dynamic pages with the search engines, and some of the pitfalls that you can avoid when submitting these to Google et al.
