that_guy84 Posted August 15, 2011 Share Posted August 15, 2011 http://xxxxxxxxxxxxxxxxxxxxx.com/me 404 (Not found) 10 pages Aug 11, 2011 http://xxxxxxxxxxxxxxx.com/picture 404 (Not found) 10 pages Aug 11, 2011 I got this URL as missing in my crawler error log. I can't find them yet all my products and catagorys are point to them at some point. How do i find out why and fix it? Link to comment Share on other sites More sharing options...
Mike Kranzler Posted August 15, 2011 Share Posted August 15, 2011 http://xxxxxxxxxxxxxxxxxxxxx.com/me 404 (Not found) 10 pages Aug 11, 2011 http://xxxxxxxxxxxxxxx.com/picture 404 (Not found) 10 pages Aug 11, 2011 I got this URL as missing in my crawler error log. I can't find them yet all my products and catagorys are point to them at some point. How do i find out why and fix it? Hi that_guy, Have you tried regenerating your htaccess file yet? -Mike Link to comment Share on other sites More sharing options...
that_guy84 Posted August 15, 2011 Author Share Posted August 15, 2011 No, Have just done that now, will see if that makes a change to the crawler errors. Thanks Link to comment Share on other sites More sharing options...
Mike Kranzler Posted August 15, 2011 Share Posted August 15, 2011 No, Have just done that now, will see if that makes a change to the crawler errors. Thanks Let us know if that works! -Mike Link to comment Share on other sites More sharing options...
that_guy84 Posted August 16, 2011 Author Share Posted August 16, 2011 It didn't work, Infact now it's worse. Think this should maybe move to the SEO board. /me and /pictures are still on the list. http://myurl.com/99-hook-mustad-7731.html 503 error unavailable 8/14/11 http://myurl.com/85-mustad-hoodlum-10827bln.html 503 error unavailable 8/14/11 http://myurl.com/24-stick-baits 503 error unavailable 8/14/11 http://myurl.com/50-mustad-circle-39951npbln.html 503 error unavailable 8/14/11 http://myurl.com/6_mustad?p=2 503 error unavailable 8/14/11 http://myurl.com/content/2-legal-notice 503 error unavailable 8/14/11 http://myurl.com/23-s-line-757-cs.html 503 error unavailable 8/14/11 http://myurl.com/32-mustad-hoodlum-10827bln.html 503 error unavailable 8/14/11 http://myurl.com/search?tag=SK4SR030 URL restricted by robots.txt unavailable 8/14/11 http://myurl.com/discount URL restricted by robots.txt unavailable 8/13/11 http://myurl.com/search?tag=SK3SR020 URL restricted by robots.txt unavailable 8/13/11 http://myurl.com/search?tag=SONIK URL restricted by robots.txt unavailable 8/13/11 http://myurl.com/search?tag=AKIOS URL restricted by robots.txt unavailable 8/13/11 http://myurl.com/search?tag=S-Line+757+CS URL restricted by robots.txt unavailable 8/13/11 http://myurl.com/search?tag=MUSTAD URL restricted by robots.txt unavailable 8/13/11 http://myurl.com/search?tag=SK4SR010 URL restricted by robots.txt unavailable 8/12/2011 http://myurl.com/attachment.php?id_attachment=6 URL restricted by robots.txt unavailable 8/12/2011 http://myurl.com/my-account URL restricted by robots.txt unavailable 8/12/2011 http://myurl.com/search?tag=S-Line+656+CS URL restricted by robots.txt unavailable 8/12/2011 http://myurl.com/attachment.php?id_attachment=4 URL restricted by robots.txt unavailable 8/12/2011 http://myurl.com/search?tag=S-Line+757CT URL restricted by robots.txt unavailable 8/12/2011 http://myurl.com/attachment.php?id_attachment=5 URL restricted by robots.txt unavailable 8/11/2011 http://myurl.com/51-mustad-circle-39951npbln.html 404 (Not found) 1 pages 8/13/11 http://myurl.com/7-surf-casters 404 (Not found) 10 pages 8/13/11 http://myurl.com/16-hooks 404 (Not found) 2 pages 8/12/2011 http://myurl.com/me 404 (Not found) 10 pages 8/11/2011 http://myurl.com/picture 404 (Not found) 10 pages 8/11/2011 Link to comment Share on other sites More sharing options...
Mike Kranzler Posted August 16, 2011 Share Posted August 16, 2011 Hi that_guy, I will move this to the SEO board for you now. Hopefully someone there will be able to help you fix this pretty quickly! -Mike Link to comment Share on other sites More sharing options...
that_guy84 Posted August 16, 2011 Author Share Posted August 16, 2011 Thanks mike Link to comment Share on other sites More sharing options...
Mike Kranzler Posted August 16, 2011 Share Posted August 16, 2011 Thanks mike It's my pleasure! I hope someone more knowledgeable than I can help you out! -Mike Link to comment Share on other sites More sharing options...
that_guy84 Posted August 16, 2011 Author Share Posted August 16, 2011 I THINK i have it figured out, I'll have to edit my sitemaps.xml There are even more 503/404's now. They are all URLS that belong to catagories I have not made active yet. I'll remove them from the site map and see how the next crawl goes. Link to comment Share on other sites More sharing options...
Mike Kranzler Posted August 17, 2011 Share Posted August 17, 2011 I THINK i have it figured out, I'll have to edit my sitemaps.xml There are even more 503/404's now. They are all URLS that belong to catagories I have not made active yet. I'll remove them from the site map and see how the next crawl goes. Let us know if that works. If it does, I'll mark it as solved so that other people having this issue can find the solution. You don't want to be "that guy" who never came back after solving his own problem, do you? -Mike Link to comment Share on other sites More sharing options...
that_guy84 Posted August 17, 2011 Author Share Posted August 17, 2011 I ended up doing something else, We'll find out later if it worked. Link to comment Share on other sites More sharing options...
that_guy84 Posted August 18, 2011 Author Share Posted August 18, 2011 Nope it didn't work, I regenerated the robots.txt, site map, and htaccess. 503 & 404 errors tripled on the last crawl. Link to comment Share on other sites More sharing options...
Mike Kranzler Posted August 18, 2011 Share Posted August 18, 2011 Hi that_guy, I'm assuming your website isn't www.xxxxxxxxx.com. Can you please post your URL so that we can check it out for you? -Mike Link to comment Share on other sites More sharing options...
that_guy84 Posted August 18, 2011 Author Share Posted August 18, 2011 www.lbg-supplies.co.nz Link to comment Share on other sites More sharing options...
that_guy84 Posted August 21, 2011 Author Share Posted August 21, 2011 Another update All the missing URL's not found are items that I added then deleted, I have since deleted the site map and generated a new sitemap how ever the URL's are still showing up. Will try manaually editing the site map today. Link to comment Share on other sites More sharing options...
that_guy84 Posted August 22, 2011 Author Share Posted August 22, 2011 The URL's that are getting 505/404 errors are not in the sitemaps.xml and have all been created and then deleted at some point. I don't know why they're still being crawled but they're not there anymore. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now