Search Engine uses a bot software which crawl the pages on web like Google uses GoogleBot, in actual bot is a software which collects documents and information from the web to build a index on which user can perform search easily.
So if you are an webmaster and wanna look why your web page is not getting good ranking then you can use a simple online tool Googlebot Spider, which allows you to look at the web page as a bot software do. Here you can not only see how Googlebot looks at page but it features other bot software’s too like Alexa, MSN, Yahoo and so on.
To use this tool visit the web site from below link and then just enter your webpage URL, make sure you add HTTP:// and trailing slash (/), otherwise it might not show webpage, then select the bot crawler and then submit it.
Within few seconds it will open your webpage, it will throw an 404 error on images which are using relative links because relative works as an negative point for you as per search engines. But don’t worry for 404 error as they will work on original page. But if you get 404 on any image then correct it.
At the bottom of page you will find HTTP header request send to bot and HTTP header request send to your browser, you can compare it and see what the difference and how bot is getting your page. It also shows the full source of you page which is get by bot, so can see what bot in crawling from your page and how.
If you are an developer of webmaster then you must try this tool as it allows you to find where you lacks in your web page s that you can increase your web page rank and traffic too.
Visit Webpage :- Googlebot Spider