If you are looking to launch a website to sell a product or service, you’re probably looking around the internet to see what you need to know or do to succeed. Eventually, the term SEO – and it’s importance – will crop up, along with the help guides telling you how to do it yourself. Doing it yourself is a lot cheaper than outsourcing to a company, so why should you bother?
Category Archives: General
What Is Robots.txt?
For search engines to keep their listings up to date, and present the most accurate search engine results, they perform an action known as a ‘crawl’. This is essentially sending a ‘bot’ (sometimes known as a ‘spider’) out to crawl the internet. The bot will then find new pages, updated pages or pages it did not previously know to exist. The end result of the crawl is that the search engine results page is updated, and all of the pages found on the last crawl are now included. It’s simply a method of finding sites on the internet.
