Getting listed in a search engine is the first step to getting your site optimised. If you have a brand new domain with no history or backlinks, it can take some time for the pages to get indexed. Below are a few ways to help search engines register your site or individual URLs quicker.
Search Engine Submission
To make a search engine aware of your content you can submit the URL to their database. This is not essential but it can help speed up the process. Most search engines add and update their index each time they crawl the web but they encourage submissions to help make them aware of new domains. To submit your site to Google simply visit www.google.com/addurl/
Creating a sitemap.xml document for your website can help make search engines aware of your content. The file is designed to help search engines register and understand your content as quickly as possible. The sitemap.xml document should be placed in the root folder of the domain and referenced in the robots.txt file. This file is recommended in the Google guidelines.
Webmaster Tools Submission
Both Bing and Google have simple webmaster tool systems where you can register your site. You can submit your sitemap.xml document to both Bing and Google to help them register the URLs. If you find your pages are still not getting indexed the systems will give you hints and warnings as to why this is. The system will flag up issues with site speed, broken links and duplicate content.
Creating a listing in Google Places or the Bing Business Portal can help speed up the indexing of your domain. The more information you can provide, the more trusting the search engine will be of your site. Give information on your address, hours of operation and telephone number. Google Map results are updated weekly giving your site a good chance of getting indexed.
Site structure can play a big part in getting your pages indexed. Make sure your site has a normal sitemap page which lists all your pages. This should be placed in the footer of every page so it is easily accessible to users and search engines. Specific pages that you want to get indexed should be as close to the Home page as possible. For example if you have specific location pages these should also be linked to from the footer.
Having a blog hosted on your site is an excellent way to get your pages indexed. Search engines value fresh content and a regularly updated blog will encourage the crawlers to visit the domain on a more frequent basis.
Dmoz is the largest and most comprehensive human-edited directory on the web. A quick listing in www.dmoz.org can help you get registered in all major search engines.
This is a relatively new tactic, but tweeting your URLs can help search engines pick up on new pages. If the tweets are retweeted or linked to from strong profiles this can aid your cause.
Social bookmarking can help you index any new pages on your domain. If you index the pages with accurate descriptions this will show search engines that the pages are popular. Sites like JumpTags, Reddit and Digg all help index fresh content.
Similar to Google places, if you set up a range of local profiles this will help your site get indexed in search engines. Setting up accurate profiles on sites like Yelp, Qype, WeLoveLocal and Brown Book will help show search engines you are a legitimate company.
With millions of start ups emerging each year it is becoming increasingly difficult for search engines to filter through ‘worthy’ content. If you have a brand new domain (or fresh content) the tactics above will help you get listed sooner rather than later.