Internal Links within your website pages and External links – pointing traffic toward your website, helps Web crawlers discover pages on the internet. XML Sitemaps make this task easier for the web crawlers to pick up all URLs in the Sitemap and learn about those URLs using the associated metadata. Uploading a XML Sitemap to the root folder of your FTP does not guarantee that web pages are indexed in search engines, but it actually provides a hint for web crawlers to do a better job when crawling your site.
What is a XML sitemap?
AS per sitemaps.org Sitemaps are the easiest way for webmasters to inform search engines about pages on their sites that are available for crawling. In its simplest form, a Sitemap is an XML file that lists URLs for a site along with additional metadata about each URL (when it was last updated, how often it usually changes, and how important it is, relative to other URLs in the site) so that search engines can more intelligently crawl the site.
Why do you need a XML sitemap?
A sitemap is required if:
- You have a really large site, Google web crawlers may not find or overlook crawling some of your newly added or updated pages.
- Sites with large archives of content pages that are isolated or are not well linked to each other. And also pages on your site doesn’t naturally refer to each other, you have to list them in a sitemap to ensure that the search engines don’t overlook any of your important pages.
- The website is relatively new and has very few external links to it, then it might be difficult for Googlebot and other web crawlers to find your website. Because crawlers crawl the web by following links from one page to another. As a result, Search Engines might not be able to discover your pages if no other site links to them.
- The site has an excessive amount of media content (video, images) or it is an online news portal. If a sitemap is provided, Search Engines can take additional information into account for search, wherever it is appropriate.
A sitemap may NOT be required if:
- Your site is relatively ‘smaller’ or has less than 500 pages on your site.
- Internal links on your site are well routed. That means Google can find all the important pages following the links starting from the homepage.
- You don’t have too many videos or image files or your site is not a news page that you want to show in search results. Sitemaps usually help Google to find and understand rich media files, or news articles, on your site. If you don’t need images, videos, or news results, to appear in the search result then you may not need to have a sitemap.
Before we tell you about the benefits of having a sitemap, let us tell you the types of sitemaps.
Sitemap formats
Google supports several sitemap formats:
- XML
- RSS, mRSS, and Atom 1.0
- Text
Search Engines like Goole expects the standard sitemap protocol in all formats. Currently, Google does not overwhelm the attribute in sitemaps.
All formats are limited to a single sitemap of up to 50MB (when uncompressed) and a maximum of 50,000 URLs. If you have a larger file with more URLs, then you will have to break your sitemap into several sitemaps. You may optionally consider creating a sitemap index file (a file that points to a list of sitemaps) and submit that single index file to Google or Bing or Yahoo.
XML Sitemap
We are giving you an example of a very basic XML sitemap that includes the location of a single URL:
<?xml version=”1.0″ encoding=”UTF-8″?>
<urlset xmlns=”http://www.sitemaps.org/schemas/sitemap/0.9″>
<url>
<loc>http://www.example.com/foo.html</loc>
<lastmod>2018-06-04</lastmod>
</url>
</urlset>
For more complex examples and full documentation or you may consider visiting sitemaps.org.
You can find examples of sitemaps that specify alternate language pages and sitemaps for news, image, or video files.
RSS, mRSS, and Atom 1.0
If your blog is with an RSS or Atom feed, you are advised to submit the feed’s URL as a sitemap. Most blogging software can create a feed for you, but be sure that this feed provides information on recent URLs only.
- RSS 2.0 and Atom 1.0 feeds are accepted by Google.
- For video content on your site, you can use an mRSS (media RSS) feed to provide details to Google.
Text Sitemap
If you are willing to include only the web page URLs, you can provide Google a simple text file containing one URL per line. For example,
http://www.example.com/file1.html
http://www.example.com/file2.html
Text file sitemaps Guidelines –
- You should encode your file using UTF-8 encoding.
- Avoid putting anything else other than URLs in the sitemap file.
- You may name the text file anything, you wish, provided it has a .txt extension preferably sitemap.txt or sitemap_index.txt
Benefits of having a XML sitemap –
1. Organize Large Websites
As your business grows or you keep adding content to your site your website will also grow in size.
going forward you may add an eCommerce store with several departments or you may wish to expand your product portfolio. Or, there is a likelihood of adding new ideas which become a part of the company.
However, this can lead to confusion for visitors who are then confused about where to go or what you have to offer.
So, sitemap works in a similar way to a departmental store or shopping mall map for the search engines.
2. Serve As A Project Manager & Architect
You may think of the sitemap as an architectural blueprint of your website.
With time the sitemap becomes like a project management tool. It outlines the structure and connections between pages and subpages within the website.
It’s also a forcing function to make sure that you have a clear hierarchy and taxonomy for the site. Every page that is created has to find a home.
3. Sitemap Highlight The Website’s Purpose
As a content-based document, the HTML sitemap enhances this benefit by using SEO to identify the most unique and relevant keywords to include on the sitemap further define your website’s specific value.
4. Help Search Engine Crawlers To Speed up the Work
If you want to help and assist the search engines to find a way out to crawl your content and move it up in the crawl queue then you should consider creating an XML sitemap.
An XML sitemap is kind of a laundry list of links, whereas HTML links are the preferred ways for search crawlers to discover the web.
The HTML sitemap helps call attention to that content by putting the spotlight on your website’s most important pages. You can also submit the text version of your sitemap to Google.
5. Increase Search Engine Visibility
For some websites, search engines may not go through the work of indexing every webpage single page. For example, if you have posted a link on one of your web pages, the search bots may choose to follow that link.
Eventually, the bot may want to verify that link, if it makes any sense. Yet, in this process, the bots may never want to return to continue indexing the remaining pages on your website.
That is why having a well-decorated XML sitemap can direct these bots to get the entire picture of your site and consider all the pages for indexing that you have been allowed to index.
How do I create a sitemap?
You can review the structure of your pages and simply create a .txt file and list all of the website URLs in an unordered list and then save it as sitemap.txt. and upload it to the root of your website using FTP.
Online tools you can use to create an XML sitemap using like
and then download it to the computer and later upload it to the root folder of your website.
Creating an XML Sitemap in WordPress
You may install the Yoast SEO plugin on your WordPress website and follow the steps described in the following link.
to know more in detail click here.
Creating a Sitemap in Screaming Frog
Screaming Frog is a well-known desktop software that offers a wide range of SEO-related tools. It is free to use and generate a sitemap for your website as long as the website contains lesser than 500 pages. For larger websites, you’ll need to upgrade to the premium version of the software.
How to submit a sitemap to Google?
After you have uploaded your XML sitemap in the root folder of your site you need to go to the Google Search Console and then
Fixing common errors that affect your sitemap
- Not Submitting Sitemap at all
- Crawl Issues due to broken links
- Not Updating Sitemap as and when required
- Duplicate Content & Pages
- Indexing Utility Pages – which are of no use for other, or the pages that you don’t want to index.
- Sitemap Size – not more than 50MB in size and not more than 500 URLs in one Sitemap.
Final thoughts
If You’re Willing to take your SEO journey to the advanced level, you definitely need to create a sitemap for your website even if it is small or a new website with few pages.
There is no reason to be intimidated by this anymore. As you can see from this guide, it’s easy to create a sitemap in just a few steps.
- Review your pages
- Generate a TXT file or generate an XML sitemap using online tools
- Download the sitemap to your computer
- Add the sitemap to the root along with robots.txt
- Submit the sitemap to Google Search Console
That’s it, you are good to go!!
You may check out our article on How to write a privacy policy for your website.
FAQs
Q: I have submitted Sitemap, but still my URLs haven’t been crawled/indexed. Isn’t that what a Sitemap is for?
A: Submitting a Sitemap will help you make sure Search Engines know about all the URLs on your site. However, it is not guaranteed that those URLs will be crawled or indexed.
Q: If a site map doesn’t get my site automatically crawled and indexed, then why should I care about having one?
A: Sitemaps provide information to Google and other search engines to help understand your site in a better manner. This can include making sure that Google knows about all your URLs, how often they’re updated, and what is their relative importance.
Q: Will a Sitemap help me rank better?
A: A Sitemap does not directly affect the ranking of your pages. However, if it helps to get your site crawled (by notifying search engines about the URLs that they didn’t know about, and/or by helping search engines prioritize the URLs on your site), that may increase the presence and visibility of your site in the Search Engine index. Learn more.
Q: What If I set all of my pages priority to 1.0, will that help them rank higher or get crawled faster?
A: Nope. Priority only helps Google understand the importance of a particular URL relative to other URLs within your sitemap. And setting priority for your page doesn’t impact the ranking in search results. Mentioning that all of your pages have the same priority is like not providing any priority information at all within the sitemap.
Q: Is it necessary to put my sitemap at the root of my site?
A: Google has recently enabled Sitemap cross-submissions, which means you can put your Sitemap just about anywhere as long as you have the following sites verified in your Webmaster Tools account:
- For the site on which the Sitemap is located
- For the site(s) whose URLs are listed in the Sitemap
Note, cross-submissions only works with Google and may not work for search engines. Learn more about Sitemap cross-submissions.
Q: Which Sitemap format is the best?
A: The XML Sitemap protocol is recommended and defined by sitemaps.org. XML Sitemaps can be upgraded: you can simply start by listing your URLs, but—unlike a text file Sitemap —you can easily upgrade an XML Sitemap in future to include more metadata. Also, XML Sitemaps are also more comprehensive than RSS feed and Atom Sitemaps.
Q: Is it like the URLs at the beginning of the file get better attention and the URLs near the end don’t get as much importance? Does placement of a URL within a Sitemap file really matter?
A: No, not at all.