What is an XML Sitemap?

XML sitemap: a URL inventory for crawlers

An XML sitemap is a structured file—usually sitemap.xml or a sitemap index—that lists URLs you want search engines to know about. It is not a ranking factor by itself. It is a discovery and recrawl hint: "these addresses exist; here is when some of them last changed."

Sitemaps supplement internal linking and external references. They do not replace weak site architecture or fix blocking robots.txt rules.

Basic sitemap structure

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/glossary/xml-sitemap/</loc>
    <lastmod>2026-08-14</lastmod>
  </url>
  <url>
    <loc>https://example.com/products/widget/</loc>
    <lastmod>2026-08-10</lastmod>
  </url>
</urlset>

Core elements:

• loc — absolute URL of the page (canonical form)
• lastmod — optional ISO 8601 date of last meaningful update
• changefreq / priority — largely ignored by Google; do not obsess over them

Sitemap index for large sites

When URL counts exceed protocol limits, split into child sitemaps:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://example.com/sitemaps/products-1.xml</loc>
  </sitemap>
  <sitemap>
    <loc>https://example.com/sitemaps/blog-1.xml</loc>
  </sitemap>
</sitemapindex>

Submit only the index in Google Search Console. Keep child files updated when sections change.

What belongs in an XML sitemap

List canonical URL variants only—never duplicates you are consolidating away.

Example: post-migration sitemap

After CMS migration, marketing submits sitemap with new URLs but forgets to remove legacy paths that now 404. Search Console reports "Sitemap contains URLs which are not indexed" with errors. Fix: regenerate from database of live 200 URLs, not historical exports.

Example: accurate lastmod

CMS sets lastmod to daily cron time without content edits. Google learns to distrust the field. Fix: emit lastmod only when body, metadata, or stock status actually changes.

Submitting and referencing sitemaps

Discovery paths:

1. Search Console — submit sitemap URL per property
2. robots.txt — Sitemap: https://example.com/sitemap.xml
3. HTTP ping (legacy) — less relied upon today

Verify sitemaps return 200 with Content-Type: application/xml or text/xml, not HTML error pages disguised as success.

# robots.txt
User-agent: *
Allow: /

Sitemap: https://example.com/sitemap_index.xml

Specialized sitemap extensions

Image sitemaps

List image URLs associated with pages when image search visibility matters—galleries, stock media sites.

Video sitemaps

Provide title, description, thumbnail, and player loc for video content.

News sitemaps

For Google News publishers—strict freshness and formatting rules apply.

Hreflang in sitemaps

Declare language alternates with xhtml:link entries when HTML implementation is difficult across CMS templates.

XML sitemap vs HTML sitemap

Use both when catalogs are deep; neither fixes orphan issues without navigation updates.

Common XML sitemap mistakes

Example: staging sitemap leak

staging.example.com/sitemap.xml accidentally submitted while disallowing production. Validate environment-specific robots and Search Console properties separately.

Measuring sitemap effectiveness

Search Console sitemap reports show:

• Submitted vs indexed counts per sitemap
• Errors (invalid URL, fetch failures)
• Warnings (optional fields, soft issues)

Cross-check with crawl stats: indexed URLs not in sitemap may rely solely on links; sitemap URLs not indexed may be blocked, duplicate, or low quality.

XML sitemap generation strategies

• CMS plugins — convenient; audit output monthly
• Database exports — accurate for ecommerce catalogs
• Crawl-generated — good snapshot; not real-time alone
• CI/CD hooks — regenerate on publish events

Automate regeneration; manual quarterly exports miss launches.

XML sitemap myths

• Myth: "Sitemaps boost rankings." Reality: discovery aid only.
• Myth: "priority=1.0 on homepage is essential." Reality: Google ignores priority field.
• Myth: "If it's in the sitemap, it will be indexed." Reality: indexation still requires crawlability, quality, and no blocking directives.
• Myth: "One sitemap fits forever." Reality: shard and index as catalogs grow.

How Crawlox helps with XML sitemaps

Crawlox compares your live crawl results to submitted sitemap files, flagging URLs that are indexed-only-in-sitemap vs missing from sitemap despite being internally linked. Detect orphan and noindex URLs incorrectly listed, validate canonical alignment in loc entries, and export clean URL lists after technical SEO audits—so your next sitemap submission matches what crawlers should actually prioritize.