The multilingual sitemap.
A multilingual sitemap is a site map in which every URL entry also declares all the language versions of that page.
The sitemap does two jobs: it makes the pages discoverable and carries extra information about each page. On multilingual sites the second job is especially valuable, because hreflang links can be declared at map level as well as page level.
Declaring in two places looks redundant but provides an assurance: a markup error at page level can be compensated by the map, and on large sites the map can be audited from a single place.
The structure
Every URL entry declares all the versions in the set as separate lines — itself included. On a trilingual site every entry thus carries three or four alternate lines.
That grows the file fast. On this site the alternate-line count comes out of a product. And not out of a single product: every URL existing in three languages gets four links each, every Turkish-only URL two each. As the translation coverage changes, the file changes with it.
The same reciprocity rule holds here too: if entry A declares B, entry B must declare A. When the map is generated from a single file, that is structurally guaranteed; in hand-written maps it is the first thing missed.
How many files
A separate sitemap per language, or everything in one file? Both are valid, and the choice depends on size.
- One file — small and mid-sized sites. Easy to audit; reciprocity is visible from one place.
- A file per language + an index — large sites. Each language managed separately, the index gathers them all.
- The size limit — a sitemap file cannot exceed fifty thousand URLs or fifty uncompressed megabytes.
- The index file — with more than one sitemap, an index file lists them all.
On this site a single file is used; the protocol’s fifty-thousand-address limit is more than enough at this scale. The file comes out of the generator; it is not edited by hand, and when a page is added its entry forms on its own.
Lastmod: the most misused field
Every entry can carry a last-modified date, and used correctly this field is a valuable signal. Used wrongly, it is ignored entirely.
The classic form of misuse: pulling every URL’s date to today on every release. Even when a single line of CSS changes, the engine hears that all pages were updated, and after a few rounds it stops trusting the signal.
The correct use is the date advancing only when that page’s content really changed. On this site the date is bound to a ledger: every page’s output is fingerprinted, and the stamp advances only when the fingerprint changes. Build time and page weight are excluded from the fingerprint — both would change on every run even with unchanged content, making the fingerprint lie.
A date pulled to today on every release turns, after a while, into a signal never read at all.
Auditing
The sitemap is a file that breaks silently: when a page is deleted its entry can remain, when a page is added its entry can be forgotten, and the hreflang links can end up one-way. All three feed the search engine wrong information.
- Does every URL exist on disk — an address in the map without a page promises a 404.
- Is every page in the map — a generated but unlisted page is left to chance discovery.
- Are the alternate links reciprocal — if A declares B, B must declare A.
- Does it include itself — every entry must list its own language too.
- Do the dates really show change — not pulled to today on every release?
- Does the robots file point to the map — the starting point of discovery.
The first two items are each other’s mirror image, and both need checking. On this site both are bound to a test: every URL in the sitemap must return a live HTTP 200, and the URL count must equal the number of generated pages on disk.
The second test was written after a defect was measured: the sitemap derived from INTENT — “does this collection exist?” — not from FACT — “was this page generated?”. The difference surfaced as three addresses being in the map but not on disk. It now derives from the list of generated pages.
Discovery and priority
The sitemap is a discovery channel but not a priority signal. Being in the map does not guarantee a page will be crawled; it only declares its existence. Priority is read from the internal link structure.
That yields an important consequence on multilingual sites: if a language version exists only in the map but no page links to it, that version is crawled rarely. The links in the language switcher are therefore needed not just for the user but for the crawler.
There is a second channel too: the map declaration in the robots file. It is the starting point of discovery, and without it the map is found only by manual submission. On this site the declaration is in place, and the verification suite measures it besides.
And a scale note: as the language count grows, the map grows fast. On a seven-language site every URL carries eight alternate lines; a thousand-page site means eight thousand lines. At that scale a map per language and an index file become practical.
A reminder to close: the sitemap is a declaration, not a guarantee. Being in the map does not show a page will be indexed; it only eases its discovery. For a page that does not get indexed, the fix is not fixing the map but fixing the content. When that distinction is confused, weeks are spent in the wrong place.
A final production note: the sitemap should not be written by hand. On a small site it looks feasible, but it goes stale at the first page addition, and a stale map is worse than no map — it shows the engine pages that do not exist.
On this site the map comes out of the generator and derives from the list of generated pages. If a page is not on disk it cannot enter the map; if a page was generated it is certainly in the map. The two-way guarantee is bound to a test.
A frame to close: on a multilingual site the sitemap does two jobs, and the second is usually skipped. Discovery is the job everyone knows; carrying the language links is the second line of defence for the page-level declaration. Building both together keeps a single error from breaking the whole set.