KAFD Sitemap Generation Using Python

Developed a custom Python-based sitemap generator for King Abdullah Financial District (KAFD) to improve crawl efficiency and index coverage. The tool automatically crawled all canonical, indexable English and Arabic URLs, filtered out parameters, non-HTML assets, and error responses, then deduplicated and organized valid URLs into clean XML sitemaps.
The script included dynamic lastmod updates based on file and HTTP timestamps, with optional changefreq and priority attributes. It also supported automatic sitemap indexing when URL count or file size limits were reached and allowed environment-safe paths for staging and production.
Deliverables included sitemap.xml or sitemap_index.xml, the reusable Python script, and a concise README with setup instructions for robots.txt inclusion and Google Search Console submission. The result was a faster, cleaner discovery process and optimized crawl budgeting for all KAFD web properties.