Linux has the Python scripting language. This can make generating a sitemap very easy. I have tried using online tools, but these are limited in various ways. Either by a 500-page limit or other issues. The best way is via a command-line solution. The knyzorg/Sitemap-Generator-Crawler is the best solution I have found to generate a full sitemap for your website. I downloaded this script by checking out the code from Github.
git clone https://github.com/knyzorg/Sitemap-Generator-Crawler.git |
Then I had to install this package to allow it to work with my PHP installation on Ubuntu.
sudo apt-get install php-curl |
This got the script working on my system.
Then I could crawl my website with this command line.
php sitemap.php file=sitemap.xml site=https://securitronlinux.com |
This takes a long time if you have a lot of pages, but it is worth it due to the fact it actually generates a proper sitemap and works very well with no limitations.
This really works and does a great job of listing all pages on your site. Download this script today and improve your website for free. This generated a massive sitemap with all of my pages and posts in it. This also solved the issues I had yet to fix on Google Search Console. So this is very good for SEO.
Add your sitemap to your website in the robots.txt like this.
Sitemap: https://www.securitronlinux.com/sitemap.xml |
On a Mac, you may install PHP with Homebrew and then this script will run just fine. This is fine. I did not need to install php-curl or anything, the base PHP package for Homebrew was enough to run this on a Macbook running Macintosh OSX Monterey.
On Fedora and Alma Linux, I would need to install php-curl by running this command.
root@localhost /root# dnf in php-curl |
But this still works very well on modern Linux, this takes a while, but this is worth it.
In addition to PHP-CURL, I found that with Linux Mint 19.3 I had to first install the PHP7.2-CLI package. With both installed, the script worked successfully.
I was searching for a sitemap.xml script for a WordPress website that I just converted to static html. I needed to create a sitemap.xml for this website. This tool is excellent.
Thanks for sharing!
This is a gem. Thank you, man! I spent hours searching for a free online tool, they are all limited. Kudos.