How to Create a Sitemap in Laravel for a Website that Contains Thousands of Records.

In Laravel, if you want to build a sitemap, there are several packages built that can help you with smaller websites. One of them is the popular Spatie Sitemap package, which crawls your site when you ask it to and builds a sitemap for you based on the pages your frontend is linked to.

For smaller websites that only contain a few hundred pages, these packages will work great, and I recommend checking them out.

For a larger website with thousands of pages, you will not want to use them.

In my case, I was working on the Nursery People website a few months ago, and I wanted to build a sitemap for the site to help Google and other Search Engines see the pages on the site more easily. The Nursery People website has well over 30,000 pages, so I learned quickly that sitemap crawlers were not able to get the job done efficiently.

This is when I learned from a Laracast discussion that the best way to go about building a sitemap for a site with a lot of records is to build a sitemap that generates markup for various collections based on records in Alphabetical order.

In this article, I'll share how you can build a sitemap for a large website like I did for the Nursery People site.

Getting Started

First, you need to have a running Laravel application set up. For this tutorial, I created a new Laravel 8 application which you can fork on Github if you would like to follow along with the code, or you can just build this in your application.

For this example, I built a simple Model, Controller, and Migration for Posts. Of course, in your app, you will have more models, but for this example, I kept it simple.

I also used the Inertia Jetstream UI scaffolding so that you can see how this works with an updated Laravel 8 SPA. This isn't necessary, and you can build this just fine with the traditional Laravel UI.

I won't go into the details of creating the Post functionality or setting up Jetstream. You can read in other articles how to do that.

Creating a Sitemap Controller

Once your application is built, you will want to build routing for your Sitemaps.

In our app, we will generate two controllers for the website but for your app, you may want to generate several controllers.

For example, you may want to create a PostSitemapController for your Post model, and a TagSitemapController for Tags.

Generate the SitemapController

In your terminal:

php artisan make:controller SitemapController

Add the Index Route

In routes/web.php add a route for the main entry point of your sitemap.

//Sitemap Routes
Route::get('/sitemap.xml', [SitemapController::class, 'index'])->name('sitemap.index');

This will be the main sitemap that will contain links to the other sitemaps on the site.

We are also giving our sitemap a name so that we can easily route to it in our blade file.

Write the index() method in the SitemapController

Next, you will want to create the index method that we just referenced in the web.php route file.

For my app, the controller looks like this:

<?php

namespace App\Http\Controllers;

use App\Models\Post;
use Illuminate\Http\Request;

class SitemapController extends Controller
{
    public function index()
    {
        $post = Post::orderBy('updated_at', 'desc')->first();

        return response()->view('sitemap.index', [
            'post' => $post,
        ])->header('Content-Type', 'text/xml');
    }
}

As you may notice, we are getting the latest post in this method. I will share why in a moment.

Also, we return a blade view with a header of "Content-Type", "text/xml". This is how we tell out app that we are returning an XML file, not an HTML page.

Build the XML Blade View

Create a sitemap folder in your /resources/views directory and create a file named index.blade.php.

Inside this file, add the following. You can also see how I am using the $post that I got inside our index method here to show the last time our sitemap was updated.

<?php echo '<?xml version="1.0" encoding="UTF-8"?>'; ?>

<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    @if($post != null)
        <sitemap>
            <loc>{{ route('sitemap.posts.index') }}</loc>
            <lastmod>{{ $post->updated_at->tz('UTC')->toAtomString() }}</lastmod>
        </sitemap>
    @endif
</sitemapindex>

Since this is the entry point to your other sitemaps, you can add as many links to other sitemaps as you would like. These will be generated in the next step.
In my simple example, I am only creating one sitemap index for my posts.

Now, my sitemap looks like this when visiting the /sitemap.xml link:

Create a Post Sitemap Controller

In your terminal:

php artisan make:controller PostSitemapController

This is the sitemap controller I will use for posts.

Create a route for the posts sitemap

Add a second route to your routes/web.php file that will go to the sitemap that we just linked to in the blade file.

//Sitemap Routes
Route::get('/sitemap.xml', [SitemapController::class, 'index'])->name('sitemap.index');
Route::get('/sitemap/posts.xml', [PostSitemapController::class, 'index'])->name('sitemap.posts.index');

We added another route and name for easy linking.

Add the Index method to PostSitemapController

In this method, we will need to get a list of letters in the alphabet. That way we can build sitemaps for posts based on which letter of the alphabet they begin with. This isn't necessary for small collections, but in our case where we want to generate thousands of routes, this is a good way to separate content so that it is limited.

<?php

namespace App\Http\Controllers;

class PostSitemapController extends Controller
{
    public function index()
    {
        $alphas = range('a', 'z');

        return response()->view('sitemap.posts.index', [
            'alphas' => $alphas,
        ])->header('Content-Type', 'text/xml');
    }
}

We are returning the array of the alphabet to the blade view.

Create the post index blade view.

Now, in the /resources/views/sitemap directory create another folder for posts and add an index.blade.php file to it.

You will use the $alphas variable to loop through each character of the alphabet and generate a sitemap to posts starting with that character.

<?php echo '<?xml version="1.0" encoding="UTF-8"?>'; ?>

<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    @foreach($alphas as $alpha)
        <sitemap>
            <loc>{{ route('sitemap.posts.show', $alpha) }}</loc>
        </sitemap>
    @endforeach
</sitemapindex>

See that since we created a route name, it is easy to use the $alpha character as the route param.

Add the Final Route and Controller Method

Sweet! We are almost done. Now, add a route to the /sitemap/posts/{$letter}.xml in your routes/web.php file.

//Sitemap Routes
Route::get('/sitemap.xml', [SitemapController::class, 'index'])->name('sitemap.index');
Route::get('/sitemap/posts.xml', [PostSitemapController::class, 'index'])->name('sitemap.posts.index');
Route::get('/sitemap/posts/{letter}.xml', [PostSitemapController::class, 'show'])->name('sitemap.posts.show');

We are using the show() method here that will show posts based on the letter in the URL.

Post Sitemap Controller

<?php

namespace App\Http\Controllers;

use App\Models\Post;

class PostSitemapController extends Controller
{
    public function index()
    {
        $alphas = range('a', 'z');

        return response()->view('sitemap.posts.index', [
            'alphas' => $alphas,
        ])->header('Content-Type', 'text/xml');
    }

    public function show($letter){
        $posts = Post::where('title', 'LIKE', "$letter%")->get();

        return response()->view('sitemap.posts.show', [
            'posts' => $posts,
        ])->header('Content-Type', 'text/xml');
    }
}

Here, we are returning the $posts in that range to our view.

Create a show blade view.

Now, create the final sitemap needed for this app. In resources/views/sitemap/posts create a new file: show.blade.php.

<?php echo '<?xml version="1.0" encoding="UTF-8"?>'; ?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    @foreach ($posts as $post)
        <url>
            <loc>{{ route('posts.show', $post->id) }}</loc>
            <lastmod>{{ $post->updated_at->tz('UTC')->toAtomString() }}</lastmod>
            <changefreq>weekly</changefreq>
            <priority>0.6</priority>
        </url>
    @endforeach
</urlset>

On this page, we are looping through each of the post pages and generating the URL to the post page!

Conclusion

Well, that is how you generate a custom sitemap for a website with thousands of records. Make sure to build as many sitemaps/controllers as you need, and make sure that each one is linked to
from the main sitemap.

Hope you enjoy this and can help you as you are building your next website.

Extra!

Make sure to go and submit your sitemap in the Google Search Console as soon as you're done, and Google will then know it exists.