Slash the Duplicate URLs

It’s not even April and I’ve made another foolish mistake on this site. Call it my own version of “March Madness” and quasi duplicate URLs. It’s an error of my own making and I have no one to blame. Chances are it may be one that you have too. Fortunately, I’ve discovered this one early so it will be easier to fix.

The funny thing is this story started out about how as webmasters we need to check our redirects and systems. I spotted some links on a Microsoft page that were problematic and was about to do a write up. However, in the process of doing that research, I stumbled on my mistake.

The issue I discovered is that I’ve referenced the same stories with different URLs. One has a trailing slash and the other does not. You can see the screen snap from one of my Screaming Frog scans.

Screaming Frog results showing similar URLs

MULTIPLE URLS TO SAME CONTENT

The URLs ending with the slash (/) are done through my CMS setup. The ones without the slash were done by me. When I referenced an article on another page or the comments, I failed to add the slash. It’s not that a slash is required; it’s just that you want consistency throughout your site.

Embedded link with no slash

LINK TO URL WITHOUT SLASH

If you spot this type of problem on your site, you’ll want to check to see which server responses are returned. If each URL responds with a 200 server code, then you run the risk of having both pages indexed by the search engines.

You can use a free tool like URI Valet to check your server headers by pasting each URL into the URI textbox and clicking Submit. In the example below, you can see where my non-slash URL, does a permanent redirect (SERVER RESPONSE 301) to the URL with the trailing slash.

Results from URI Valet showing redirect

REDIRECT & SERVER RESPONSE CODES

Fixing the Problem

Just as there are many ways to create this problem, there are multiple solutions. If you use a content management system (CMS), there is probably a links toolbar button within your editor. Within that module, you may also see references to your site articles. In the screen snap below, I can link to an existing article by clicking the title. This method ensures that the same URL is being used when referenced. For example, it will pick the URL that has the trailing slash.

Example of using CMS to link existing articles

LINKING VIA PAGE TITLE

If you have access to your .HTACCESS file, you could fix the issue at the server. If you’re not familiar with this file, I’d suggest you turn in a support ticket with your webhost and explain what you’d like to do. If you are comfortable with this file, you might try reading this write up on creating a rewrite rule.

The last option is to delegate the task so you can do more productive stuff. There are a number of tools that can give you a URL list. The key is finding which URL(s) have the offending outbound link. My preference is Screaming Frog as it’s so versatile and cross-platform. However, there are other options like Xenu Link Sleuth, which just runs on Windows.

In my case, the URLs without the slashes do redirect to the URL with the slash. All the same, I still want to go back and fix the problem because redirects can slow users from finding the content.