Duplicate content, URLs and anchor text

How SiteAud!tLint finds exact and near-duplicate pages with MinHash, flags URL structure problems and reports non-descriptive anchor text.

Three checks look at how pages relate to each other rather than at a single page.

Exact and near-duplicate content

SiteAud!tLint takes the main text of each page, with menus, headers, footers and forms removed, and compares pages using MinHash. It reports exact duplicate content and near-duplicate pages that are about 85% or more similar.

Typical causes are filter and sort parameters, printer-friendly versions, product variants with near-identical descriptions, and location pages built from one template. Fix by merging pages, adding a canonical tag to the preferred URL, or writing unique content.

URL structure

The URL Structure view flags URLs with uppercase letters, underscores, query parameters, more than 115 characters, non-ASCII characters and double slashes. None of these stops a page ranking on its own, but they create duplicate URL variants and are harder to share. Change URLs only with 301 redirects from the old address.

Redirect loops and chains

Every redirect is followed to its end. A redirect loop is critical, because neither visitors nor crawlers reach a page. Chains show their hop count.

Anchor text

The Anchor Text view lists every internal link's text. Generic anchors such as "click here" and "read more", and links with no anchor text, give search engines and screen reader users no idea where the link goes. Use words that describe the target page.

Applies to SiteAud!tLint 0.3 and later. Current version 0.7.1.