SEO Then and Now: Two Decades of Search Engine Optimization

AI OVERVIEW

SEO has evolved over two decades, from keywords and links to mobile, technical SEO, search intent, and AI-assisted search.

SEO History · 2006 → 2016 → 2026

SEO has changed dramatically in two decades. The websites are different, search engines are different, mobile changed how people browse the web, and AI is becoming part of how information is discovered and presented.

But something interesting happened along the way. Many of the fundamentals did not disappear.

Search systems still need to discover pages. Pages still need understandable content. Links still connect documents. Titles still describe pages. Headings still organize information. Broken links are still broken links.

What changed was the environment surrounding those fundamentals.

Two decades of change

2006 → 2016 → 2026

2006

Keywords + Links

SEO was heavily associated with keyword targeting, backlinks, directories, PageRank, basic on-page optimization and rankings.

→
2016

Mobile + Content + Technical

Mobile search, responsive websites, structured data, JavaScript, content quality and technical SEO became increasingly important.

→
2026

Intent + Technical + AI Search

SEO now operates across increasingly complex websites, search features and AI-assisted discovery experiences.

SEO in 2006: Keywords, Links and the Desktop Web

In 2006, SEO was much more closely associated with keywords, links and search engine rankings.

The web was overwhelmingly desktop-oriented compared with today. Mobile browsing existed, but smartphones had not yet transformed how people interacted with websites.

SEO discussions frequently revolved around questions such as:

  • How many times should a keyword appear?
  • Where should the keyword be placed?
  • How many backlinks does a page have?
  • How can a website acquire more links?
  • What does PageRank mean?
  • Should a website submit itself to directories?

The language of SEO was often much more mechanical. A page had a target keyword, and the SEO task was frequently described in terms of getting the right signals onto the page and obtaining enough external signals to compete.

The important distinction: SEO in 2006 was not simply "keyword stuffing." Practitioners were already thinking about crawlability, information architecture, relevance, links and how search engines interpreted websites.

The 2006 SEO Mindset

Step 01
Find a keyword
Identify a search term worth targeting.
Step 02
Optimize + build links
Place relevant terms on the page and work on external authority.
Step 03
Check rankings
Measure positions and repeat the process.

Some practices from this period eventually disappeared, while others became associated with manipulation and spam.

01

Keyword density

The idea that there was a specific percentage of keyword usage that made a page "optimized" became one of SEO's most persistent concepts.

02

Link quantity

Backlink counts received enormous attention, encouraging a variety of link acquisition and exchange tactics.

03

Directories

Web directories were much more prominent in the search ecosystem than they are today.

04

PageRank

Google's PageRank concept was a major part of how many people thought about link-based authority.

SEO in 2016: The Web Became More Complicated

Ten years later, SEO looked considerably different.

Websites were becoming more dynamic. JavaScript applications were increasingly common. Responsive design was becoming standard. Mobile search was growing rapidly. Structured data and rich search features were becoming increasingly important.

Google announced its mobile-first indexing experiments in November 2016, explaining that its systems had historically evaluated the desktop version of pages while moving toward using mobile content for indexing.

2016 marked an important transition: SEO could no longer be considered purely from a desktop perspective. The mobile web was becoming part of the core search environment.

The 2016 SEO Mindset

01
Research
Keywords, topics and the intent behind searches.
02
Build useful content
Create pages that address what people are actually looking for.
03
Technical SEO
Mobile, JavaScript, structured data, crawlability and performance.

The keyword had not disappeared. It was increasingly becoming part of a larger question:

What does the searcher actually want?

A page could contain an exact keyword and still be a poor result if it failed to satisfy the underlying intent.

2026: SEO Is No Longer Just About Rankings

Fast-forward another decade.

The web is now significantly more complex. A website may need to work across desktop, smartphones, search engines, image search, video search, local search, social discovery and increasingly AI-assisted search experiences.

That changes the questions SEO professionals ask.

01

Can search systems discover it?

Important pages need a crawlable path through the website and an accessible URL structure.

02

Can they understand it?

Content, headings, links, structured data and page structure all contribute to how information can be interpreted.

03

Does it satisfy intent?

A page needs to provide useful information rather than simply repeating a target phrase.

04

Does the website work?

Broken links, redirect problems, inaccessible pages and technical issues can undermine an otherwise useful site.

2006 vs 2016 vs 2026

Looking at the three periods side by side makes the evolution easier to see.

SEO Area Era 012006 Era 022016 Era 032026
Primary environment Desktop web Desktop + rapidly growing mobile Mobile, desktop and multiple discovery surfaces
Keyword strategy Exact keywords heavily emphasized Keywords + variations + search intent Intent, language, context and entities
Content Keyword-focused pages More comprehensive and intent-focused content Useful, original, structured and contextually relevant information
Links Quantity received major attention Quality and relevance increasingly emphasized Relevance, authority, discovery and context
Mobile SEO Secondary concern Major strategic concern Fundamental part of the web
Technical SEO Mostly basic technical checks Much broader and increasingly complex Large-scale crawling, rendering, indexing and technical accessibility
JavaScript Less central Growing technical concern Major consideration for modern sites
Structured data Not a mainstream SEO workflow Increasingly important Important for machine-readable information and eligible search features
SERP Mostly traditional results Rich snippets, images, local and other features Highly varied search experiences, including AI-assisted interfaces
SEO measurement Rankings + basic traffic measurement Analytics + Search Console + rankings Search, technical, content and site-level analysis
SEO auditing Manual checking was common Automated crawling increasingly practical Automated crawling and large-scale issue discovery
Meta keywords Commonly discussed Obsolete for Google Still obsolete
Authority metrics PageRank was widely discussed DA, PA and other third-party metrics gained popularity DA, DR and similar metrics remain third-party measurements, not Google ranking scores
Important: This table describes broad SEO trends across each period. SEO practices varied considerably by site, industry, search engine and practitioner.

What Happened to Keywords?

Keywords are probably the easiest example of how SEO evolved.

In 2006, keyword placement could receive enormous attention. SEO discussions frequently revolved around whether a keyword appeared in the title, URL, meta description, headings, body copy and image attributes.

That led to the infamous concept of keyword density.

The modern question is different: Instead of asking whether a keyword appears a specific number of times, ask whether the page clearly addresses the searcher's underlying need.

The keyword itself has not become meaningless. It can still help identify a subject and understand the language people use when searching. What changed is the role it plays within the larger context of a page.

What Happened to Link Building?

Links provide another example of how SEO evolved.

In earlier SEO, acquiring links could become a numbers game. That encouraged practices such as:

  • Directory submissions intended primarily to obtain links.
  • Reciprocal linking used as a systematic acquisition tactic.
  • Link exchanges designed around manipulating authority.
  • Paid links used to influence search visibility.
  • Link farms built around large volumes of interconnected links.
  • Article networks created primarily for links rather than readers.

Over time, search engines became better at detecting attempts to manipulate rankings through links.

The conversation increasingly shifted toward relevance, quality, editorial context and genuinely useful references.

Links did not disappear. The simplistic idea that a bigger backlink count automatically equals better SEO became much less useful.

The Death of Meta Keywords

If there is one SEO feature that provides a perfect example of something that simply disappeared, it is the meta keywords tag.

For years, website owners could place keywords into a dedicated meta keywords field. The element became heavily abused and was eventually abandoned by major search engines for ranking purposes.

Google has explicitly stated that it does not use the meta keywords tag for web ranking.

Not every SEO element survives forever.

Some elements become irrelevant. Some become associated with spam. Some are replaced by better approaches. Others remain useful but for different reasons than people originally expected.

Mobile Changed the Web

Perhaps the biggest structural change across these two decades was the move from desktop-first thinking toward mobile-first indexing and mobile-centered web experiences.

In 2006, a website could largely be designed around a desktop screen. By 2016, that assumption was becoming increasingly difficult to defend.

Google's mobile-first indexing experiments, announced in 2016, were part of this broader transition.

2006

Desktop-first thinking

Desktop screens were the dominant design assumption for many sites. Mobile browsing existed, but it was not yet the foundation of the modern web experience.

2016

Mobile becomes a major SEO concern

Responsive design, mobile usability and mobile search became major considerations. Search engines increasingly needed to understand the mobile version of websites.

2026

Mobile is part of the baseline

Mobile is no longer an optional SEO enhancement. It is part of the fundamental environment in which modern websites are accessed and evaluated.

Technical SEO Became Much Bigger

This may be one of the most important differences between 2006 and 2026.

Technical SEO did not suddenly appear in 2016. It simply became much more complicated as websites themselves became more complicated.

  • Thousands or millions of URLs can create large-scale crawling challenges.
  • JavaScript applications can change how content is rendered and discovered.
  • URL parameters can create multiple versions of similar content.
  • Canonicalization can become complicated on large sites.
  • Redirects can create chains, loops or unexpected destinations.
  • Structured data adds another machine-readable layer to pages.
  • Lazy-loaded resources can affect how content and media are delivered.
  • International websites introduce additional technical considerations.
  • Internal linking becomes harder to manage as websites grow.
  • Templates can create the same SEO problem across hundreds or thousands of pages.
A modern SEO problem is often a pattern, not a page. A single missing title is one issue. Ten thousand missing titles generated by a template are a website-level problem.

From Manual SEO to Crawling at Scale

Imagine auditing a website in 2006.

You might inspect pages manually. Check the title. Look at the source. Check links. Review content. Repeat.

For a small site, that can work.

For a website with thousands or hundreds of thousands of URLs, it becomes impractical.

Modern SEO crawlers changed the workflow by making it possible to ask a much larger question:

What's wrong across the entire website?
01

Find broken links

Identify internal links pointing to unavailable or incorrect URLs.

02

Find missing metadata

Detect pages with missing or problematic titles and descriptions.

03

Find redirect problems

Surface redirects that create unnecessary or problematic paths.

04

Find structural problems

Examine headings, internal links, URLs and other page-level patterns across the site.

What Didn't Change?

This may be the most interesting part of the entire comparison.

Despite everything that changed, some SEO fundamentals survived.

01

Search engines still need discovery

A page still needs an accessible path for search systems to discover and process it.

02

Content still needs meaning

A page should communicate clearly what it is about and why it exists.

03

Links still connect the web

Internal links connect related pages while external links connect websites and information.

04

Broken links are still broken

A broken URL was a problem in 2006 and remains a problem in 2026. Modern sites simply create more opportunities for them to occur.

Titles Still Matter

The technology around search changed, but telling users and search systems what a page is about remains important.

Headings Still Organize Information

H1 through H6 are not magical ranking switches. They are part of a page's content structure and help organize information for readers and systems interpreting the document.

Internal Links Still Matter

Internal links continue to connect pages, establish relationships and provide pathways through a website.

SEO Myths That Survived Two Decades

Myth

There is a perfect keyword density.
There is no universal percentage that makes a page optimized.

Myth

More backlinks automatically means better rankings.
A raw backlink count does not describe relevance, quality or context.

Myth

DA is a Google ranking score.
Domain Authority is a third-party metric, not a Google ranking metric.

Myth

Meta descriptions are a direct ranking switch.
They can influence search-result presentation and user behavior but should not be treated as a simple ranking control.

Myth

H1 is a ranking button.
Headings primarily provide structure and communicate the organization of the page.

Myth

SEO is just about rankings.
SEO also involves discoverability, crawlability, accessibility, structure, content and useful search visibility.

The Biggest Change May Not Be an Algorithm

The biggest change in SEO over two decades may actually be the complexity of the web itself.

In 2006, many websites were comparatively simple.

Today, a single website can contain:

HTML + CSS + JavaScript + APIs + images + video + structured data + multiple templates + dynamic content + thousands of URLs.

That creates more opportunities for things to go wrong.

SEO therefore moved from:

Optimize a page.

toward:

Understand the website.

And increasingly:

Understand how the entire website behaves.

SEO Then and Now: The Workflow Changed

2006

Keywords

Target search terms and optimize pages.

2006

Links

Build external authority and monitor rankings.

2016

Content

Address topics and search intent more comprehensively.

2016+

Technical

Mobile, crawling, rendering and structured data.

2026

Website-wide

Discover, audit, fix, compare and improve at scale.

Two Decades of SEO in One View

Question 2006 2016 2026
What keyword should I target? Very important Still important Part of broader intent and context
How often should I use it? Frequently discussed Less central No universal density target
How many links do I have? Major focus Quality increasingly emphasized Context and relevance matter
Is the site mobile-friendly? Often secondary Major concern Baseline expectation
Can search systems crawl it? Important Very important Fundamental
Can search systems render it? Less prominent Growing concern Major technical consideration
Is the content useful? Important Increasingly central Fundamental
Are pages internally linked? Important Important Still important
Are URLs broken? Problem Problem Still a problem at potentially much larger scale
Are titles descriptive? Yes Yes Yes
Are headings structured? Yes Yes Yes
Can the whole site be audited automatically? Much more limited Increasingly practical Core workflow for large sites

What Two Decades of SEO Actually Teach Us

The history of SEO is not simply a story about keywords becoming irrelevant.

It is not simply a story about links becoming irrelevant.

It is not even a story about search engines constantly changing the rules.

It is a story about search becoming better at understanding the web while the web itself became dramatically more complicated.

In 2006, SEO often focused on individual signals.

In 2016, those signals increasingly existed within a larger ecosystem of mobile, content, technical SEO and search intent.

In 2026, SEO operates across complex websites, diverse search experiences and increasingly sophisticated systems for interpreting information.

But the foundation remains surprisingly familiar. A website still needs to be discoverable, crawlable, understandable, accessible, well structured, connected and useful.

The Part That Hasn't Changed

After two decades, perhaps the simplest definition of SEO is still one of the most useful:

Help search systems discover, understand and surface useful information for people.

The tools used to do that have changed.

The search interfaces have changed.

The devices have changed.

The algorithms have changed.

The terminology has changed.

But the underlying job remains remarkably recognizable.

From Crawl to Improvement

Modern technical SEO brings the two decades together.

A crawler can tell you that a page returns a 404. It can identify a missing title, find broken internal links, surface redirect problems and reveal pages that are difficult to discover.

But finding an issue is only the beginning.

Crawl. Find. Fix. Compare. Improve.

Crawl the website to understand what is there.

Find the problems and patterns affecting the site.

Fix the issues that need attention.

Compare the site before and after the changes.

Improve continuously instead of treating an SEO audit as a one-time event.

Two Decades Later

SEO did not die.

It grew up.

The industry moved from a relatively narrow focus on keywords and links into a much broader discipline covering content, information architecture, mobile, technical accessibility, structured data, performance, search features and increasingly complex ways of discovering information.

Some practices disappeared.

Some evolved.

Some myths survived.

And some fundamentals never left.

The lesson from two decades of SEO: Technology changes faster than the fundamentals. Sometimes, the SEO problems that matter most are still the simplest ones.