What's Shaping SEO: Robots.txt Tips, CTR Insights, and Smarter Content


The Growth Digest

Your biweekly dose of content marketing news and advice to help you grow your audience, level up your content game, and get your site to the top results in Google.

Hey Reader,

Welcome back to our newsletter!

In this edition, we’ll break down Google’s guidance on using robots.txt and noindex for optimal crawling and indexing, examine the latest shifts in click-through rates across industries, and explore how emotional context transforms search intent into more impactful content.

Let’s dive into these key insights to sharpen your strategy and keep your site competitive.


What's The Buzz?

When tackling search intent, you need to think beyond the surface. As Tommy Walker shared in a recent LinkedIn post, understanding the emotional context behind a search query can elevate your content from transactional to transformational. What was happening in the searcher’s life before they typed that phrase? Were they stressed from a deadline, excited about a big opportunity, or frustrated by an unresolved problem? Tapping into this emotional backdrop helps you craft content that feels personal and compelling.

Just like in the acting exercise described—where a knock on the door changes the emotional dynamic—your content should respond to the reader’s unspoken needs. This post is a must-read for anyone rethinking how to approach search intent.

--Heather


SEO News You Need To Know

UNDERSTANDING NOINDEX VS. DISALLOW -- Google has provided fresh guidance on when to use noindex tags versus robots.txt disallow directives to manage website crawling and indexing effectively. Martin Splitt, a Developer Advocate at Google, emphasized that while both tools influence how search engines interact with your site, they serve distinct purposes and shouldn’t be combined. The noindex tag is ideal for keeping a page out of search results while still allowing it to be crawled, such as thank-you pages or internal search results. In contrast, robots.txt disallow is best for completely blocking crawlers from accessing sensitive or irrelevant pages.

video preview

By following this guidance, site owners can better align their crawling and indexing strategies with their SEO goals.

GOOGLE SEARCH CTR TRENDS -- A Q3 report from Advanced Web Ranking reveals notable shifts in Google Search click-through rates (CTR) across industries, emphasizing the growing importance of mobile and informational queries. Mobile searches with terms like "what," "when," and "how" saw a 1.63-point CTR increase, while commercial queries declined across devices, dropping 3.51 points on mobile. Branded searches and short keyword queries also performed well, showcasing the need for concise, mobile-friendly content strategies to capture user attention.

Industry performance varied significantly. The Science sector rebounded with desktop and mobile CTRs increasing by 2.48 and 4.16 points, respectively, alongside a 33.78% jump in impressions. Shopping also saw recovery, while Arts & Entertainment experienced declines. These trends highlight the value of tracking CTR benchmarks and adapting strategies by industry, focusing on sectors with rising demand and CTRs, and addressing challenges in areas facing engagement drops.

--Nora


New On Our Blog

You don’t need to be a marketing expert to master lower-funnel marketing — it’s all-star content that makes converting a no-brainer for your readers. This guide breaks down proven strategies for crafting funnel content that turns heads.


More Recent Posts

Check out this guide to learn how to read SEO case studies and a few of Growth Machine’s most compelling case studies for our favorite marketing clients.

To stay competitive and visible, a well-rounded and dynamic SEO strategy is crucial. Here’s how you can apply top SEO tactics to stand out and get seen online.

Brought to you by Growth Machine

www.growthmachine.com

Enjoying this newsletter? Consider sharing the sign-up link with a friend or colleague!

Follow us:

Unsubscribe

Growth Machine

Read more from Growth Machine

The Growth Digest Your biweekly dose of content marketing news and advice to help you grow your audience, level up your content game, and get your site to the top results in Google. Hey Reader, For the first time in years, Google’s market share has dipped below 90%, raising questions about the future of search. Meanwhile, SEOs are buzzing about the latest core update and the debate over compression ratio as a ranking factor. We’re diving into what’s happening, what it means for you, and how...

The Growth Digest Your biweekly dose of content marketing news and advice to help you grow your audience, level up your content game, and get your site to the top results in Google. Hey Reader, SEO is constantly changing, and so is the way we approach marketing. This week, we’re looking at why AI Overviews don’t always align with organic search results, the long game of building customer trust, and whether adding an audio version of your blog is worth it. Here’s what you need to know. What's...

The Growth Digest Your biweekly dose of content marketing news and advice to help you grow your audience, level up your content game, and get your site to the top results in Google. Hey Reader, Search and marketing are shifting fast, and this week, we’re diving into three major trends that could reshape your strategy. Google is still the dominant search engine, but new data shows alternative platforms like ChatGPT and Perplexity are gaining ground. At the same time, interactive content is...