AI Web Crawlers Are Spiking—What This Means for Your Site


Hey Reader,

Welcome to the Growth Drivers newsletter from MBE Group! If you’re new here, we’re glad to have you. And if you’ve been with us for a while, you’ll notice we’re evolving.

This newsletter now brings together a wider range of marketing insights including SEO, digital strategy, creative, PR, and more—delivered in one place to help you stay ahead.

If there is anything specific that you'd like to see from this newsletter going forward, just respond to this edition and let us know!


What's The Buzz?

Most marketers use AI in their workflow. But are they truly optimizing it?

Chase Freezman recently highlighted a common mistake: many people stick with the default ChatGPT model without considering the alternatives. Each model inside ChatGPT is designed for specific tasks, and switching strategically can save time and improve results.

For example, GPT-4o is great for content editing and tone refinement, while GPT-4.5 excels at long-form content and campaign narratives. If you're looking for speed over depth, models like o3-mini are ideal for data classification and quick reasoning.

Being intentional about which model you use can make a big difference in efficiency and performance.

--Nora


Marketing News You Need To Know

MARCH CORE UPDATE WRAP-UP – In our last edition, we covered the March 2025 Core Update as it was rolling out. Now, it’s officially complete, and the impact has been significant.

This update brought some of the biggest ranking volatility we’ve seen in the past year, shaking up search results across industries. One of the biggest takeaways? Google has recalibrated how it values forum content. While Reddit continues to thrive, many other forums that benefited from past updates are now seeing major drops. Additionally, sites relying on mass programmatic content have taken a hit, reinforcing Google's push for quality over quantity.

Now that the dust has settled, SEOs can assess the full impact and adjust their strategies. If you’ve noticed ranking drops, it may be time to re-evaluate content quality, reduce low-value pages, and focus on providing genuinely useful information. As always, staying adaptable is key.

What changes have you noticed since the update? Let’s discuss!

AI WEB CRAWLERS ARE SURGING – Website operators are reporting a sharp rise in AI web crawler activity, with bots from OpenAI, Anthropic, and others consuming significant bandwidth. This surge is causing performance issues, increased server loads, and potential distortions in analytics data. Some companies, like SourceHut, have already taken action by blocking cloud providers to curb excessive bot traffic. Data from Vercel highlights the scale of the issue, with GPTBot generating 569 million requests in a month; amounting to 20% of Google’s search crawler volume.

Beyond performance concerns, AI crawlers also disrupt analytics, inflating bot traffic metrics and affecting ad verification. Identifying and managing these crawlers is challenging, as they frequently change user agents and use residential proxies to evade detection. For site owners, balancing visibility with resource management is becoming a critical issue. Monitoring server logs, tracking bandwidth spikes, and reviewing Core Web Vitals can help determine if AI crawlers are impacting your site’s performance.

--Leilani


The Latest From MBE Group

It’s Q&A time! Nora and Heather tackle listener questions on building your content strategy, best resources for learning SEO, whether you should be optimizing your content for AI tools, and more!

If you're thinking about hiring a content agency, listen up. Let this episode serve as your guide to prepare working with a content agency so the engagement goes as smoothly as possible, and you get the best possible ROI.

SEO and social media are no longer separate entities; they’re complementary forces that, when strategically aligned, can dramatically increase your brand’s visibility and drive substantial traffic to your site. So what steps can you take to create an effective SEO social media strategy? Read our newest article to find out!


More Recent Posts From Our Blog

See how a well-executed email marketing campaign can play right into your SEO efforts, plus tips to maximize the value of email and SEO.

Social signals in SEO — likes, shares, and comments — may not have a direct effect on your rankings but can still boost your SEO performance.

Your biweekly dose of marketing news and advice to help you grow your audience, level up your content game, and get your site to the top results in Google.

Brought to you by MBE Group

mbe.group

Enjoying this newsletter? Consider sharing the sign-up link with a friend or colleague!

Follow us:

Unsubscribe

MBE Group

Read more from MBE Group

Hello! Welcome back to the Brand Drivers newsletter. In this week’s edition, we’re looking at why brand building is far more than just a “nice to have” for startups. We’re also digging into how Google’s AI Mode is quietly changing traffic patterns in Search Console, and why now is the time for marketing teams to step into a leadership role, not just a supporting one. Here’s what’s happening and why it matters. What's The Buzz? Brand Building Is Not a Luxury, It's a Growth Strategy Brand...

Hey Reader, Welcome back to the Brand Drivers newsletter! This week, we’re breaking down how people make buying decisions and why early visibility is your edge. Plus: what today’s economy means for your marketing strategy and how map apps are changing the way customers find you. Let’s take a closer look.... What's The Buzz? How People Actually Make Buying Decisions Buyers don’t move in straight lines anymore. They loop, ghost, binge content, and may even show up ready to buy without ever...

Hey Reader, Welcome back to the Growth Drivers newsletter! This week, we’re sharing three powerful insights to sharpen your marketing strategy. First, new data from over half a million LinkedIn posts challenges the advice to avoid links in your posts. Next, Chrome just rolled out a performance testing update that gives developers more accurate insights into how websites perform on real mobile devices. And finally, we’re looking at how most teams misuse website visitor data and how to make...