In this episode of SEO’s Getting Coffee, Emina Demiri-Watson and Sean Carroll went all in with JavaScript SEO with Patrick Hathaway, the co-founder and CEO of Sitebulb, a popular website auditing tool!
“Fundamentally the first and most important thing is understanding how much of the site is dependent upon JavaScript."
JavaScript introduces various possibilities for improving website functionality and user engagement and has evolved to become more integral to SEO strategies, especially with the advent of complex web applications that rely heavily on JavaScript for dynamic content rendering.
As an SEO expert, analyst and online marketer, Patrick offers his unique insights into how crawlers work and the challenges they face when dealing with JavaScript-dependent websites. From server-side and client-side rendering to the challenges of JavaScript in SEO, this episode is set to be a feast for all those tech SEO lovers.
Let’s dive in.
Watch Video
SEOs Getting Coffee - Episode #26: Javascript and SEO
The Evolution of Google’s Crawling and Rendering
The conversation kicks off with Emina’s question on how Google crawls and renders websites, particularly when it comes to JavaScript and how has this evolved throughout the years.
Patrick starts by discussing the evolution of Google’s crawling capabilities, emphasising the transition from simple HTML crawling to complex JavaScript rendering. The discussion highlights how JavaScript has become integral to modern web development, influencing the design and functionality of websites.
Patrick explains how, initially, the Sitebulb website crawler had an HTML crawling mode which functioned by analysing the raw HTML content served directly from the server. This mode is akin to viewing the source code of a webpage, where the crawler does not execute JavaScript or any client-side scripts.
This was a basic method used to understand the structure of a webpage by examining the static HTML.
While this mode is good for quick and surface-level insights, it does not reflect changes made by JavaScript which are increasingly common as websites become more dynamic.
The JavaScript crawling mode was an advancement added by Sitebulb to better simulate how search engines like Google understand and index modern websites that use a lot of client-side JavaScript.
This transition is crucial for SEO professionals to understand as it directly affects how content is indexed and served to users.
Challenges and Issues of JavaScript for SEO
The talk dives into specific challenges that arise from JavaScript-driven website elements and the evolution of SEO practices concerning the analysis of web page HTML
Patrick outlines various issues such as delayed indexing, the non-recognition of dynamic content by search engines, and the discrepancies between server-side and client-side rendering.
Many SEOs still rely on viewing the source HTML of a page as the definitive representation of what search engines see. Traditionally, SEOs might simply “view source” on a web page to analyse its HTML, which used to serve as a reliable snapshot of how search engines see a page. However, with the increasing use of JavaScript to dynamically generate web content, this method has become inadequate for a comprehensive understanding of a page’s SEO attributes.
Google’s crawlers now perform a more complex process where they initially scan the HTML (as delivered by the server) to determine basic indexing signals like the presence of ‘noindex’ tags and to gather straightforward hyperlinks. After this initial crawl, the page content is passed through a renderer which processes JavaScript, potentially altering or adding content and links that are crucial for accurate indexing.
This rendered version of the page reflects what users and, more importantly, what Google’s index sees, which could be significantly different from the initial HTML snapshot viewed through the source code. Therefore, modern SEO tools and practices have adapted to use both HTML and “Chrome” crawlers that mimic this rendering process to provide a true representation of how a page is interpreted by search engines. This dual-view approach is particularly vital for pages heavily reliant on JavaScript, where significant content and structure might only appear post-rendering.
As websites become more intricate and visually appealing, primarily driven by JavaScript, traditional SEO techniques that relied heavily on static HTML have become less effective. Patrick notes the challenge this presents in auditing and optimising websites because JavaScript can dynamically alter crucial SEO elements like meta tags and content visibility, which are not immediately apparent in the source HTML.
Collaboration Between SEOs and Developers
This shift demands a deeper understanding and collaboration between SEOs and developers to ensure that SEO considerations are integrated into the design and deployment of web elements.
In this segment of the discussion, Emina Demiri-Watson emphasises the crucial importance of integrating SEO considerations early in the web development process, particularly for small to medium-sized businesses. She points out a common pitfall where SEO professionals are brought into a project only after most of the development is completed. This often results in a “too late” scenario where SEO must work within constraints set by design and development choices that may not be optimized for search engines. Emina highlights the difficulties of retrofitting SEO into an already completed site, describing it as a push-and-pull struggle between desired functionality and SEO efficacy.
Patrick emphasises the necessity for SEOs to adapt and enhance their technical skills to accurately assess and communicate the SEO impacts of JavaScript-driven changes on websites.
Most Important Things SEOs Should Remember About JavaScript
In the podcast, Patrick delves into the critical considerations SEOs must keep in mind when dealing with JavaScript. He emphasises the need for a thorough understanding of how much a site relies on JavaScript, which determines the initial approach to its SEO. Patrick stresses the importance of the discovery phase, where identifying the extent of JavaScript usage helps in crafting a tailored SEO strategy.
This includes understanding whether elements like MetaRobots, Canonicals, or HREFlangs are dynamically altered by JavaScript, which could influence their indexing and ranking potential. He points out that issues such as delayed content rendering, JavaScript errors, or server-side rendering failures need careful attention, as they can lead to significant visibility problems.
This detailed approach not only aids in diagnosing current issues but also in anticipating potential future complications, ensuring a robust and resilient SEO framework.
The Role of Continuous Education for SEOs
In this segment of the podcast “SEOs Getting Coffee,” Sean Carroll and Patrick Hathaway discuss the importance of ongoing education for SEO professionals, particularly regarding the complexities of JavaScript in SEO. Patrick highlights the upcoming release of a Sitebulb study which delves into how well SEOs understand JavaScript and identifies significant knowledge gaps in the industry. This study aims to shed light on the proficiency levels within the SEO community and provide insights that can help bridge these gaps.
Patrick also mentions Sitebulb’s collaboration with Women in Tech SEO and their work with experts Sam Torres and Tori Gray from Cray.com. This partnership focuses on developing training and resources that enhance the ability of SEOs to effectively audit and optimize JavaScript-heavy sites. The initiative is part of a broader effort to improve communication between SEOs and developers, ensuring that SEO considerations are integrated early and effectively in the web development process.
The discussion underscores the need for SEOs to continually update their skills and knowledge to keep pace with technological advancements. Patrick’s mention of various resources, such as blog posts, webinars, and upcoming courses, emphasises Sitebulb’s commitment to supporting SEO professionals in enhancing their technical expertise, particularly in the challenging area of JavaScript SEO. This segment highlights the critical role of continuous learning and adaptation in the rapidly evolving field of SEO.
Room 404
During the conversation, Patrick expresses his specific frustration with over SEO-optimised recipe search results. He critiques the prevalent practice where recipe pages are filled with lengthy, narrative content that users must scroll through before reaching the actual recipe. This setup, often designed to improve search rankings by increasing page content, ends up detracting from user experience. Patrick is particularly annoyed by the unnecessary stories that precede the recipes, which he believes do not add value and only serve to frustrate users who are simply looking for cooking instructions.
Highlighting his displeasure, Patrick declares his wish to banish these over-extended recipe pages to “Room 404.” This act symbolizes his desire to eliminate practices in SEO that prioritize search engine algorithms over direct and useful user experience.
Final Thoughts
After hearing likely the best explanation ever on how Google crawls websites these days you are ready to take on the key takeaways from this episode.
It’s evident that SEO professionals must refine their technical skills to effectively navigate modern web technologies. Merely using ‘Inspect’ to view a webpage’s source code falls short in adequately handling JavaScript’s complexities. A crucial insight from the discussion is the imperative for web developers and SEO professionals to collaborate closely, ensuring websites are optimized both for functionality and search engine visibility.
Additionally, Patrick Hathaway emphasises the ongoing need for professional development within the SEO field. He highlights Sitebulb’s forthcoming studies and resources, specifically designed to bridge the knowledge gaps in JavaScript SEO, underscoring the importance of continuous learning and adaptation in the ever-evolving SEO landscape.
For more insights and in-depth conversations on the latest in SEO and digital marketing strategies, stay stuned for upcoming episodes of “SEOs Getting Coffee.” Subscribe to our channel for regular updates and expert opinions.
