Crawling is the first gait learned by most human beings, and no other word could have been used in digital strategies to define the first essential steps aimed at creating value in marketing campaigns, specifically when search engine optimisation (SEO) is involved.
Toddlers
It generally brings to mind the world of toddlers, but it is arguably adults’ stuff in the SEO world.
It’s not only for experienced folks — everybody in the industry knows that crawling plays a vital part in what is commonly known as “Technical SEO” — but not many so-called SEO experts can read the signs in the stars when the crawling is actually over.
In fact, I’ve personally dealt with a slew of experts who: a) have never performed a crawling (it’s not difficult, you just have to press “run” on the software, most of the times), and; b) do not stand a chance to understand key outputs or any other side data.
Here is why it’s important, though.
Crawling Tools
Crawling programs commonly used are Screaming Frog and Xenu. How do they work?
A robot (“bot”) browses a number of urls and creates an index of data based on elements of the website’s pages. It shows you your website just as any bot of any search engines (Google, Yahoo, Bing) would read it.
Of course, the crawling process is part of several other factors and actions that a full-fledged analysis based on SEO takes into account, but it should certainly be the starting point.
Anybody who does not consider crawling as a vital part of the learning process, perhaps looking only at a few pages of a website that should be actually crawled, he forgoes the opportunity to understand the basic issues of a website — all of which, quite simply, could hinder performance.
Those who do not follow closely these guidelines — which we have made ours — tend to cost a lot more, justifying their approach on a superior understanding of the problem. Most of times, however, as part of a comprehensive SEO strategy, crawling not only is more efficient, but also saves money to the clients.
We reiterate the view that SEO is a methodical, almost scientific approach that requires experience, commitment and discipline.
Three key points:
- Crawling is important not only for SEO, but also for other marketing channels — such as PPC — that aim to provide a better web experience to the end user (detecting 404 error pages and site architecture issues, for example).
- SEO audits should be based upon crawling, but the interpretation of the data and execution are equally important. Experience plays a very important role.
- Crawling is not a one-off. Rather, it should be part of an ongoing strategy aimed at optimising the output.
If you want to discuss this topic with our team, please email us at info@hedgingbeta.com