Pssst! 16 Sneaky Snooping Tools for Spying on Competitors

Take advantage of these free online tools and dig up data on on business competitors. Set up a digital stakeout and discover what your rivals are doing online. 

There’s tonnes of opportunities to gather intelligence and it’s never been easier to take notes on what the other guys are up to.

Although privacy may have gone out of the window years ago, you must admit it is fascinating to have facts, reveal data and get the drop.

1) Google

Yep. Google is the all seeing eye that rarely forgets. Indexed search engine content about a person, their business, their operations and their actions is always going to be useful if it affects your business.

Competitive analysis using a search engine is one of the easiest steps you can take to feel out what other businesses are doing.

If, for example, your local pizza takeaway gives everyone food poisoning and earns a nil hygiene rating, that would be worth knowing about if you also run a pizza shop.

Worth mentioning is the EU’s right to be forgotten ruling that affects some search engine indexing.

According to this Wikipedia entry about the right to be forgotten, the issue has arisen from desires of individuals to “determine the development of their life in an autonomous way, without being perpetually or periodically stigmatized as a consequence of a specific action performed in the past.”

Between May 2014 and May 2015 only 40 percent of removal requests have been approved by Google.

Google Search Operators

There are a few ways to use Google beyond the basic keyword queries.

One such method is to search keywords within certain websites, thereby excluding the rest of the internet and producing results only for the site you specified.

The following example is looking for the word “apples” inside www.fruitjuice.co.uk.

site:www.fruitjuice.co.uk apples

It’s also possible to see pages that link to a certain website or page on that website:

link:www.fruitjuice.co.uk

This list of Google search operators are worth taking a look at if you’ve never used them before. And don’t worry about remembering them all, because there is Google advanced search which fulfills some of the same functionality.

I should point out that the link: operator is not always accurate and will not always return accurate backlink analysis. Sometimes you have to put a space between the colon and the www for it to work.

There’s an article by Rand Fishkin of the SEO brand Moz in which he debunks some of the hype surrounding these Google operators. You can read more about that here.

Search operators are still useful for quick searches and general analysis but will not tell you the whole story.

Change Your Google Search Location

Something else worth mentioning is that you can change your Google search location to get local results.

This is handy if you want to make Google believe you’re performing a search in a specific geographic location. When you look up particular keywords, Google will often take your location into account and try to show you results it thinks are useful to you.

2) Companies House

Is the competitor you’re “staking out” a registered company? You can access lots of juicy details about a company by visiting the UK business public registrar website at www.companieshouse.gov.uk.

Companies House as the record keeper of births, deaths and marriages for companies. When a company is created, renamed, restructured or closed the information is logged.

Entering the company name or company number will reveal a lot.

You can get information for free including the registered office address, previous company names, details of the directors, company dissolution, when accounts are filed or due and a history of it’s filed documents.

Here’s a “dummy” search result I created to give you an idea of what to expect:

On the same page you’ll see some links related to ordering more information or monitoring the company:

You might be pleased to know that Companies House has decided to release all public digital data held on the UK register of companies. It’s now available on their new public beta service free of charge.

Check this link https://beta.companieshouse.gov.uk/help/welcome for an improved interface and access to downloadable PDF documents about any company.

3) Tweetdeck

The amount of information disclosed on Twitter is enormous. It moves so fast that is almost represents a real time social consciousness. 

Tweetdeck is the official Twitter management and is excellent for monitoring particular user activity. Set up a few dozen columns and into these you can load the Tweet timelines of the Twitter accounts you want to watch.

The fact that rivals could be putting so much information about themselves online means you might be able to second guess their behaviour, strategies, inclinations etc all by paying close attention to social activity.

Some people type more words per day on Twitter than the total amount of words spoken to human beings. 

It’s also possible to monitor keywords or hashtags to see what conversations are happening around these.

4) Followerwonk

Would you like to know how a particular Twitter account is being used? Do you envy the number of retweets someone gets, or are you just curious to know what the gender makeup of followers is?

Followerwonk is a brilliant fremium tool for analysing any @username by running a report to uncover stats.

Report data includes information such as what times of the day a user is most active and what type of Tweets they publish (Retweets, normal tweets etc).

 

In Followerwonk the “most active hours” part of the report shows when they are sending the personal @reply style Tweets to other users. 

This is an indication of when, on average, a user is actually logged into Twitter and talking to others. It’s useful to know this in situations where you want to monitor those conversations.

The larger, more detailed reports are generated and autosaved for viewing later, and is accessed by first signing into your Twitter account and then signing into Followerwonk.

5) Wayback Machine

I bet you’d love to take a look at how a competitor’s website looked years ago. Wouldn’t it be revealing to see what they are doing differently now compared to then?

With the Archive.org Wayback Machine cache it is possible to view historical archived “snapshots” of any website.

This means you can look at the website in different states over the course of years, months, weeks and days as far back as the very first snapshot.

Taking a peek at a previous version of a website could be useful:

• Pricing, products and services changes will be apparent
• It might be that content has been rewritten to cover something up
• There may have been fake testimonials recently removed
• The company trading name and number could have changed even if the website URL remains as is

By the way, it is possible to exclude the Wayback Machine from crawling your website by placing a command in the robots.txt file in your site root.

Overall, Wayback Machine is dead handy for looking back.

You can use it on YouTube to see what a particular user’s channel looked like. The videos might not play, however.

6) WHOIS

If you want to dig up contact information that isn’t available on the website of a business, you’ll like this.

WHOIS operates as a public data register cataloging the individuals, businesses and organisations creating domain names.

Every year millions of entries are added to the WHOIS archive, creating an internet phonebook of sorts:

WHOIS does not act as a central database, but more a search engine.

Unless this person has purchased “domain privacy”, the details they used to sign up to the domain registrar are used in WHOIS.

Many times, it’s a business owner’s home address that gets used. Their personal mobile phone number might be visible too.

All URL registrars have a mandatory contact details section in their account dashboard. If you chose to buy domain privacy, it will be invisible to WHOIS, kind of like being unlisted in the phonebook.

In terms of competitive analysis, you might want to acquire a certain URL that is already registered. To find out who owns it, dip into WHOIS to uncover the details.

A few years ago I wanted the URL www.smallbizgeek.com but it was unavailable. Someone had bought it but had not built a website for it. They were sitting on the URL but apparently not using it.

A search for www.smallbizgeek.com in WHOIS revealed the owner details so I emailed her to make enquiries.

Unfortunately acquiring the domain was a no-go, but at least I was able to have the conversation and that was because WHOIS revealed the contact details needed.

The same person had purchased other URLs pointed at active websites they were running as part of their current business. From there I was able to look at their website and competitively analyse their approach.

Here’s a quick tip while I think about it: if you’re a business operating from home, you should use a virtual address as part of your WHOIS registration information. It protects your personal details and adds a layer of security.

7) Screaming Frog SEO Spider

Screaming Frog is a freemium SEO auditing tool sporting impressive features and benefits galore for competitive analysis. If you need to dig deep into any website, simply enter the address, click Start and the software will crawl up 500 URI’s.

A URI (uniform resource identifier) is any public “resource” used by the website. Resources include HTML documents, CSS documents, images and javascript.

At a basic level, Screaming Frog is a desktop application useful for discovering what is hiding behind the scenes of websites. 

It reveals on page SEO criteria such as page titles, meta descriptions, keywords, heading tags, search engine crawl instructions, and even word count.

I recently looked at a website that was offering a PDF download brochure but the link was broken. The webmaster had obviously made a typo in the URL of this PDF document because it was returning a 404 not found error report.

Using Screaming Frog, I ran an audit and within seconds pinpointed the true location of the elusive PDF. Bingo!

While I was running the crawl, other PDF documents that weren’t being advertised publicly on the website became available for me to look at via the Screaming Frog interface.

There’s a hell of a lot you can do with this software. Use it to get the insights you’re not supposed to be privy to.

8) Majestic Site Explorer

There are free web based tools that allow you to gather intelligence on a URL and find out its relationship to the rest of the internet. 

Majestic is one such tool aimed at SEO professionals, media analysts, developers and entrepreneurs wanting to examine URLs. 

Under the free plan it is possible to view statistical information as well as determine what websites are linking to the site you are exploring.

One thing I like about the backlink analysis section is that if the website owner/manager/webmaster leaves blog comments on other websites, you can follow the links back to these sites and read the comments they have left. 

Tracking comments can reveal what online circles your quarry mixes with, and with whom they might have forged professional relationships. 

Of course, all domain data is limited since we’re talking about the free version of Majestic.

The full feature set is behind a paywall and unregistered access to the tools is curtailed (boo!) but what you can do is register a free account and lift some of those restrictions (yay!)

The good news is, other similar free website explorer tools exist and these can pick up the slack to compensate any missing data on the other website explorers.

9) Moz Open Site Explorer

This website URL explorer is similar to the other tools and is by Moz, the developers of Followerwonk. Once again, the full features are accessible only if you take out a paid subscription BUT there’s much to be gleaned from the basic free version.

Enter a website address and you’ll be granted access to lists of stats and data, including external webpages linking to the URL you’re exploring.

It is the same principle as Majestic SEO, yet you might see slightly different results. The way these domain explore tools work are all different.

There’s a daily report limit under the free account. If you want to run more than three reports per day, consider signing up for a free Moz community account.

10) SEM Rush

Yet another freemium tool with domain exploring features, I like to use SEM Rush in combination with the other similar domain analysis services I’ve mentioned.

Enter the URL, specify location and click search to get limited stats. You will have to upgrade to unlock all features but I believe there’s a free trial you can sign up for.

I suppose you could create new email accounts and continually leverage the 14 day Pro plan. Remember to cancel before your payment card is debited.

The domain overview presents information including organic search rank and keyword positions as well as organic domain competitors.

Some of the competitors to creativebloq.com include smashingmagazine.com, digitalartsonline.co.uk and canva.com. 

Then there’s backlinks, anchor text, dofollow and nofollow ratio.

When you compare some of the data to other tools like Majestic SEO and Open Site Explorer, don’t be surprised if the data varies slightly.

11) Ahrefs

Another freemium website explorer useful for gaining a quick overview of backlinks, anchors and organic traffic is Ahrefs.

As ever, the real data is hidden behind a subscription paywall but you can still steal a quick idea of what a competitor’s website looks like from an SEO point of view.

If you’re not willing to pay for the subscription – and I wouldn’t blame your sharp intake of breath when you look at the prices of these things – you can do most people do and register a free trial.

You can glean overall website data for a single URL by using the various site explorers we’ve already discussed and combining the data in venn diagram fashion.

12) Google Adwords Keyword Planner

Get a Google account and you can plunder the Google Adwords repository for search terms a competing website contains or is ranking for.

Do a quick Google search for your main keyword (for example “small business technology”) and note the top 5 ranking URLs.

Visit the Adwords Keyword Planner and plug one of these URLs into the field labelled Your landing page.

Click the blue Get Ideas button and you’ll be presented with Ad group ideas and keyword ideas  relevant to your site.

Feel free to change the targeting by using a different country if you want to niche down a bit more.

13) Talkwalker Alerts

Talkwalker Alerts is a free tool lets you monitor the web for content containing keywords you’re interested in.

You might want to keep tabs on phrases you’re trying to rank for, words that describe your products or services or instances in which your competitor is mentioned in an online article, press release etc.

Every time a match is found online, you’ll receive an email showing you the usage of the phrase online. There is also a link back to the original webpage where it appeared.

You can set up specific parameters that can identify usages that include or exclude certain criteria.

For example, I am monitoring a set of keywords that include “Derby” but I have told Talkwalker Alerts to only return results in the UK. I do not want to receive alerts pertaining to United States locations like Derby in Kansas, Connecticut or Vermont

Use it to monitor keywords you already target and see if any other company, brand or individual is also using the same keywords or keyphrase.

By staying on top of the use of particular words related to your business, you might be able to react quickly to breaking news inside a limited window of opportunity.

It’s similar to Google Alerts, but I hear that is being retired in the not too distant future.

14) Website Grader

Website Grader is free fool from HubSpot that analyses a website and provides suggestions. You could use this to asses a competitor and see how fast their site is (or isn’t) whether it’s mobile responsive and how their on page SEO looks.

Plug in the URL and click “Get Your Answer” to see the results of the audit. The details returned are not granular, so do not take too much of it at face value. Competition-wise, it could provide some insight into the quality and standards of the website you’re examining. 

 

 

 

15) BuzzSumo

BuzzSumo is a search engine for measuring the most shared content in particular topic areas. Using URLs, individual’s names or just a topic keyword, you can monitor your competitors and see how popular their posts are.

Using filters and advanced search options you can dig into what your peers are up to and how popular their webpages are on social media.

It’s also a great way of finding out if someone you’ve been keeping an eye on has been writing guest posts for other websites, therefore revealing their online associations.

The tool has strict limitations and a daily use allowance under the free version. Sign up for a free account to run unlimited searches, but if you want to view who shared and who is linking to the content you’ll need a pro subscription.

I should mention that Twitter have revoked the part of their API that handles the share count.

This means that you don’t get to see how many people tweeted a piece of content and because of this any third party plugins (including BuzzSumo) will not be showing those numbers; at least not as of late 2015.

BuzzSumo may still be showing a figure for the supposed number of Twitter shares but that number stopped being updated as of November 2015.

I emailed BuzzSumo about this and they said they’ve developed a Google Chrome plugin that’s supposed to take up the slack. 

You can read about the BuzzSumo Chrome extension here.

16) IceRocket

Instantly search blogs, Twitter and Facebook for specific terms using the free IceRocket. No account is needed.

Enter your term(s) in the search box and choose whether to search blogs, Twitter, Facebook or all of them.

The filter options personalise the search. Use the settings for limiting the results to certain authors or filtering Twitter searches by hashtags, text or links.

Analysis

This list is a good place to start intelligence gathering and should keep you busy for a while.

By analysing the movements, history and strategy of similar or competing businesses, you might be able to siphon their resources, their contacts, their ideas and whatever else has been left out in full view.

Become a savvy investigator. Arm yourself with knowledge. But make sure you’re covering your own flank too.

If you don’t know what to try first, go for Talkwalker Alerts (free) because you can set up some monitoring and start receiving email alerts. 

Can you recommend any similarly sneaky apps not mentioned in this post?

Do you know some good free services that dig up the data? Do tell.