Google Dorking
Go to room
Explaining how Search Engines work and leveraging them into finding hidden content!
Task 1 - Ye Ol’ Search Engine
- Roger dodger!
No answer needed
Task 2 - Let’s Learn About Crawlers
Name the key term of what a “Crawler” is used to do
Index
What is the name of the technique that “Search Engines” use to retrieve this information about websites?
Crawling
What is an example of the type of contents that could be gathered from a website?
Keywords
Task 3 - Enter: Search Engine Optimisation
- Use the same SEO checkup tool and other online alternatives to see how their results compare for https://tryhackme.com and http://googledorking.cmnatic.co.uk
No answer needed
Task 4 - Beepboop - Robots.txt
Where would “robots.txt” be located on the domain “ablog.com”
ablog.com/robots.txt
If a website was to have a sitemap, where would that be located?
/sitemap.xml
How would we only allow “Bingbot” to index the website?
User-agent: Bingbot
How would we prevent a “Crawler” from indexing the directory “/dont-index-me/“?
Disallow: /dont-index-me/
What is the extension of a Unix/Linux system configuration file that we might want to hide from “Crawlers”?
.conf
Task 5 - Sitemaps
What is the typical file structure of a “Sitemap”?
XML
What real life example can “Sitemaps” be compared to?
Map
Name the keyword for the path taken for content on a website
Route
Task 6 - What is Google Dorking?
What would be the format used to query the site bbc.co.uk about flood defences
site: bbc.co.uk flood defences
What term would you use to search by file type?
filetype:
What term can we use to look for login pages?
intitle: login