o  	  Utku Sen's
				 \_/\o   
				( Oo)                    \|/
				(_=-)  .===O-  ~~U~R~L~~ -O-
				/   \_/U'        hunter  /|\
				||  |_/
				\\  |    utkusen.com
				{K ||	twitter.com/utkusen
Copy the code

Urlhunter is a reconstruction tool that allows searching for urls exposed through shortener services such as bit.ly and goo.gl. The project was written in Go.

How to?

A group called URLTeam (give them a thumb-up) is violently attacking the URL shortener service and Posting matching results every day. Urlhunter downloads their collections and lets you analyze them.

The installation

Slave binary

You can download the pre-built binaries from the release page and run them. For example.

The tar XZVF urlhunter_0. 1.0 _Linux_amd64. Tar. Gz

./urlhunter --help

From the source

  1. Install Go on your system

  2. Run.

go get -u github.com/utkusen/urlhunter

** Note for Windows users: ** UrlHunter uses XZ Utils, which is pre-installed on Linux and macOS systems. For Windows, you’ll need to download it from tukaani.org/xz/.

Method of use

Urlhunter requires three arguments to run: -keywords,-date, and -o.

For example. urlhunter -keywords keywords.txt -date 2020-11-20 -o out.txt

-keywords

You need to specify a TXT file containing the keywords to search on the URL. Keywords must be written line by line. You can specify keywords in three different ways.

** Single keyword: ** URlHunter searches for a given keyword as a substring. For example.

The acme.com keyword matches both https://acme.com/blabla and https://another.com/?referrer=acme.com

** Multiple keywords: ** URlHunter will use AND logic to search for a given keyword. This means that a URL must include all the keywords provided. Keywords must be separated by characters. For example.

Acme.com, admin will be matching with https://acme.com/secret/adminpanel, but not * * * * match at https://acme.com/somethingelse

** Weight code value: ** URlHunter will search for the given weight code value. In the keyword file, the line containing the regular expression formula must begin with a regex character. The format is: Regex REGEXFORMULA. For instance

Regex 1 \ d {10} will be matching with https://example.com/index.php?id=12938454312, but not * * * * match at https://example.com/index.php?id=abc223

The date –

Urlhunter downloads archive files for a given date. You can specify a date in three different ways.

** Latest: ** UrlHunter will download the latest files. -date latest

** Single date: ** UrlHunter will download files for a given date. The date format is YYY-MM-DD.

For example. -date 2020-11-20

** Date range: ** UrlHunter will download all files between the given start and end dates.

For example. -date 2020-11-10:2020-11-20

-o

You can specify the output file with the -o argument. For example – o out. TXT

Demonstration video

The speed problem

Archive.org sets speed limits for downloading files. As a result, downloading files takes more time than usual. As a workaround, you can download files via Torrent and place them in archive/, which is in the same directory as the urlHunter binaries. The directory tree will look like.

|-urlhunter
|---urlhunter(binary)
|---archive
|-----urlteam_2020-11-20-11-17-04
|-----urlteam_2020-11-17-11-17-04
Copy the code

Using the instance

Urlhunter can be useful for cyber intelligence and error bounty purposes. For example.

Docs.google.com/a/acme.com drive.google.com/a/acme.com keyword allows you to find the Acme company public Google Docs&Drive Shared links.

Acme.com, the password_reset_token keyword lets you find the working password reset token for acme.com.

Trello.com allows you to find publicly available Trello addresses.

Thank you

Special thanks to Samet Bekmezci(@sametbekmezci) who gave me the idea for this tool.

donation

Bitcoin address. 1MVGYtuYAbZ1MubRFb7dxhQH35w7RQBcVe

GitHub

Github.com/utkusen/url…