o Utku Sen's
\_/\o
( Oo) \|/
(_=-) .===O- ~~U~R~L~~ -O-
/ \_/U' hunter /|\
|| |_/
\\ | utkusen.com
{K || twitter.com/utkusen
Copy the code
Urlhunter is a reconstruction tool that allows searching for urls exposed through shortener services such as bit.ly and goo.gl. The project was written in Go.
How to?
A group called URLTeam (give them a thumb-up) is violently attacking the URL shortener service and Posting matching results every day. Urlhunter downloads their collections and lets you analyze them.
The installation
Slave binary
You can download the pre-built binaries from the release page and run them. For example.
The tar XZVF urlhunter_0. 1.0 _Linux_amd64. Tar. Gz
./urlhunter --help
From the source
-
Install Go on your system
-
Run.
go get -u github.com/utkusen/urlhunter
** Note for Windows users: ** UrlHunter uses XZ Utils, which is pre-installed on Linux and macOS systems. For Windows, you’ll need to download it from tukaani.org/xz/.
Method of use
Urlhunter requires three arguments to run: -keywords,-date, and -o.
For example. urlhunter -keywords keywords.txt -date 2020-11-20 -o out.txt
-keywords
You need to specify a TXT file containing the keywords to search on the URL. Keywords must be written line by line. You can specify keywords in three different ways.
** Single keyword: ** URlHunter searches for a given keyword as a substring. For example.
The acme.com keyword matches both https://acme.com/blabla and https://another.com/?referrer=acme.com
** Multiple keywords: ** URlHunter will use AND logic to search for a given keyword. This means that a URL must include all the keywords provided. Keywords must be separated by characters. For example.
Acme.com, admin will be matching with https://acme.com/secret/adminpanel, but not * * * * match at https://acme.com/somethingelse
** Weight code value: ** URlHunter will search for the given weight code value. In the keyword file, the line containing the regular expression formula must begin with a regex character. The format is: Regex REGEXFORMULA. For instance
Regex 1 \ d {10} will be matching with https://example.com/index.php?id=12938454312, but not * * * * match at https://example.com/index.php?id=abc223
The date –
Urlhunter downloads archive files for a given date. You can specify a date in three different ways.
** Latest: ** UrlHunter will download the latest files. -date latest
** Single date: ** UrlHunter will download files for a given date. The date format is YYY-MM-DD.
For example. -date 2020-11-20
** Date range: ** UrlHunter will download all files between the given start and end dates.
For example. -date 2020-11-10:2020-11-20
-o
You can specify the output file with the -o argument. For example – o out. TXT
Demonstration video
The speed problem
Archive.org sets speed limits for downloading files. As a result, downloading files takes more time than usual. As a workaround, you can download files via Torrent and place them in archive/, which is in the same directory as the urlHunter binaries. The directory tree will look like.
|-urlhunter
|---urlhunter(binary)
|---archive
|-----urlteam_2020-11-20-11-17-04
|-----urlteam_2020-11-17-11-17-04
Copy the code
Using the instance
Urlhunter can be useful for cyber intelligence and error bounty purposes. For example.
Docs.google.com/a/acme.com drive.google.com/a/acme.com keyword allows you to find the Acme company public Google Docs&Drive Shared links.
Acme.com, the password_reset_token keyword lets you find the working password reset token for acme.com.
Trello.com allows you to find publicly available Trello addresses.
Thank you
Special thanks to Samet Bekmezci(@sametbekmezci) who gave me the idea for this tool.
donation
Bitcoin address. 1MVGYtuYAbZ1MubRFb7dxhQH35w7RQBcVe
GitHub
Github.com/utkusen/url…