7fb6d26cc7
Previous code was failing the flake8 tests because the random module was not imported.
2019-01-30 18:37:03 -06:00
f69be05803
Rebase & bump version
2019-01-29 10:11:18 +03:30
465f4c85c3
Typo in printout when reading proxies from file.
2019-01-29 10:10:31 +03:30
9f523365f7
Finally importing load_proxies module.
2019-01-29 10:10:31 +03:30
8587d1a835
If the ProxyError gets raised in the 'get_response' function, the request will be tried with another proxy selected from the 'proxy_list' global var. New parameter 'retry_no' is the number of retries that will be made before throwing a final ProxyError.
2019-01-29 10:10:31 +03:30
6bf8358342
Set new parameter 'retry_no' of the 'get_response' function to 3 (can be changed). This will be used if retrying a ProxyError.
2019-01-29 10:10:31 +03:30
855f154d9b
If the 'proxy_list' we select a random member and pass it as the proxy to the session. If the list is empty, the proxy parameter will be set to arg.proxy, which defaults to None if the user did not pass an individual proxy as well.
2019-01-29 10:10:31 +03:30
2accdcafea
If the user selected --check_proxies option along with --proxy_list option, proxies loaded from the .csv file are checked using the check_proxies function from the load_proxies module. Proxies which pass the test are stored in the proxy_list global var.
2019-01-29 10:10:06 +03:30
6cc4e22898
If the user selected --proxy_list option, we attempt to read proxies from the csv, and store the list in global var proxy_list.
2019-01-29 10:10:06 +03:30
bd683022b3
Exception is raised if both a single proxy and the proxy_list are used. As needed, this can be changed to merging the single proxy with the proxy list, but seems a bit unnecessary at this time.
2019-01-29 10:10:06 +03:30
dc32d473e0
Exception will now be raised if etiher a single proxy or proxy_list options are used along with Tor.
2019-01-29 10:10:06 +03:30
c5e06b068e
Added two new arguments, '--proxy_list'/'-pl' and '--check_proxies'/'-cp', for users to activate options of reading proxies from a document (at this time, only .csv is supported), and check their anonimity before using them.
2019-01-29 10:10:06 +03:30
166d224423
First change to 'sherlock.py' for use of load_proxies module. Global variable proxy_list is created, and by default set to an empty list. This variable will store proxies from a proxy list (if this option is used), and will enable different threads to access proxies at the same time.
2019-01-29 10:09:33 +03:30
67108071e5
bump version
2019-01-29 10:02:54 +03:30
011df7af55
bump version
2019-01-27 13:57:03 +03:30
8b681158bc
small corrections to rank sort
2019-01-25 17:36:38 +00:00
78ade00dee
Update outdated REAME.md
2019-01-25 15:10:03 +00:00
5d972a3138
add --rank -r option to sherlock
2019-01-25 15:05:38 +00:00
0e6b8d0dca
Added optional parameters for outputting files and folders.
2019-01-24 21:59:06 +02:00
f511faab23
Added the ability to load external json files.
2019-01-24 21:16:49 +02:00
7a93f73b23
bump version
2019-01-23 23:03:01 +03:30
a18065c1a7
Bump version. Update usage results in readme.
2019-01-22 20:55:10 -06:00
a538c07ab3
Merge branch 'pr/n133_cclauss'
2019-01-22 20:51:32 -06:00
65b38592c4
Change "response_url" detection strategy completely.
...
Previously, there was a problem with sites that redirect an attempt to view a non-existing username to the main site. For example, if you try to go to https://devrant.com/users/dfoxxxxxxxxx (a user name that does not exist), then we get a redirect to the https://devrant.com/ root of the site. But, the "response_url" checking algorithm was only looking for the configured error URL being included in the response. So, these sites always indicated that the username was not found.
Update the "response_url" detection method so that the request does not allow redirects. If we get a 200 response of some type, then the username has been found. However, if we get something like a 302, then we know that the username was not found as we are being redirected.
This whole method seems fragile, but I did exhaustively test all of the supported sites, and they all work. So, this change is clearly an improvement.
2019-01-22 20:37:05 -06:00
0b3c5f2ba0
Fix the top of the deerstalker
...
https://en.m.wikipedia.org/wiki/Deerstalker
2019-01-22 20:15:13 +01:00
a380e48057
Make BANNER an r'string' and move it up front
...
Sherlock has a cool logo so this PR moves it to the top of sherlock.py to show it off when potential contributors read our source. It also makes a BANNER an __r'string'__ so we do not need backslashes to escape characters and the ascii art is more what-you-see-is-what-you-get. This also suppresses [a bunch of flake8 complaints](https://travis-ci.com/TheYahya/sherlock/jobs/172028552#L490-L503 ) about _invalid escape sequences_. Finally this PR runs isort on the imports for readability. Standard Lib imports come first, etc.
2019-01-22 18:55:03 +01:00
4c6bb61483
Update version number. This version does have basic tests.
2019-01-21 19:27:08 -06:00
ee4cd8e14b
While running tests, I was getting a ResourceWarning about an unclosed file. The result file was not being closed, so it was a valid warning. Close the file to avoid this warning.
2019-01-21 16:30:07 -06:00
d4e29b1327
bump version
2019-01-19 19:24:02 +03:30
ff9e239928
bump version
2019-01-15 00:32:30 +03:30
29d89921d0
add proxy functionality
2019-01-12 12:55:19 +08:00
ed196b7466
bump version
2019-01-10 17:23:55 +03:30
1c5e07645e
Merge branch 'Made_parameter_for_site_name_not_case_sensitive'
2019-01-10 17:22:56 +03:30
2247f556f2
bump version
2019-01-10 17:17:17 +03:30
b245a6e5cc
fixed wrong condition place
2019-01-09 19:46:02 +02:00
34a0db9cd8
implemented #115 : make parameter for site name not case sensitive
2019-01-09 19:43:00 +02:00
d0ecb70d84
Merge pull request #109 from nareddyt/response-time-metrics
...
Response time metrics
2019-01-08 19:08:47 -05:00
5ed2c9a740
bump version
2019-01-09 01:03:11 +03:30
038d531483
doc: response time
...
- Document new feature for response times
- Refactor
2019-01-07 12:04:55 -05:00
4c3967fd6d
feat: expose response times
...
- Add hook to FuturesSession to calculate absolute response time (ms)
- In verbose mode, display response times for each request
- Always outputs response times to CSV
2019-01-06 22:33:36 -05:00
7e5bac0cb8
bump version
2019-01-06 14:02:35 +03:30
18ee78f9e0
Merge branch 'masta2898/master'
2019-01-06 14:01:45 +03:30
c165952ca8
bump version
2019-01-06 13:55:50 +03:30
33e8beb5b4
Add command line option to only run a report on specified sites (as opposed to all of them). Move loading of JSON file out of the query logic proper: we need to keep the database and the query logic separate anyway for future changes, so this is a first step in the refactoring. Update readme file with latest information.
2019-01-05 22:52:53 -06:00
4596f7121e
Specify the encoding to be UTF-8 for the csv file. The Rajče.net site was causing this to fail.
2019-01-05 20:39:56 -06:00
5dc147e39a
efficient usage of GET request only for github
2019-01-05 23:05:23 +02:00
81d15482f7
bump version
2019-01-06 00:30:32 +03:30
69cdbe83f2
fix #95 : head request may cause 301 redirect which can be interpreted wrong i.e. github's cause
2019-01-05 19:45:19 +02:00
1e50c92d8b
update from upstream
2019-01-05 18:50:01 +02:00
0d85703093
bump version
2019-01-05 16:57:31 +03:30
Christopher K. Hoadley