Add Codolio (coding portfolio tracker) as a new site target for username detection.
Detection method: Message-based using title tag differences
- Existing profiles: '<title>Username | Codolio</title>'
- Non-existing profiles: '<title>Page Not Found | Codolio</title>'
Tested with multiple usernames to confirm accurate detection.
- Add Odysee platform to sherlock database- Uses canonical link detection for non-existent users- URL pattern: https://odysee.com/@\{username\}- Detects error via canonical redirect to main site
- Add Odysee platform to sherlock database- Uses canonical link detection for non-existent users- URL pattern: https://odysee.com/@\{username\}- Detects error via canonical redirect to main site
- Add Odysee platform to sherlock database- Uses canonical link detection for non-existent users- URL pattern: https://odysee.com/@\{username\}- Detects error via canonical redirect to main site
- Updated OpenCollective to use status_code detection (previously used message detection)
- Added Realmeye with message detection
Both sites were previously removed due to false positives but have been verified to work correctly now:
- OpenCollective: Returns 200 for existing profiles, 404 for non-existent
- Realmeye: Shows 'Sorry, but we either:' error message for non-existent players
Tested with known usernames:
- OpenCollective: sindresorhus
- Realmeye: rotmg
Note: PowerShell Gallery was initially included but removed after discovering their /profiles/ endpoint no longer works.
- Added BoardGameGeek back using the new API endpoint suggested by @ppfeister
- Uses https://api.geekdo.com/api/accounts/validate/username?username={} for detection
- errorMsg checks for '"isValid":true' to detect valid usernames
- This approach avoids the previous issues with:
* HTML parsing returning false positives
* User API returning JSON with '[]' substrings that caused detection problems
- Successfully tested with both valid (blue) and invalid usernames
Thanks @ppfeister for the API suggestion and @akh7177 for the initial guidance
- Adds docker-build-test job to regression.yml
- Runs on push/merge to master and release branches
- Extracts VERSION_TAG from pyproject.toml for build
- Tests that Docker image builds and runs successfully
- Resolves dockerfile syntax warnings
- Resolves#2196"
Threads was showing false positives for non-existent users because
the error message detection was incorrect.
Updated errorMsg:
- Old: "<title>Threads</title>" (generic, matches valid pages too)
- New: "<title>Threads • Log in</title>" (specific to non-existent users)
When a user doesn't exist, Threads redirects to a login page with the
title "Threads • Log in". Valid user profiles have titles like
"Username (@username) • Threads, Say more".
Tested with:
- Invalid user (impossibleuser12345): Correctly not found
- Valid user (zuck): Correctly found
This fixes the false positive issue where non-existent Threads profiles
were being reported as found.
BoardGameGeek cannot be reliably detected with Sherlock's current capabilities:
- Original HTML detection: Returns false positives
- API endpoint approach: The API returns status 200 for both valid and invalid users
- Invalid user: Returns exactly '[]'
- Valid user: Returns JSON containing '[]' substrings (e.g., "adminBadges":[])
Since Sherlock's 'message' errorType uses substring matching, it incorrectly
identifies valid users as "not found" when checking for '[]' in the response.
The site's API response format is fundamentally incompatible with Sherlock's
detection methods (message/status_code/response_url), so removal is the only
viable solution to prevent false positives and false negatives.
Addresses false positive issue originally reported in testing.
Using the API endpoint suggested by akh7177:
https://api.geekdo.com/api/users?username={}
However, there's an edge case where valid users contain empty arrays
in their JSON response (adminBadges[], userMicrobadges[], supportYears[])
which causes Sherlock's substring matching to incorrectly flag them
as 'not found' when looking for the '[]' error pattern.
The API correctly returns:
- Valid user: JSON object with user data (but contains [] substrings)
- Invalid user: Exactly '[]' (2 characters total)
This needs further refinement to distinguish between the exact '[]'
response vs JSON containing '[]' substrings.
BoardGameGeek returns identical pages for both existing and non-existing
users, making reliable username detection impossible with HTTP-based
methods. The site likely uses JavaScript to load user-specific content
dynamically.
BoardGameGeek changed from /user/{} to /profile/{} URL structure.
Also updated from message to status_code detection as the site
no longer returns clear error messages for non-existent users.
This PR adds AWS Skills Profile to Sherlock’s supported sites in data.json. The configuration uses a unique substring (`shareProfileAccepted":false`) for reliable detection of non-existent usernames, addressing the challenge of JavaScript-rendered error messages.
- Site details and detection logic follow Sherlock’s contributing guidelines and Code of Conduct.
- No changes to core logic; only a new site entry.
- Reviewed for schema compliance and duplicate key cleanup as noted.
This fix addresses a critical security vulnerability where HTTP requests
could hang indefinitely, potentially causing denial of service.
Changes:
- Added 10-second timeout to version check API call
- Added 10-second timeout to GitHub pull request API call
- Added 30-second timeout to data file downloads (larger timeout for data)
- Added 10-second timeout to exclusions list download
Impact:
- Prevents infinite hangs that could freeze the application
- Improves user experience with predictable response times
- Fixes security issue flagged by Bandit static analysis (B113)
- Makes the application more robust in poor network conditions
The timeouts are conservative enough to work with slow connections
while preventing indefinite blocking that could be exploited.
> Packages for ParrotOS and Ubuntu 24.04, maintained by a third party, appear to be __broken__.
> Users of these systems should defer to pipx/pip or Docker.
> Users of these systems should defer to [`uv`](https://docs.astral.sh/uv/)/`pipx`/`pip` or Docker.
| Method | Notes |
| - | - |
| `pipx install sherlock-project` | `pip` may be used in place of `pipx` |
| `pipx install sherlock-project` | `pip` or [`uv`](https://docs.astral.sh/uv/) may be used in place of `pipx` |
| `docker run -it --rm sherlock/sherlock` |
| `dnf install sherlock-project` | |
@@ -97,24 +97,6 @@ optional arguments:
--local, -l Force the use of the local data.json file.
--nsfw Include checking of NSFW sites from default list.
```
## Apify Actor Usage [](https://apify.com/netmilk/sherlock?fpr=sherlock)
<a href="https://apify.com/netmilk/sherlock?fpr=sherlock"><img src="https://apify.com/ext/run-on-apify.png" alt="Run Sherlock Actor on Apify" width="176" height="39" /></a>
You can run Sherlock in the cloud without installation using the [Sherlock Actor](https://apify.com/netmilk/sherlock?fpr=sherlock) on [Apify](https://apify.com?fpr=sherlock) free of charge.
Read more about the [Sherlock Actor](../.actor/README.md), including how to use it programmatically via the Apify [API](https://apify.com/netmilk/sherlock/api?fpr=sherlock), [CLI](https://docs.apify.com/cli/?fpr=sherlock) and [JS/TS and Python SDKs](https://docs.apify.com/sdk?fpr=sherlock).
## Credits
@@ -124,7 +106,7 @@ Thank you to everyone who has contributed to Sherlock! ❤️
"description":"Used to clarify important target information if (and only if) a commit message would not suffice.\nThis key should not be parsed anywhere within Sherlock."
"description":"Used to clarify important target information if (and only if) a commit message would not suffice.\nThis key should not be parsed anywhere within Sherlock."
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Siddharth Dushantha