BoardGameGeek cannot be reliably detected with Sherlock's current capabilities:
- Original HTML detection: Returns false positives
- API endpoint approach: The API returns status 200 for both valid and invalid users
- Invalid user: Returns exactly '[]'
- Valid user: Returns JSON containing '[]' substrings (e.g., "adminBadges":[])
Since Sherlock's 'message' errorType uses substring matching, it incorrectly
identifies valid users as "not found" when checking for '[]' in the response.
The site's API response format is fundamentally incompatible with Sherlock's
detection methods (message/status_code/response_url), so removal is the only
viable solution to prevent false positives and false negatives.
Addresses false positive issue originally reported in testing.
Using the API endpoint suggested by akh7177:
https://api.geekdo.com/api/users?username={}
However, there's an edge case where valid users contain empty arrays
in their JSON response (adminBadges[], userMicrobadges[], supportYears[])
which causes Sherlock's substring matching to incorrectly flag them
as 'not found' when looking for the '[]' error pattern.
The API correctly returns:
- Valid user: JSON object with user data (but contains [] substrings)
- Invalid user: Exactly '[]' (2 characters total)
This needs further refinement to distinguish between the exact '[]'
response vs JSON containing '[]' substrings.
BoardGameGeek returns identical pages for both existing and non-existing
users, making reliable username detection impossible with HTTP-based
methods. The site likely uses JavaScript to load user-specific content
dynamically.
BoardGameGeek changed from /user/{} to /profile/{} URL structure.
Also updated from message to status_code detection as the site
no longer returns clear error messages for non-existent users.
This PR adds AWS Skills Profile to Sherlock’s supported sites in data.json. The configuration uses a unique substring (`shareProfileAccepted":false`) for reliable detection of non-existent usernames, addressing the challenge of JavaScript-rendered error messages.
- Site details and detection logic follow Sherlock’s contributing guidelines and Code of Conduct.
- No changes to core logic; only a new site entry.
- Reviewed for schema compliance and duplicate key cleanup as noted.
This fix addresses a critical security vulnerability where HTTP requests
could hang indefinitely, potentially causing denial of service.
Changes:
- Added 10-second timeout to version check API call
- Added 10-second timeout to GitHub pull request API call
- Added 30-second timeout to data file downloads (larger timeout for data)
- Added 10-second timeout to exclusions list download
Impact:
- Prevents infinite hangs that could freeze the application
- Improves user experience with predictable response times
- Fixes security issue flagged by Bandit static analysis (B113)
- Makes the application more robust in poor network conditions
The timeouts are conservative enough to work with slow connections
while preventing indefinite blocking that could be exploited.
Abhyuday K Hegde