211 Commits

Author SHA1 Message Date
Juan Rodriguez Donado 2c55ebf406 Merge pull request #16 from sjdonado/automated/update-parsers
chore: Update UA regexes and GeoLite2 database
2026-05-05 08:27:37 +02:00
sjdonado 2a4264e4c5 chore: update parsers 2026-05-05 00:38:32 +00:00
Juan Rodriguez Donado adfebe9d63 Merge pull request #15 from sjdonado/automated/update-parsers 2026-02-25 10:23:53 +01:00
sjdonado 45bf499d21 chore: update parsers 2026-02-20 00:24:21 +00:00
Juan Rodriguez Donado 1552b5ce09 Merge pull request #14 from sjdonado/automated/update-parsers 2026-02-06 21:53:16 +01:00
sjdonado 81f3c95c2b chore: update parsers 2026-02-05 00:26:49 +00:00
Juan Rodriguez Donado 3776621fe9 Merge pull request #13 from sjdonado/automated/update-parsers
chore: Update UA regexes and GeoLite2 database
2026-01-13 05:41:34 +01:00
sjdonado 0d68b0d6e1 chore: update parsers 2026-01-11 00:23:25 +00:00
Juan Rodriguez Donado 8048277f1d Merge pull request #12 from sjdonado/automated/update-parsers
chore: Update UA regexes and GeoLite2 database
2025-12-23 20:22:58 +01:00
sjdonado dcff88f55e chore: update parsers 2025-12-20 00:20:09 +00:00
Juan Rodriguez Donado f7add0116e Merge pull request #11 from sjdonado/automated/update-parsers
chore: Update UA regexes and GeoLite2 database
2025-12-19 06:53:05 +01:00
sjdonado 5f702e69c9 chore: update parsers 2025-12-14 00:22:55 +00:00
Juan Rodriguez Donado 2feeff70bc Merge pull request #9 from sjdonado/automated/update-parsers
chore: Update UA regexes and GeoLite2 database
2025-11-13 21:34:01 +01:00
sjdonado 1bb42684c3 chore: update README 2025-11-13 21:33:08 +01:00
sjdonado b6e7c45c80 chore: update parsers 2025-11-09 00:21:26 +00:00
sjdonado 7d275685b4 fix: missing paths to dockerignore 2025-11-02 15:25:30 +01:00
sjdonado 44dab3ca5a chore: update README 2025-11-02 11:40:50 +01:00
Juan Rodriguez Donado b8c1269d6e Merge pull request #8 from sjdonado/automated/update-parsers
chore: Update UA regexes and GeoLite2 database
2025-11-02 11:32:03 +01:00
sjdonado 9fcd478f86 chore: simplify docs structure 2025-11-02 11:31:17 +01:00
sjdonado 7c6d67c0c7 chore: update parsers 2025-11-02 10:03:33 +00:00
Juan Rodriguez Donado 353cf68852 Merge pull request #7 from sjdonado/refactor/benchmark-native-and-updates
Refactor/benchmark native and updates
2025-11-02 10:59:57 +01:00
sjdonado fef015ce53 feat: openapi swagger ui 2025-11-02 10:57:02 +01:00
sjdonado 80a59094f9 refactor: publish workflow replace manifest with multi-platform build in one step 2025-11-02 10:29:55 +01:00
sjdonado 6f5ad76718 refactor: Dockerfile replace debian with alpine 2025-11-02 10:29:08 +01:00
sjdonado 52497235b9 refactor: benchmark.cr configure a separate database 2025-11-02 10:24:06 +01:00
sjdonado aec073b696 fix: tests type errors 2025-11-02 10:16:19 +01:00
sjdonado 6e587f0176 ci: update parsers workflow 2025-11-02 08:55:34 +01:00
sjdonado 0c89fad713 refactor: improve click buffer performance 2025-11-02 08:48:10 +01:00
sjdonado 046b15bdce refactor: replace docker benchmark with native ps 2025-11-02 08:31:24 +01:00
sjdonado d0412d802b chore: update parsers 04.2025 2025-04-07 19:27:41 +02:00
sjdonado c9e7ad1d99 refactor: rename update-data to update-parsers 2025-04-07 19:26:23 +02:00
sjdonado 9fa7142ea7 feat: improve performance with dynamic batch capacity 2025-03-23 22:04:38 +01:00
Juan Rodriguez c539662235 chore: Update README.md 2025-03-23 16:18:44 +01:00
sjdonado a68259a0f4 fix: click_channel decrease buffer size and processor batch size 2025-03-23 13:33:17 +01:00
sjdonado 136e4d44c9 refactor: link controller buffered channel to hold click data 2025-03-23 13:30:34 +01:00
sjdonado 660d536618 fix: replace user_agent_parser with UserAgent + pre compiled regexes 2025-03-23 12:27:52 +01:00
sjdonado e1d3ec480d refactor: IpLookup lazy load reader 2025-03-23 12:25:19 +01:00
sjdonado 0180f36a62 refacgor: move cors handler to middlewares 2025-03-23 12:10:12 +01:00
sjdonado 4500c89904 fix: convert IpLookup to struct and remove reader instance 2025-03-23 11:55:34 +01:00
sjdonado 3df4642c90 chore: update README 2025-03-20 20:53:37 +01:00
sjdonado e67ed7165b feat: click controller performance improvements
- return tuple directly from the block
- avoid to parse remote_address in the main thread
- avoid to replace headers, add only
- avoid separate variables for one-single use
2025-03-20 20:43:15 +01:00
sjdonado 4ae6ef39d5 refactor: replace ClickController class with struct 2025-03-20 20:42:51 +01:00
sjdonado f2b63c00a3 chore: run benchmark 2025-03-20 13:03:15 +01:00
sjdonado 6a151301b8 refactor: replace click tracker with direct spawn 2025-03-20 13:02:38 +01:00
sjdonado d1be283318 refactor: rewrite benchmark in cr 2025-03-20 12:01:13 +01:00
sjdonado bf717dc38f fix: return inserted_link on create 2025-03-20 08:50:55 +01:00
sjdonado 73ee4c4479 fix: user id and link id int64 types 2025-03-20 08:13:07 +01:00
sjdonado 38f9cfd48e fix: replace uuid columns with rowid aliases 2025-03-20 07:34:33 +01:00
sjdonado e14fc266bb chore: cleanup 2025-03-19 06:11:04 +01:00
sjdonado 917a79c536 chore: bump version to 1.5.2 2025-03-18 11:29:48 +01:00
sjdonado 2c951fd834 chore: update API.md response status codes 2025-03-18 11:23:28 +01:00
sjdonado 3983102caa refactor: click tracker remove unused imports 2025-03-18 11:07:34 +01:00
sjdonado fba2039efc refactor: request thread safety context 2025-03-18 11:04:50 +01:00
sjdonado b22381cb7f fix: idx_links_slug_optimized avoid to duplicate id already included by rowid 2025-03-18 10:56:38 +01:00
sjdonado eb0db67358 refactor: LinkController remove unnecessary overhead 2025-03-18 09:39:39 +01:00
sjdonado 1f41d13667 refactor: ClickTracker service 2025-03-18 08:53:03 +01:00
sjdonado 222e408a16 chore: update SETUP.md dokku same network section 2025-03-18 07:58:06 +01:00
sjdonado 67c27d3056 feat: performance improvement replace spawn with Async::Future.execute 2025-03-18 07:51:51 +01:00
sjdonado 001caffba6 feat: performance improvement replace slug index with covering index 2025-03-18 07:46:18 +01:00
sjdonado 006d99a9e7 chore: update SETUP.md 2025-03-17 09:52:22 +01:00
sjdonado 6bd0d195bf refactor: link get raw query 2025-03-17 09:46:41 +01:00
sjdonado 68e00e7c85 feat: database pool_size 2025-03-17 09:40:42 +01:00
sjdonado 4aefd3ff06 chore: update docs 2025-03-17 07:49:57 +01:00
sjdonado bbc900cd05 refactor: update benchmark to use httpbin.org instead of example.com 2025-03-17 07:49:26 +01:00
sjdonado 21f53f257c chore: upgrade version to 1.5.1 2025-03-16 18:56:33 +01:00
sjdonado 8ca6a450a3 chore: update API documentation 2025-03-16 18:55:18 +01:00
sjdonado 58d8d52194 test: update cursor-based pagination test cases 2025-03-16 18:55:02 +01:00
sjdonado 7d617bbb30 feat: api links/:id/clicks endpoint 2025-03-16 18:34:46 +01:00
sjdonado cd6dfa345b feat: links all cursor pagination 2025-03-16 18:30:52 +01:00
sjdonado 1967cc2c22 fix: get remote address cloudfare proxy 2025-03-16 18:04:36 +01:00
sjdonado 60ebac7150 chore: update API docs 2025-03-16 14:22:32 +01:00
sjdonado a066b5e5ab chore: upgrade version to 1.5.0 2025-03-16 13:41:54 +01:00
sjdonado cd8e2433a5 feat(cli): --update-data download UA Parser and GeoLite2 2025-03-16 13:40:33 +01:00
sjdonado b538a379d1 chore: update README 2025-03-16 11:54:48 +01:00
sjdonado 8cab7a51ad feat: country ip lookup 2025-03-16 11:42:01 +01:00
sjdonado ece74226d4 feat: add country to clicks 2025-03-16 11:26:20 +01:00
sjdonado d26aa2f18a feat: links redirect forward client ip 2025-03-16 10:20:01 +01:00
sjdonado ce2f73dfe3 fix(ci): release pipeline determine version 2025-03-16 10:18:51 +01:00
sjdonado 30fb539289 ci: speed up build with crystal arm64 binary 2025-02-07 19:37:54 +01:00
sjdonado 66fd6db3c2 ci: docker hub release tags 2025-02-07 19:05:31 +01:00
sjdonado 93a91cd76e ci: publish github workflow separate version tags from master 2025-02-07 18:53:55 +01:00
sjdonado 1d1444234b ci: publish workflow extract tag version 2025-02-07 18:44:49 +01:00
sjdonado 49ac63210e chore: bump version 2025-02-07 17:51:55 +01:00
sjdonado 702491cb39 chore: improve documentation local development guidelines 2025-02-07 17:50:10 +01:00
Juan Rodriguez d55dbe0471 Merge pull request #4 from sjdonado/chore/documentation
Chore/documentation
2025-02-07 17:34:25 +01:00
sjdonado 55969b03b5 chore: fix docs broken links 2025-02-07 17:31:05 +01:00
sjdonado 70a036e158 chore: cleanup README 2025-02-07 17:23:10 +01:00
sjdonado 6ade7d295b chore: CONTRIBUTING and CODE_OF_CONDUCT 2025-02-07 17:22:54 +01:00
sjdonado e6ae133449 chore: separate API reference from README 2025-02-07 17:22:41 +01:00
sjdonado d3706a8778 chore: separate SETUP docs from README 2025-02-07 17:22:22 +01:00
sjdonado a271e7c35d chore: bump version 2024-11-27 22:51:52 +01:00
sjdonado a46a50b429 chore: update README with new ENV variables 2024-11-27 22:51:20 +01:00
sjdonado dc8c359bfc test: admin env variables cases 2024-11-27 22:51:18 +01:00
sjdonado dfb6b10caf feat: setup admin user via env variables 2024-11-27 22:27:17 +01:00
Juan Rodriguez 3fa30b3a32 chore: Update README.md 2024-10-27 13:01:40 +01:00
Juan Rodriguez 80ed6033d1 chore: update README.md 2024-10-27 12:56:06 +01:00
Juan Rodriguez 4640522d5d chore: refactor create_links with curl parallel 2024-10-27 12:55:38 +01:00
Juan Rodriguez 848232cc11 fix: create links pipe ipc 2024-10-27 12:01:38 +01:00
Juan Rodriguez 98dedc4494 refactor: powered_by_header kemal config 2024-10-27 11:07:59 +01:00
Juan Rodriguez e6f64ea026 chore: update benchmark with bombardier 2024-10-27 11:07:23 +01:00
Juan Rodriguez ea71d3825e fix: generate slug by user + check existing link on update 2024-07-31 22:09:24 +02:00
Juan Rodriguez afa9b33568 tests: update error messages assertions 2024-07-31 21:50:35 +02:00
Juan Rodriguez a93189411b fix: test suite drop database before all 2024-07-31 21:39:24 +02:00
Juan Rodriguez 98f103f5cf fix: url validate format 2024-07-31 21:38:57 +02:00
Juan Rodriguez 6fc48dae83 refactor: replace slug generation with CRC32 + base62 2024-07-31 21:38:36 +02:00
Juan Rodriguez d039add340 chore: bump version 2024-07-31 08:08:38 +02:00
Juan Rodriguez 0214d6f46d ci: fix publish extract version step 2024-07-31 08:08:15 +02:00
Juan Rodriguez 37e14ec2f8 refactor: sha256 slug generation 2024-07-31 08:07:08 +02:00
Juan Rodriguez a85d5a8c73 chore: bump version 2024-07-14 22:30:15 +02:00
Juan Rodriguez 80cebe3357 fix: update slug size after 2nd attempt 2024-07-14 22:30:15 +02:00
Juan Rodriguez 451a5fbf0f fix: link email validate format regex 2024-07-14 14:33:34 +02:00
Juan Rodriguez aeb6d1164b fix: remove X-Powered-By header 2024-07-14 14:33:19 +02:00
Juan Rodriguez 2f14cd82dd fix: error handling override kemal default response 2024-07-14 11:19:59 +02:00
Juan Rodriguez faedd0bc7a fix: missing errors content type 2024-07-14 09:26:38 +02:00
Juan Rodriguez 1d207fae64 fix: log level error for production env 2024-07-14 09:26:25 +02:00
Juan Rodriguez a71f345f66 refactor: APP_URL + DATABASE_URL default values 2024-07-14 08:46:29 +02:00
Juan Rodriguez 7cc6c1197f refactor: auto run migrations on startup 2024-07-14 08:46:11 +02:00
Juan Rodriguez 115bbf7366 refactor: reduce docker image size 2024-07-12 23:18:06 +02:00
Juan Rodriguez 36a06ac670 chore: update README v1.1.0 2024-07-12 21:53:32 +02:00
Juan Rodriguez 772897cb27 ci: bump version to 1.1.0 2024-07-12 21:44:59 +02:00
Juan Rodriguez 1f42798d12 chore: update READMe with rest API response examples 2024-07-12 21:44:36 +02:00
Juan Rodriguez 08302e4457 ci: fix publish docker image extract_version 2024-07-12 21:29:15 +02:00
Juan Rodriguez cd9a44fcc9 fix: link controller preload + headers validation 2024-07-12 21:28:44 +02:00
Juan Rodriguez 6532ff466e test: add missing cases for link controller integration tests 2024-07-12 21:28:25 +02:00
Juan Rodriguez 8cbed80fd0 chore: update benchmark script 2024-07-12 20:52:57 +02:00
Juan Rodriguez 0758bf4cee fix: existing_links preload clicks 2024-07-12 20:26:43 +02:00
Juan Rodriguez f5c296bee3 Merge branch 'master' of https://github.com/sjdonado/bit 2024-07-12 19:00:55 +02:00
Juan Rodriguez 8f33375b5f fix: link clicks empty array validation 2024-07-12 19:00:44 +02:00
Juan Rodriguez cebbd48237 ci: update dockerfile to include data folder 2024-07-12 10:03:37 +02:00
Juan Rodriguez 88d81ecfe3 ci: update dockerfile to include data folder 2024-07-12 08:50:14 +02:00
Juan Rodriguez 3050c2b100 feat: get link by id endpoint 2024-07-12 08:47:08 +02:00
Juan Rodriguez a2aa586dae feat: get all links clicks join 2024-07-12 07:58:21 +02:00
Juan Rodriguez ebc9c6852e ci: download regexes script 2024-07-12 07:56:19 +02:00
Juan Rodriguez a1b67b8553 fix: avoid create links duplicates 2024-07-12 07:46:58 +02:00
Juan Rodriguez dbc81796d6 feat: store click data on redirect 2024-07-12 07:46:49 +02:00
Juan Rodriguez 2f796dbdab feat: create clicks model + migration 2024-07-12 07:28:27 +02:00
Juan Rodriguez ff06e10b8f ci: github workflow extract release tag from shard.yml 2024-07-11 22:45:46 +02:00
Juan Rodriguez 050fd6f1e3 refactor: rename url-shortener to bit 2024-07-10 22:29:34 +02:00
Juan Rodriguez b0160f8127 ci: build linux/amd64,linux/arm64 docker images 2024-07-10 21:51:12 +02:00
Juan Rodriguez bac73f2a47 fix: stream is closed before call_next 2024-07-10 21:41:48 +02:00
Juan Rodriguez d7a4ce25aa chore: remove bruno (deprecated) 2024-07-10 21:32:33 +02:00
Juan Rodriguez 72654ab9fb ci: fix Generate artifact attestation subject-path 2024-05-20 19:04:39 +02:00
Juan Rodriguez be10cb1c9a refactor: update benchmark to handle more links concurrently 2024-05-20 18:43:37 +02:00
Juan Rodriguez 7cbd469842 chore: benchmark script 2024-05-20 18:43:18 +02:00
Juan Rodriguez 8c209f0036 test: services unit tests 2024-05-20 13:14:37 +02:00
Juan Rodriguez f03e0092c8 refactor: move cli methods to app service 2024-05-20 13:13:49 +02:00
Juan Rodriguez 89fa00b80b ci: missing env variables publish workflow 2024-05-20 13:07:31 +02:00
Juan Rodriguez 2c3bb8deaf chore: rename bruno folder 2024-05-20 11:46:10 +02:00
Juan Rodriguez de12095d19 ci: fix generate artifact attestation 2024-05-14 07:03:40 +02:00
Juan Rodriguez 0d053a042a Merge pull request #3 from sjdonado/refactor/upgrade-to-crystal
Refactor/upgrade to crystal
2024-05-14 06:37:19 +02:00
Juan Rodriguez be0ebd1763 test: fix Closed stream (IO::Error) 2024-05-14 06:34:29 +02:00
Juan Rodriguez aa8a84af0e ci: run tests workflow 2024-05-14 06:28:13 +02:00
Juan Rodriguez f28dfdfd89 Merge branch 'master' of https://github.com/sjdonado/url-shortener into refactor/upgrade-to-crystal 2024-05-14 06:24:30 +02:00
Juan Rodriguez a7d52797d9 ci: publish to docker hub workflow 2024-05-14 06:21:55 +02:00
Juan Rodriguez 80a91583fe ci: move binaries to path 2024-05-14 06:21:13 +02:00
Juan Rodriguez abb35019f3 ci: Dockerfile + docker-compose 2024-05-14 05:50:28 +02:00
Juan Rodriguez adbc07e605 feat: migrate script 2024-05-14 05:50:13 +02:00
Juan Rodriguez 814fd83d32 test: ping spec 2024-05-13 22:59:22 +02:00
Juan Rodriguez a8e2e971d6 test: integration link spec
- spec_helper (spec-kemal setup)
- spec_helper (micreate migration)
- create_test_link + get_test_link + delete_test_link helpers
- fix test env variables
2024-05-13 22:57:13 +02:00
Juan Rodriguez 6b21bc0cd6 feat: cli create_user + list_users + delete_user 2024-05-13 22:56:06 +02:00
Juan Rodriguez a47722cd54 refactor: Link serializer - refer attr 2024-05-13 22:54:49 +02:00
Juan Rodriguez 7f2a27ec79 feat: auth middleware
- CORS headers
- Get all links that belong to user
2024-05-13 22:35:22 +02:00
Juan Rodriguez 0ad534065c feat: user migration + model
- link belongs to user
- user api_key index
2024-05-13 22:33:38 +02:00
Juan Rodriguez 80feadfbd2 feat: delete link 2024-05-13 22:30:44 +02:00
Juan Rodriguez ded84e7fa5 feat: update link 2024-05-13 10:15:08 +02:00
Juan Rodriguez 8400e5fe71 chore: bruno collection (create link) 2024-05-13 09:30:30 +02:00
Juan Rodriguez b20417d579 refactor: error handling return exception message 2024-05-13 09:22:36 +02:00
Juan Rodriguez 9564559610 refactor: rename src folder with app 2024-05-13 08:14:47 +02:00
Juan Rodriguez e4ae0c2ac4 feat: link uuid + increase counter concurrently 2024-05-13 08:08:20 +02:00
Juan Rodriguez b34a52f3e0 feat: redirect by slug 2024-05-12 23:21:14 +02:00
Juan Rodriguez c806953b9a feat: create links - route + controller + model + migration 2024-05-12 22:50:47 +02:00
Juan Rodriguez 3e86e761b6 chore: database setup crecto + micrate 2024-05-12 22:40:11 +02:00
Juan Rodriguez b7b47b133d feat: ping route + controller 2024-05-12 21:25:57 +02:00
Juan Rodriguez db42ed2b24 feat: kemal setup 2024-05-12 18:35:45 +02:00
Juan Rodriguez 9e1fcf2d48 chore: kemal + crystal setup 2024-05-12 15:08:09 +02:00
Juan Rodriguez 720b70c6a0 chore: remove rails app 2024-05-12 15:07:56 +02:00
Juan Rodriguez a77ef21d45 Update README.md 2024-01-31 08:26:53 +01:00
Juan Rodriguez 8eb27f2c8a chore: replace sjdonado.de with donado.co 2024-01-31 05:27:46 +01:00
Juan Rodriguez 7a095fb045 ci: deploy to sjdonado.de with dokku 2023-04-19 17:07:33 +02:00
Juan Rodriguez f74ec3af20 fix: validate domains, allow only google.com + sjdonado.de for demo purposes (to avoid phishing scam) 2023-04-19 17:07:05 +02:00
Juan Rodriguez cebdfb35d7 fix: parsed_url + stripped_url
- store and redirect urls without protocol
- update README
- increment counter with SQL COALESCE
- add linksHelper
- update tests
2023-03-27 09:53:09 +02:00
Juan Rodriguez af9c7b0024 feat: meta tags 2023-03-26 21:50:19 +02:00
Juan Rodriguez 211e4f40f4 refactor: improve url regex 2023-03-26 19:37:29 +02:00
Juan Rodriguez fee04cc26d chore: remove heroku from Github actions 2023-03-26 19:03:04 +02:00
Juan Rodriguez 63acab3cf7 Dokku deploy setup (#1)
* fix: remove config.key from Dockerfile

* refactor: remove redis

* chore: update README

* feat: precompile assets on build Dockerfile
2023-03-26 19:00:09 +02:00
Juan Rodriguez 38ce72618a fix: 🐛 save sessionUsername in localStorage 2021-06-16 10:24:33 -05:00
Juan Rodriguez e3ab670eac ci: 💚 enable rubocop-rails support 2021-06-15 15:39:44 -05:00
Juan Rodriguez 73b674b613 refactor: Username and slug unique indexes, before actions for set link and user
confirm_password_validation around action, unnecessary helpers removed, rubocop-rails suggestions applied

Add username index and unique slug index migrations
2021-06-15 15:24:34 -05:00
Juan Rodriguez 9204abf2e3 refactor: 🔥 Generate slug with attempts
Link short method removed, generate_slug test added, deleted unused files
2021-06-15 12:08:01 -05:00
Juan Rodriguez 391c62e99a ci: 💚 Rubocop step added to main workflow 2021-06-15 10:48:29 -05:00
Juan Rodriguez e9e7c22bfc refactor: 📱 Responsive modals and meta tags 2021-06-15 06:51:19 -05:00
Juan Rodriguez 06dfd59753 fix: 💚 Enable public file server in production 2021-06-14 19:38:18 -05:00
Juan Rodriguez d404cbf3b8 ci: 💚 Create master.key, Dockerfile precompile assets, uglifier harmony true 2021-06-14 18:50:51 -05:00
Juan Rodriguez 33eb56f686 ci: 👷 Github actions setup, heroku deployment
Redis cache store, Production Dockerfile, Github actions workflow
2021-06-14 17:14:48 -05:00
Juan Rodriguez 54bff064d1 feat: Fecth link click_counter on come back to site
LinksController counter route, tests updated, render link created_by field, order user links by desc
2021-06-14 17:03:11 -05:00
Juan Rodriguez d134be737a test: Tests updated with branch coverage
SimpleCov consolse formatter
2021-06-14 15:12:47 -05:00
Juan Rodriguez 3feaa5d88f refactor: Modal layout and turbolinks optimization
Reload with turbolinks, error messages, confirm password validation

Login and Signup modals
2021-06-14 15:10:56 -05:00
Juan Rodriguez 9c7146820c feat: Sessions controller
Sessions helper methods, login and signup modals, load partial views with stimulus

Situmuls usersController, Create and Destroy user sessions
2021-06-14 11:46:25 -05:00
Juan Rodriguez f63be42b4c feat: Create users
Users migration, model, controller. user_id to links. signup view
2021-06-14 08:24:34 -05:00
Juan Rodriguez 3e8bdee17a feat: Generate short url view
stimulus links controller, tailwindcss setup, links controller post route, tests updated

Generate short links view
2021-06-14 00:41:53 -05:00
Juan Rodriguez e50da9b3e2 chore: 🔧 docker entrypoint for init db
docker-compose.test removed, create testing and dev databases on init db
2021-06-13 20:36:39 -05:00
Juan Rodriguez d72ad2d43b feat: Links controller
Show method, Home page, redirection validation

LinksController show method
2021-06-13 20:34:41 -05:00
Juan Rodriguez 7e81f47473 build: 🔧 Stimulus setup
webpack service added to docker-compose, rails webpacker config, application js and css tags updated

rails webpacker support
2021-06-13 16:28:45 -05:00
Juan Rodriguez d27fb94095 test: Link unit test finished
100% coverage
2021-06-13 12:28:37 -05:00
Juan Rodriguez 38d0aff7f8 build: 🔧 default url_options host added
simplecov added, docker-compose updated, test_helper updated
2021-06-13 12:27:23 -05:00
Juan Rodriguez 587f9552c8 feat: Link model
create_links and slug index migrations. generate_slug and shorten model methods

- Create links migration - Add slug index migration
2021-06-13 10:55:05 -05:00
Juan Rodriguez 441bf0919d chore: 🔧 Rubocop config file 2021-06-13 10:53:34 -05:00
Juan Rodriguez a9d3cbe544 refactor: ♻️ Rubocop auto-correct
frozen_string_literal and single quote strings
2021-06-13 10:46:52 -05:00
Juan Rodriguez 019e2516d7 feat: 🏗️ Rails setup
postgresql database
2021-06-13 09:05:20 -05:00
Juan Rodriguez a17a42e792 build: 🔨 Docker setup
Dockerfile, docker-compose and entrypoint
2021-06-13 09:01:22 -05:00
Juan Rodriguez d02df35d86 feat: 🔧 Initial commit
vscode config and TODO
2021-06-13 08:01:12 -05:00
67 changed files with 9356 additions and 62 deletions
+44
View File
@@ -0,0 +1,44 @@
.git
.gitignore
.github
/bin/
/bit
/cli
/benchmark
*.dwarf
*.o
*.a
# Dependencies cache
/.shards/
/lib/.shards/
/spec/
# Database files (should be mounted as volumes)
/sqlite/
*.db
*.db-shm
*.db-wal
# Logs and temporary files
*.log
# Documentation
/docs/
*.md
README.md
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE
DOCKER_MIGRATION.md
# Development environment
.env*
.editorconfig
# Docker files (not needed inside image)
Dockerfile
docker-compose.yml
.dockerignore
+9
View File
@@ -0,0 +1,9 @@
root = true
[*.cr]
charset = utf-8
end_of_line = lf
insert_final_newline = true
indent_style = space
indent_size = 2
trim_trailing_whitespace = true
+2
View File
@@ -0,0 +1,2 @@
DATABASE_URL=sqlite3://./sqlite/data.db?journal_mode=wal&synchronous=normal&foreign_keys=true
APP_URL=http://localhost:4000
+3
View File
@@ -0,0 +1,3 @@
PORT=4001
DATABASE_URL=sqlite3://./sqlite/data.test.db?journal_mode=wal&synchronous=normal&foreign_keys=true
APP_URL=http://localhost:4001
+33
View File
@@ -0,0 +1,33 @@
name: Deploy API Documentation
on:
push:
branches:
- master
paths:
- 'docs/openapi.yaml'
- '.github/workflows/deploy-docs.yml'
workflow_dispatch:
permissions:
contents: write
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Generate Swagger UI
uses: Legion2/swagger-ui-action@v1
with:
output: swagger-ui
spec-file: docs/openapi.yaml
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Deploy to GitHub Pages
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: swagger-ui
+51
View File
@@ -0,0 +1,51 @@
name: Publish Docker images
on:
push:
branches:
- master
release:
types: [published]
jobs:
build-and-push:
name: Build and Push Multi-Platform
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Determine version
id: version
run: |
if [ "${{ github.event_name }}" = "release" ]; then
echo "version=${{ github.event.release.tag_name }}" >> $GITHUB_OUTPUT
else
echo "version=latest" >> $GITHUB_OUTPUT
fi
- name: Build and push multi-platform image
uses: docker/build-push-action@v5
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: sjdonado/bit:${{ steps.version.outputs.version }}
cache-from: type=gha
cache-to: type=gha,mode=max
+21
View File
@@ -0,0 +1,21 @@
name: Run tests
on:
pull_request:
branches: [master]
env:
ENV: test
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Download source
uses: actions/checkout@v3
- name: Install Crystal
uses: crystal-lang/install-crystal@v1
- name: Install dependencies
run: shards install
- name: Run tests
run: crystal spec
+63
View File
@@ -0,0 +1,63 @@
name: Update Parsers
on:
schedule:
# Run every two weeks on Sunday at 00:00 UTC (1st and 3rd Sunday of each month)
- cron: '0 0 1-7,15-21 * 0'
workflow_dispatch: # Allow manual trigger
jobs:
update-parsers:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Install Crystal
uses: crystal-lang/install-crystal@v1
- name: Install dependencies
run: shards install
- name: Build CLI
run: crystal build scripts/cli.cr -o cli
- name: Update parsers
run: ./cli --update-parsers
- name: Check for changes
id: changes
run: |
if [ -n "$(git status --porcelain)" ]; then
echo "has_changes=true" >> $GITHUB_OUTPUT
else
echo "has_changes=false" >> $GITHUB_OUTPUT
fi
- name: Create Pull Request
if: steps.changes.outputs.has_changes == 'true'
uses: peter-evans/create-pull-request@v6
with:
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: 'chore: update parsers'
title: 'chore: Update UA regexes and GeoLite2 database'
body: |
## Automated Parser Update
This PR updates the following data files:
- User Agent parsing regexes
- GeoLite2 database
**Triggered by**: Scheduled workflow (runs every 2 weeks)
**Date**: ${{ github.event.repository.updated_at }}
Please review the changes and merge if everything looks good.
branch: automated/update-parsers
delete-branch: true
labels: |
dependencies
automated
+12
View File
@@ -0,0 +1,12 @@
/lib/
/bin/
/.shards/
*.dwarf
.DS_Store
/sqlite/
.env.production
*.log
bit
cli
+132
View File
@@ -0,0 +1,132 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official email address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
[INSERT CONTACT METHOD].
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series
of actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within
the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
[https://www.contributor-covenant.org/version/2/0/code_of_conduct.html][v2.0].
Community Impact Guidelines were inspired by
[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
For answers to common questions about this code of conduct, see the FAQ at
[https://www.contributor-covenant.org/faq][FAQ]. Translations are available
at [https://www.contributor-covenant.org/translations][translations].
[homepage]: https://www.contributor-covenant.org
[v2.0]: https://www.contributor-covenant.org/version/2/0/code_of_conduct.html
[Mozilla CoC]: https://github.com/mozilla/diversity
[FAQ]: https://www.contributor-covenant.org/faq
[translations]: https://www.contributor-covenant.org/translations
+75
View File
@@ -0,0 +1,75 @@
# Contributing Guidelines
We welcome contributions from the community! Please follow these guidelines to help maintain consistency and quality in the project.
## Code of Conduct
This project adheres to the [Contributor Covenant Code of Conduct](CODE_OF_CONDUCT.md). By participating, you agree to uphold its terms.
## How to Contribute
### 1. Fork the Repository
Click the "Fork" button at the top-right of the [repository page](https://github.com/sjdonado/bit).
### 2. Clone Your Fork
```bash
git clone https://github.com/YOUR_USERNAME/bit.git
cd bit
```
### 3. Create a Feature Branch
```bash
git checkout -b feat/your-feature-name
```
### or for bug fixes:
```bash
git checkout -b fix/issue-description
```
### 4. Develop Your Changes
- Check [Local Development](docs/SETUP.md#local-development) guidelines
- Ensure changes match the project scope
- Write clear commit messages
- Include tests for new functionality
- Update documentation when applicable
### 5. Commit Changes
```bash
git commit -am 'Add descriptive commit message'
```
### 6. Push to GitHub
```bash
git push origin your-branch-name
```
### 7. Create a Pull Request
1. Go to the [original repository](https://github.com/sjdonado/bit)
2. Click "New Pull Request"
3. Select your fork and branch
4. Add a clear description including:
- Purpose of changes
- Related issues (if applicable)
- Testing performed
## Pull Request Guidelines
- Keep PRs focused on a single feature/bugfix
- Ensure all tests pass
- Update documentation in the same PR
- Use descriptive titles (e.g., "Add URL validation" not "Update code")
- Reference related issues using #issue-number
## Reporting Issues
When opening an issue, please include:
1. Description of the problem
2. Steps to reproduce
3. Expected vs actual behavior
4. Environment details (OS, Crystal version, etc)
For feature requests:
- Explain the problem you're trying to solve
- Suggest potential implementations
## License
By contributing, you agree that your contributions will be licensed under the [license](LICENSE).
+43
View File
@@ -0,0 +1,43 @@
FROM alpine:edge AS build
ENV ENV=production
WORKDIR /usr/src/app
RUN apk add --no-cache \
crystal \
shards \
openssl-dev \
yaml-dev \
sqlite-dev \
libevent-dev \
tzdata
COPY . .
RUN shards install --production
RUN shards build --release --no-debug --progress --stats
FROM alpine:latest AS runtime
ENV ENV=production
WORKDIR /usr/src/app
RUN apk add --no-cache \
gc-dev \
pcre2 \
libevent \
sqlite-libs \
openssl \
yaml \
gmp \
libgcc \
tzdata
RUN mkdir -p sqlite
COPY --from=build /usr/src/app/db db
COPY --from=build /usr/src/app/data data
COPY --from=build /usr/src/app/bin /usr/local/bin
EXPOSE 4000/tcp
CMD ["bit"]
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2024 Juan Rodriguez <sjdonado@icloud.com>
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+93
View File
@@ -0,0 +1,93 @@
[![Docker Pulls](https://img.shields.io/docker/pulls/sjdonado/bit.svg)](https://hub.docker.com/r/sjdonado/bit)
[![Docker Image Size](https://img.shields.io/docker/image-size/sjdonado/bit/latest)](https://hub.docker.com/r/sjdonado/bit)
## Features
- Minimal tracking setup: Country, browser, OS, referer. No cookies or persistent tracking mechanisms are used beyond what's available from a basic client's request.
- Includes `X-Forwarded-For` header.
- Multiple users are supported via API key authentication. Create, list and delete keys via the [CLI](docs/SETUP.md#cli).
- Easy to extend, Ruby on Rails inspired setup.
- Auto update UA regexes and GeoLite2 database.
## Why bit?
**Fast:** **11k req/sec**, latency 11ms, 40MiB avg memory usage (100k requests using 125 connections, [benchmark](docs/SETUP.md#benchmark)).
**Lightweight:** Minimal dependencies, image size under 20 MiB, memory usage under 60 MiB at peak.
**Self-hosted:** [Dokku](docs/SETUP.md#dokku), [Docker Compose](docs/SETUP.md#docker-compose).
**Production ready:** Feature-complete by design, simple and reliable without unnecessary bloat. Bug fixes will continue, but new features aren't planned.
## Run It Anywhere
All images available on [Docker Hub](https://hub.docker.com/r/sjdonado/bit/tags).
### Docker
```bash
docker run \
--name bit \
-p 4000:4000 \
-e ENV="production" \
-e DATABASE_URL="sqlite3://./sqlite/data.db" \
-e APP_URL="http://localhost:4000" \
-e ADMIN_NAME="Admin" \
-e ADMIN_API_KEY=$(openssl rand -base64 32) \
sjdonado/bit
# Create a new user
# docker exec -it bit cli --create-user=Admin
```
### Docker Compose
```bash
docker-compose up
# Optional: Generate an api key
# docker-compose exec -it app cli --create-user=Admin
```
### Dokku
- Dockerfile
```dockerfile
FROM sjdonado/bit
```
- Over ssh
```bash
dokku apps:create bit
dokku domains:set bit bit.yourdomain.com
dokku letsencrypt:enable bit
dokku storage:ensure-directory bit-sqlite
dokku storage:mount bit /var/lib/dokku/data/storage/bit-sqlite:/usr/src/app/sqlite/
dokku config:set bit DATABASE_URL="sqlite3://./sqlite/data.db" APP_URL=https://bit.yourdomain.com ADMIN_NAME=Admin ADMIN_API_KEY=$(openssl rand -base64 32)
dokku ports:add bit http:80:4000
dokku ports:add bit https:443:4000
# Create a new user
# dokku run bit cli --create-user=Admin
```
### Dokku (subnetwork)
Recommended for lower latency communication (no host network traversal)
```bash
dokku network:create bit-net
dokku network:set bit attach-post-create bit-net
dokku network:set myapp attach-post-create bit-net
```
## Documentation
- [API Reference](https://sjdonado.github.io/bit/)
- [Local Development](docs/SETUP.md)
## Contributing
Found an issue or have a suggestion? Please follow our [contribution guidelines](CONTRIBUTING.md).
+15
View File
@@ -0,0 +1,15 @@
require "log"
ENV["ENV"] ||= "development"
ENV["APP_URL"] ||= "http://localhost:4000"
ENV["DATABASE_URL"] ||= "sqlite3://./sqlite/data.db?journal_mode=wal&synchronous=normal&foreign_keys=true
"
{% if env("ENV") != "production" %}
require "dotenv"
Dotenv.load ".env.#{ENV["ENV"]}" # File must exist in non-production!
{% end %}
{% if env("ENV") == "production" %}
Log.setup(:error)
{% end %}
+7
View File
@@ -0,0 +1,7 @@
require "kemal"
Kemal.config.env = ENV["ENV"]? || "development"
Kemal.config.port = ENV["PORT"]?.try(&.to_i) || 4000
Kemal.config.host_binding = ENV["HOST"]? || "0.0.0.0"
Kemal.config.powered_by_header = false
+45
View File
@@ -0,0 +1,45 @@
module App::Controllers
struct ClickController
include App::Models
include App::Lib
include App::Services
def self.redirect_handler
->(env : HTTP::Server::Context) {
link_id, url = Database.raw_query("SELECT id, url FROM links WHERE slug = (?) LIMIT 1", env.params.url["slug"]) do |result|
result.move_next ? {result.read(Int64), result.read(String)} : nil
end || raise App::NotFoundException.new(env)
remote_address = env.request.headers["Cf-Connecting-Ip"]? || env.request.remote_address.to_s
# Send redirect immediately
env.response.status_code = 301
env.response.headers.add("Location", url)
env.response.headers.add("X-Forwarded-For", remote_address)
if user_agent = env.request.headers["User-Agent"]?
env.response.headers.add("User-Agent", user_agent)
end
# non-blocking click proccessing
spawn do
begin
client_ip = IpLookup.ip_from_address(remote_address)
family, _, _, os = UserAgent.parse(env.request.headers["User-Agent"]? || "")
click = App::Models::Click.new
click.link_id = link_id
click.country = client_ip ? IpLookup.country(client_ip) : nil
click.user_agent = env.request.headers["User-Agent"]?
click.browser = family
click.os = os.try &.[0]
click.referer = env.request.headers["Referer"]?.try { |r| URI.parse(r).host rescue r } || env.params.query["utm_source"]? || "Direct"
Database.insert(click)
rescue ex
Log.error { "Click tracking error: #{ex.message}" }
end
end
}
end
end
end
+153
View File
@@ -0,0 +1,153 @@
module App::Controllers
class LinkController < App::Lib::BaseController
include App::Models
include App::Lib
include App::Services
def initialize(@env : HTTP::Server::Context)
super(@env)
end
def create
body = parse_body(["url"])
url = body["url"].to_s
query = Database::Query.where(url: url, user_id: current_user_id).limit(1)
existing_link = Database.all(Link, query, preload: [:clicks]).first?
if existing_link
return render_json({"data" => App::Serializers::Link.new(existing_link)})
end
link = Link.new
link.url = url
link.user_id = current_user_id
link.slug = SlugService.shorten_url(url, current_user_id)
changeset = Database.insert(link)
if !changeset.valid?
raise App::UnprocessableEntityException.new(@env, map_changeset_errors(changeset.errors))
end
inserted_link = Database.get!(Link, changeset.instance.id)
render_json({"data" => App::Serializers::Link.new(inserted_link)}, 201)
end
def list_all
limit, cursor = pagination_params
query = Database::Query.where(user_id: current_user_id)
query = query.where("id < ?", cursor) if cursor
query = query.order_by("id DESC").limit(limit + 1)
links = Database.all(Link, query)
paginated_response(links, limit) { |link| App::Serializers::Link.new(link) }
end
def get
link_id = @env.params.url["id"].to_i64
query = Database::Query.where(id: link_id, user_id: current_user_id).limit(1)
link = Database.all(Link, query).first?
raise App::NotFoundException.new(@env) if link.nil?
clicks_query = Database::Query.where(link_id: link_id)
.order_by("id DESC")
.limit(100)
link.clicks = Database.all(Click, clicks_query)
render_json({"data" => App::Serializers::Link.new(link)})
end
def list_clicks
link_id = @env.params.url["id"].to_i64
# Verify link exists and belongs to user
link_query = Database::Query.where(id: link_id, user_id: current_user_id).limit(1)
link = Database.all(Link, link_query).first?
raise App::NotFoundException.new(@env) if link.nil?
limit, cursor = pagination_params
query = Database::Query.where(link_id: link_id)
query = query.where("id < ?", cursor) if cursor
query = query.order_by("id DESC").limit(limit + 1)
clicks = Database.all(Click, query)
paginated_response(clicks, limit) { |click| App::Serializers::Click.new(click) }
end
def update
id = @env.params.url["id"].to_i64
body = parse_body(["url"])
new_url = body["url"].to_s
query = Database::Query.where(id: id).limit(1)
link = Database.all(Link, query, preload: [:clicks]).first?
raise App::NotFoundException.new(@env) if link.nil?
raise App::ForbiddenException.new(@env) if link.user_id != current_user_id
# Check for existing URL
existing_query = Database::Query.where(url: new_url, user_id: current_user_id).limit(1)
if Database.all(Link, existing_query).first?
raise App::UnprocessableEntityException.new(@env, { "url" => ["URL already exists"] })
end
link.url = new_url
link.slug = SlugService.shorten_url(new_url, current_user_id)
changeset = Database.update(link)
if !changeset.valid?
raise App::UnprocessableEntityException.new(@env, map_changeset_errors(changeset.errors))
end
render_json({"data" => App::Serializers::Link.new(link)})
end
def delete
id = @env.params.url["id"].to_i64
link = Database.get(Link, id)
raise App::NotFoundException.new(@env) if !link
raise App::ForbiddenException.new(@env) if link.user_id != current_user_id
result = Database.raw_exec("DELETE FROM links WHERE id = (?)", link.id)
if result.rows_affected == 0
raise App::UnprocessableEntityException.new(@env, { "id" => ["Row delete failed"] })
end
@env.response.status_code = 204
end
private def current_user : User
@env.get("user").as(User)
end
private def current_user_id : Int64
current_user.id.as(Int64)
end
private def pagination_params
limit = (@env.params.query["limit"]? || "100").to_i32
cursor = @env.params.query["cursor"]?
{limit, cursor}
end
private def paginated_response(items, limit)
has_more = items.size > limit
items = items[0...limit] if has_more
next_cursor = has_more ? items.last.id : nil
render_json({
"data" => items.map { |item| yield item },
"pagination" => {
"has_more" => has_more,
"next" => next_cursor
}
})
end
end
end
+13
View File
@@ -0,0 +1,13 @@
require "../lib/controller.cr"
module App::Controllers
class PingController < App::Lib::BaseController
def initialize(@env : HTTP::Server::Context)
super(@env)
end
def ping
render_json({data: "pong"})
end
end
end
+45
View File
@@ -0,0 +1,45 @@
module App::Lib
abstract class BaseController
protected getter env : HTTP::Server::Context
def initialize(@env : HTTP::Server::Context); end
# Convert changeset errors to API-friendly format
protected def map_changeset_errors(errors)
errors.reduce({} of String => Array(String)) do |memo, error|
field = error[:field].to_s
message = error[:message].to_s
memo[field] ||= [] of String
memo[field] << message
memo
end
end
protected def parse_body(required_fields : Array(String) = [] of String)
json_params = @env.params.json.try(&.to_h) || {} of String => JSON::Any
json_params = json_params.transform_values(&.to_s) # Convert JSON::Any to String
missing_fields = required_fields.reject { |field| json_params.has_key?(field) }
unless missing_fields.empty?
error_message = "#{missing_fields.first}: Required field"
raise App::BadRequestException.new(@env, error_message)
end
json_params
end
protected def render_json(data, status_code : Int32 = 200)
@env.response.status_code = status_code
@env.response.content_type = "application/json"
data.to_json
end
protected def param(key : String) : String
@env.params.url[key]
rescue KeyError
raise App::BadRequestException.new(@env, "Missing required parameter: #{key}")
end
end
end
+34
View File
@@ -0,0 +1,34 @@
require "sqlite3"
require "crecto"
require "micrate"
module App::Lib
class Database
extend Crecto::Repo
Query = Crecto::Repo::Query
config do |conf|
base_url = ENV["DATABASE_URL"]
separator = base_url.includes?("?") ? "&" : "?"
db_url = base_url + separator +
"&journal_mode=WAL" +
"&synchronous=NORMAL" + # Better performance with reasonable safety
"&foreign_keys=true"
conf.uri = db_url
end
if ENV["ENV"] == "development"
Crecto::DbLogger.set_handler(STDOUT)
end
def self.run_migrations
Micrate::DB.connection_url = ENV["DATABASE_URL"]
Micrate::Cli.run_up
end
run_migrations
end
end
+57
View File
@@ -0,0 +1,57 @@
require "kemal"
module App
class InternalServerErrorException < Kemal::Exceptions::CustomException
def initialize(context)
context.response.content_type = "application/json"
context.response.status_code = 500
context.response.print({ "error" => "Internal Server Error" }.to_json)
super(context)
end
end
class BadRequestException < Kemal::Exceptions::CustomException
def initialize(context, message : String)
context.response.content_type = "application/json"
context.response.status_code = 400
context.response.print({ "error" => message }.to_json)
super(context)
end
end
class UnauthorizedException < Kemal::Exceptions::CustomException
def initialize(context)
context.response.content_type = "application/json"
context.response.status_code = 401
context.response.print({ "error" => "Unauthorized access" }.to_json)
super(context)
end
end
class ForbiddenException < Kemal::Exceptions::CustomException
def initialize(context)
context.response.content_type = "application/json"
context.response.status_code = 403
context.response.print({ "error" => "Access not allowed" }.to_json)
super(context)
end
end
class NotFoundException < Kemal::Exceptions::CustomException
def initialize(context)
context.response.content_type = "application/json"
context.response.status_code = 404
context.response.print({ "error" => "Resource not found" }.to_json)
super(context)
end
end
class UnprocessableEntityException < Kemal::Exceptions::CustomException
def initialize(context, message : Hash(String, Array(String)))
context.response.content_type = "application/json"
context.response.status_code = 422
context.response.print({ "errors" => message }.to_json)
super(context)
end
end
end
+45
View File
@@ -0,0 +1,45 @@
require "maxminddb"
require "log"
module App::Lib
struct IpLookup
MMDB_PATH = "data/GeoLite2-Country.mmdb"
@@reader : MaxMindDB::Reader? = nil
@@reader_mutex = Mutex.new
private def self.get_reader : MaxMindDB::Reader
@@reader_mutex.synchronize do
@@reader ||= MaxMindDB.open(MMDB_PATH)
end
end
def self.country(ip_address : String) : String?
return nil if ip_address == "Unknown" || ip_address.empty?
begin
lookup = get_reader.get(ip_address)
lookup["country"]?.try &.["iso_code"]?.try &.as_s
rescue ex
Log.error { "IP lookup failed: #{ex.message}" }
nil
end
end
def self.ip_from_address(address_string : String?) : String?
return nil if address_string.nil?
if address_string.includes?('[') # IPv6 with port: [2001:db8::1]:8080
address_string.split(']').first.sub('[', '\'')
elsif address_string.includes?(':')
if address_string.count(':') > 1 # IPv6 without port
address_string
else # IPv4 with port: 192.168.1.1:8080
address_string.split(':').first
end
else # Address without port
address_string
end
end
end
end
+123
View File
@@ -0,0 +1,123 @@
require "yaml"
require "semantic_version"
module App::Lib
struct UserAgent
REGEXES_PATH = "data/uap_core_regexes.yaml"
@@regexes_cache : YAML::Any? = nil
@@compiled_regexes = {} of String => Array(Tuple(Regex, YAML::Any))
@@mutex = Mutex.new
private def self.load_regexes
@@mutex.synchronize do
if @@regexes_cache.nil?
begin
regexes_yaml = File.read(REGEXES_PATH)
@@regexes_cache = YAML.parse(regexes_yaml)
# Pre-compile all regexes for better performance
["user_agent_parsers", "os_parsers", "device_parsers"].each do |parser_type|
@@compiled_regexes[parser_type] = [] of Tuple(Regex, YAML::Any)
@@regexes_cache.not_nil![parser_type].as_a.each do |parser|
regex_str = parser["regex"].as_s
options = parser["regex_flag"]?.try(&.as_s) == "i" ?
Regex::Options::IGNORE_CASE : Regex::Options::None
begin
compiled_regex = Regex.new(regex_str, options)
@@compiled_regexes[parser_type] << {compiled_regex, parser}
rescue
# Skip invalid regexes
end
end
end
rescue ex
# If loading fails, set an empty cache to prevent repeated failures
@@regexes_cache = YAML.parse("{}")
@@compiled_regexes = {} of String => Array(Tuple(Regex, YAML::Any))
end
end
end
end
def self.parse(user_agent_string : String)
return {nil, nil, nil, nil} if user_agent_string.empty?
# Load regexes only once and cache them
load_regexes
family = nil
version = nil
device = nil
os = nil
@@compiled_regexes["user_agent_parsers"]?.try &.each do |regex_tuple|
regex, parser = regex_tuple
match = regex.match(user_agent_string)
next unless match
family = match[1]? || nil
v1 = (match[2]? || "0").to_i
v2 = (match[3]? || "0").to_i
v3 = (match[4]? || "0").to_i
# Apply replacements if defined
if replacement = parser["family_replacement"]?
family = replacement.as_s.gsub("$1", family.to_s)
end
version = SemanticVersion.new(v1, v2, v3)
break
end
@@compiled_regexes["os_parsers"]?.try &.each do |regex_tuple|
regex, parser = regex_tuple
match = regex.match(user_agent_string)
next unless match
os_family = match[1]? || nil
os_v1 = (match[2]? || "0").to_i
os_v2 = (match[3]? || "0").to_i
os_v3 = (match[4]? || "0").to_i
# Apply replacements if defined
if replacement = parser["os_replacement"]?
os_family = replacement.as_s.gsub("$1", os_family.to_s)
end
os = {os_family, SemanticVersion.new(os_v1, os_v2, os_v3)}
break
end
@@compiled_regexes["device_parsers"]?.try &.each do |regex_tuple|
regex, parser = regex_tuple
match = regex.match(user_agent_string)
next unless match
model = match[1]? || nil
device_name = model
brand = nil
# Apply replacements if defined
if device_replacement = parser["device_replacement"]?
device_name = device_replacement.as_s.gsub("$1", device_name.to_s)
end
if model_replacement = parser["model_replacement"]?
model = model_replacement.as_s.gsub("$1", model.to_s)
end
if brand_replacement = parser["brand_replacement"]?
brand = brand_replacement.as_s
end
device = {model, brand, device_name}
break
end
{family, version, device, os}
end
end
end
+19
View File
@@ -0,0 +1,19 @@
module App::Middlewares
class Auth < Kemal::Handler
include App::Models
include App::Lib
exclude ["/api/ping", "/:slug"]
def call(env)
return call_next(env) if exclude_match?(env)
begin
user = Database.get_by!(User, api_key: env.request.headers["X-Api-Key"])
env.set "user", user
rescue exception
raise App::UnauthorizedException.new(env)
end
call_next(env)
end
end
end
+29
View File
@@ -0,0 +1,29 @@
module App::Middlewares
class CORSHandler < Kemal::Handler
exclude ["/api/ping", "/:slug"]
def initialize(
@allow_origin = "*",
@allow_methods = "GET, POST, PUT, DELETE, OPTIONS",
@allow_headers = "Content-Type, Accept, Origin, X-Api-Key"
)
end
def call(env)
return call_next(env) if exclude_match?(env)
env.response.headers["Access-Control-Allow-Origin"] = @allow_origin
env.response.headers["Access-Control-Allow-Methods"] = @allow_methods
env.response.headers["Access-Control-Allow-Headers"] = @allow_headers
if env.request.method == "OPTIONS"
env.response.status_code = 200
env.response.content_type = "text/plain"
env.response.print("")
return env
end
call_next(env)
end
end
end
+18
View File
@@ -0,0 +1,18 @@
require "crecto"
module App::Models
class Click < Crecto::Model
schema :clicks do
field :id, Int64, primary_key: true
field :user_agent, String
field :country, String
field :browser, String
field :os, String
field :referer, String
belongs_to :link, Link
end
validate_required [:user_agent, :referer]
end
end
+22
View File
@@ -0,0 +1,22 @@
require "sqlite3"
require "crecto"
require "./user.cr"
module App::Models
class Link < Crecto::Model
schema :links do
field :id, Int64, primary_key: true
field :slug, String
field :url, String
belongs_to :user, User
has_many :clicks, Click
end
unique_constraint :slug
validate_required [:slug, :url]
validate_format :url, /\A(?:(https?:\/\/)?(?:[\w-]+\.)+[a-z]{2,})(?::\d+)?(?:[\/?#]\S*)?\z/i
end
end
+16
View File
@@ -0,0 +1,16 @@
require "sqlite3"
require "crecto"
module App::Models
class User < Crecto::Model
schema :users do
field :id, Int64, primary_key: true
field :name, String
field :api_key, String
end
validate_required [:name, :api_key]
has_many :links, Link, dependent: :destroy
end
end
+44
View File
@@ -0,0 +1,44 @@
require "./controllers/**"
require "kemal"
add_handler App::Middlewares::CORSHandler.new
add_handler App::Middlewares::Auth.new
module App
get "/:slug", &App::Controllers::ClickController.redirect_handler
# Namespace /api
get "/api/ping" do |env|
Controllers::PingController.new(env).ping
end
get "/api/links" do |env|
Controllers::LinkController.new(env).list_all
end
get "/api/links/:id" do |env|
Controllers::LinkController.new(env).get
end
get "/api/links/:id/clicks" do |env|
Controllers::LinkController.new(env).list_clicks
end
post "/api/links" do |env|
Controllers::LinkController.new(env).create
end
put "/api/links/:id" do |env|
Controllers::LinkController.new(env).update
end
delete "/api/links/:id" do |env|
Controllers::LinkController.new(env).delete
end
error 500 do |env|
App::InternalServerErrorException.new(env)
""
end
end
+22
View File
@@ -0,0 +1,22 @@
require "json"
require "../models/click"
module App::Serializers
class Click
def initialize(@click : App::Models::Click)
end
def to_json(builder : JSON::Builder)
builder.object do
builder.field("id", @click.id)
builder.field("user_agent", @click.user_agent)
builder.field("country", @click.country)
builder.field("browser", @click.browser)
builder.field("os", @click.os)
builder.field("referer", @click.referer)
builder.field("created_at", @click.created_at)
end
end
end
end
+31
View File
@@ -0,0 +1,31 @@
require "json"
require "../models/link"
require "./click"
module App::Serializers
class Link
getter refer
def initialize(@link : App::Models::Link)
@refer = "#{ENV["APP_URL"]}/#{@link.slug}"
end
def to_json(builder : JSON::Builder)
builder.object do
builder.field("id", @link.id)
builder.field("refer", @refer)
builder.field("origin", @link.url)
begin
clicks = @link.clicks
unless clicks.empty?
builder.field("clicks", clicks.map { |click| App::Serializers::Click.new(click) })
end
rescue Crecto::AssociationNotLoaded
# Association not loaded, skip this field silently
end
end
end
end
end
+140
View File
@@ -0,0 +1,140 @@
require "file_utils"
require "http/client"
require "../config/*"
require "../lib/*"
require "../models/*"
module App::Services::Cli
def self.create_user(name, api_key = nil)
user = App::Models::User.new
user.name = name
user.api_key = api_key || Random::Secure.urlsafe_base64()
changeset = App::Lib::Database.insert(user)
return changeset.errors unless changeset.valid?
"New user created: Name: #{user.name}, X-Api-Key: #{user.api_key}"
end
def self.list_users
users = App::Lib::Database.all(App::Models::User)
return "No users found " if users.empty?
output = "Users:\n"
users.each do |user|
output += "ID: #{user.id}, Name: #{user.name}, X-Api-Key: #{user.api_key}\n"
end
output
end
def self.delete_user(user_id)
result = App::Lib::Database.raw_exec("DELETE FROM users WHERE id = (?)", user_id) # tempfix: Database.delete does not work
return "Failed to delete user: #{result}" if result.rows_affected == 0
"User with ID #{user_id} deleted successfully"
end
def self.setup_admin_user
admin_name = ENV["ADMIN_NAME"]?
admin_api_key = ENV["ADMIN_API_KEY"]?
if admin_name && admin_api_key
query = App::Lib::Database::Query.where(name: admin_name, api_key: admin_api_key).limit(1)
existing_user = App::Lib::Database.all(App::Models::User, query).first?
return if existing_user
puts "Admin user setup detected. Creating admin user..."
result = create_user(admin_name, admin_api_key)
puts result
else
puts "Admin setup skipped: Missing ADMIN_NAME or ADMIN_API_KEY environment variables."
end
end
def self.update_uap_regexes
puts "Downloading User-Agent Parser core regexes..."
FileUtils.mkdir_p("data")
url = "https://raw.githubusercontent.com/ua-parser/uap-core/master/regexes.yaml"
output_file = "data/uap_core_regexes.yaml"
begin
http_get_with_redirect(url) do |response|
File.write(output_file, response.body_io.gets_to_end)
end
puts "User-Agent regexes downloaded to #{output_file}"
rescue e
puts "Error: Failed to download UAP core regexes: #{e.message}"
end
end
def self.update_geolite_db
puts "Downloading GeoLite2 Country database..."
FileUtils.mkdir_p("data")
url = "https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-Country.mmdb"
output_file = "data/GeoLite2-Country.mmdb"
begin
File.open(output_file, "wb") do |file|
http_get_with_redirect(url) do |response|
IO.copy(response.body_io, file)
end
end
puts "GeoLite2 database downloaded to #{output_file}"
rescue e
puts "Error: Failed to download GeoLite2 database: #{e.message}"
end
end
private def self.http_get_with_redirect(url : String, max_redirects = 5)
redirects = 0
while redirects < max_redirects
uri = URI.parse(url)
client = HTTP::Client.new(uri)
success = false
follow_redirect = false
redirect_url = nil
begin
client.get(uri.request_target) do |response|
case response.status_code
when 200
yield response
success = true
when 301, 302
if new_location = response.headers["Location"]?
puts "Following redirect to: #{new_location}"
redirect_url = new_location
follow_redirect = true
else
raise "Received redirect status but no Location header"
end
else
raise "Failed request with status code: #{response.status_code}"
end
end
ensure
client.close
end
return if success
if follow_redirect && redirect_url
url = redirect_url
redirects += 1
else
break
end
end
raise "Too many redirects (#{max_redirects})"
end
end
+12
View File
@@ -0,0 +1,12 @@
require "digest"
require "base64"
module App::Services::SlugService
def self.shorten_url(url : String, user_id : Int64) : String
combined = "#{user_id}-#{url}"
crc32_hash = Digest::CRC32.digest(combined)
base62_encoded = Base64.urlsafe_encode(crc32_hash).strip.tr("-_=", "")
base62_encoded
end
end
+15
View File
@@ -0,0 +1,15 @@
require "kemal"
require "./app/config/*"
require "./app/lib/*"
require "./app/models/*"
require "./app/serializers/*"
require "./app/middlewares/*"
require "./app/services/*"
require "./app/routes"
add_context_storage_type(App::Models::User)
App::Services::Cli.setup_admin_user
Kemal.run
Binary file not shown.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,16 @@
-- +micrate Up
-- SQL in section 'Up' is executed when this migration is applied
CREATE TABLE links (
id TEXT PRIMARY KEY NOT NULL,
user_id TEXT NOT NULL,
slug VARCHAR(4) UNIQUE NOT NULL,
url TEXT NOT NULL,
created_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);
-- +micrate Down
-- SQL section 'Down' is executed when this migration is rolled back
DROP TABLE links;
@@ -0,0 +1,7 @@
-- +micrate Up
-- SQL in section 'Up' is executed when this migration is applied
CREATE INDEX IF NOT EXISTS index_links_slug ON links (slug);
-- +micrate Down
-- SQL in section 'Down' is executed when this migration is rolled back
DROP INDEX IF EXISTS index_links_slug;
@@ -0,0 +1,13 @@
-- +micrate Up
-- SQL in section 'Up' is executed when this migration is applied
CREATE TABLE users (
id TEXT PRIMARY KEY NOT NULL,
name VARCHAR(100) NOT NULL,
api_key VARCHAR(64) UNIQUE NOT NULL,
created_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL
);
-- +micrate Down
-- SQL section 'Down' is executed when this migration is rolled back
DROP TABLE users;
@@ -0,0 +1,7 @@
-- +micrate Up
-- SQL in section 'Up' is executed when this migration is applied
CREATE INDEX IF NOT EXISTS index_users_api_key ON users (api_key);
-- +micrate Down
-- SQL section 'Down' is executed when this migration is rolled back
DROP INDEX IF EXISTS index_users_api_key;
@@ -0,0 +1,18 @@
-- +micrate Up
-- SQL in section 'Up' is executed when this migration is applied
CREATE TABLE clicks (
id TEXT PRIMARY KEY NOT NULL,
link_id TEXT NOT NULL,
user_agent TEXT,
browser TEXT,
os TEXT,
source TEXT,
created_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL,
FOREIGN KEY (link_id) REFERENCES links(id) ON DELETE CASCADE
);
-- +micrate Down
-- SQL section 'Down' is executed when this migration is rolled back
DROP TABLE clicks;
@@ -0,0 +1,49 @@
-- +micrate Up
-- SQL in section 'Up' is executed when this migration is applied
-- Step 1: Create a new table with the desired column type
CREATE TABLE links_new (
id TEXT PRIMARY KEY NOT NULL,
user_id TEXT NOT NULL,
slug VARCHAR(8) UNIQUE NOT NULL,
url TEXT NOT NULL,
created_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);
-- Step 2: Copy data from the old table to the new table
INSERT INTO links_new (id, user_id, slug, url, created_at, updated_at)
SELECT id, user_id, slug, url, created_at, updated_at FROM links;
-- Step 3: Drop the old table
DROP TABLE links;
-- Step 4: Rename the new table to the old table's name
ALTER TABLE links_new RENAME TO links;
-- +micrate Down
-- SQL section 'Down' is executed when this migration is rolled back
-- Step 1: Create a new table with the original column type
CREATE TABLE links_old (
id TEXT PRIMARY KEY NOT NULL,
user_id TEXT NOT NULL,
slug VARCHAR(4) UNIQUE NOT NULL,
url TEXT NOT NULL,
created_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);
-- Step 2: Copy data from the current table to the old table
INSERT INTO links_old (id, user_id, slug, url, created_at, updated_at)
SELECT id, user_id, substr(slug, 1, 4), url, created_at, updated_at FROM links;
-- Step 3: Drop the current table
DROP TABLE links;
-- Step 4: Rename the old table to the current table's name
ALTER TABLE links_old RENAME TO links;
@@ -0,0 +1,8 @@
-- +micrate Up
-- SQL in section 'Up' is executed when this migration is applied
ALTER TABLE clicks ADD COLUMN country TEXT;
ALTER TABLE clicks RENAME COLUMN source TO referer;
-- +micrate Down
-- SQL section 'Down' is executed when this migration is rolled back
ALTER TABLE clicks RENAME COLUMN referer TO source;
@@ -0,0 +1,13 @@
-- +micrate Up
-- SQL in section 'Up' is executed when this migration is applied
UPDATE clicks SET user_agent = NULL WHERE user_agent = 'Unknown';
UPDATE clicks SET browser = NULL WHERE browser = 'Unknown';
UPDATE clicks SET os = NULL WHERE os = 'Unknown';
UPDATE clicks SET referer = NULL WHERE referer = 'Unknown';
-- +micrate Down
-- SQL section 'Down' is executed when this migration is rolled back
UPDATE clicks SET user_agent = 'Unknown' WHERE user_agent IS NULL;
UPDATE clicks SET browser = 'Unknown' WHERE browser IS NULL;
UPDATE clicks SET os = 'Unknown' WHERE os IS NULL;
UPDATE clicks SET referer = 'Unknown' WHERE referer IS NULL;
@@ -0,0 +1,9 @@
-- +micrate Up
-- SQL in section 'Up' is executed when this migration is applied
DROP INDEX IF EXISTS idx_links_slug; -- Remove old composite index
CREATE INDEX IF NOT EXISTS idx_links_slug_optimized ON links (slug, url);
-- +micrate Down
-- SQL in section 'Down' is executed when this migration is rolled back
DROP INDEX IF EXISTS idx_links_slug_optimized;
CREATE INDEX IF NOT EXISTS idx_links_slug ON links (id, slug, url);
@@ -0,0 +1,102 @@
-- +micrate Up
-- SQL in section 'Up' is executed when this migration is applied
-- 1. Create new users table with INTEGER PK
CREATE TABLE users_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name VARCHAR(100) NOT NULL,
api_key VARCHAR(64) UNIQUE NOT NULL,
created_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL
);
-- Create a mapping table to track old and new user IDs
CREATE TEMPORARY TABLE user_id_map (
old_id TEXT,
new_id INTEGER
);
-- Insert users data and capture the mappings
INSERT INTO users_new (name, api_key, created_at, updated_at)
SELECT name, api_key, created_at, updated_at FROM users;
INSERT INTO user_id_map
SELECT u.id, u_new.id
FROM users u
JOIN users_new u_new ON u_new.api_key = u.api_key;
-- 2. Create new links table with INTEGER PK
CREATE TABLE links_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
slug VARCHAR(8) UNIQUE NOT NULL,
url TEXT NOT NULL,
created_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL,
FOREIGN KEY (user_id) REFERENCES users_new(id) ON DELETE CASCADE
);
-- Create a mapping table for links
CREATE TEMPORARY TABLE link_id_map (
old_id TEXT,
new_id INTEGER
);
-- Insert links data with new user_id foreign keys
INSERT INTO links_new (user_id, slug, url, created_at, updated_at)
SELECT
(SELECT new_id FROM user_id_map WHERE old_id = l.user_id),
l.slug,
l.url,
l.created_at,
l.updated_at
FROM links l;
-- Create the mapping for links
INSERT INTO link_id_map
SELECT l.id, l_new.id
FROM links l
JOIN links_new l_new ON l_new.slug = l.slug AND l_new.url = l.url;
-- 3. Create new clicks table with INTEGER PK
CREATE TABLE clicks_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
link_id INTEGER NOT NULL,
user_agent TEXT,
browser TEXT,
os TEXT,
referer TEXT,
country TEXT,
created_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at INTEGER DEFAULT CURRENT_TIMESTAMP NOT NULL,
FOREIGN KEY (link_id) REFERENCES links_new(id) ON DELETE CASCADE
);
-- Insert clicks data with new link_id foreign keys
INSERT INTO clicks_new (link_id, user_agent, browser, os, referer, country, created_at, updated_at)
SELECT
(SELECT new_id FROM link_id_map WHERE old_id = c.link_id),
c.user_agent,
c.browser,
c.os,
c.referer,
c.country,
c.created_at,
c.updated_at
FROM clicks c;
-- 4. Drop old tables and rename new tables
DROP TABLE clicks;
DROP TABLE links;
DROP TABLE users;
ALTER TABLE clicks_new RENAME TO clicks;
ALTER TABLE links_new RENAME TO links;
ALTER TABLE users_new RENAME TO users;
-- 5. Drop unused indexes
DROP INDEX IF EXISTS index_users_api_key;
DROP INDEX IF EXISTS idx_links_slug;
DROP INDEX IF EXISTS idx_links_slug_optimized;
-- +micrate Down
-- SQL section 'Down' is executed when this migration is rolled back
+69
View File
@@ -0,0 +1,69 @@
INSERT INTO users (name, api_key)
VALUES
('User 1', 'secure_api_key_1'),
('User 2', 'secure_api_key_2');
-- Create 10,000 links (5,000 per user)
WITH RECURSIVE link_numbers(n) AS (
SELECT 1
UNION ALL
SELECT n+1 FROM link_numbers
LIMIT 10000
)
INSERT INTO links (user_id, slug, url)
SELECT
((n-1) % 2) + 1, -- User ID (1-2)
'slug' || n, -- Unique slug
'https://sjdonado.com/page/' || n
FROM link_numbers;
-- Create 1,000 clicks per link (10 million total)
WITH RECURSIVE counts(n) AS (
SELECT 1
UNION ALL
SELECT n+1 FROM counts
LIMIT 1000
)
INSERT INTO clicks (link_id, user_agent, browser, os, referer, country)
SELECT
l.id,
CASE (c.n % 5)
WHEN 0 THEN 'Mozilla/5.0 (Windows NT 10.0; Win64; x64)'
WHEN 1 THEN 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15)'
WHEN 2 THEN 'Mozilla/5.0 (iPhone; CPU iPhone OS 14_0)'
WHEN 3 THEN 'Mozilla/5.0 (X11; Linux x86_64)'
ELSE 'Mozilla/5.0 (Android 11; Mobile)'
END,
CASE (c.n % 3)
WHEN 0 THEN 'Firefox'
WHEN 1 THEN 'Chrome'
ELSE 'Safari'
END,
CASE (c.n % 4)
WHEN 0 THEN 'macOS'
WHEN 1 THEN 'Windows'
WHEN 2 THEN 'iOS'
ELSE 'Android'
END,
CASE (c.n % 6)
WHEN 0 THEN 'https://sjdonado.com'
WHEN 1 THEN 'https://donado.co'
WHEN 2 THEN 'https://idonthavespotify.donado.co'
WHEN 3 THEN 'https://spookyplanning.com'
WHEN 4 THEN 'https://github.com/sjdonado'
ELSE NULL
END,
CASE (c.n % 10)
WHEN 0 THEN 'Colombia'
WHEN 1 THEN 'Brazil'
WHEN 2 THEN 'Canada'
WHEN 3 THEN 'Germany'
WHEN 4 THEN 'France'
WHEN 5 THEN 'Japan'
WHEN 6 THEN 'Australia'
WHEN 7 THEN 'Brazil'
WHEN 8 THEN 'India'
ELSE 'China'
END
FROM links l
CROSS JOIN counts c;
+15
View File
@@ -0,0 +1,15 @@
services:
app:
container_name: bit
build: .
environment:
ENV: production
ADMIN_NAME: 'Tester'
ADMIN_API_KEY: '0p+mDvbpZGLPGVCXnV+EDduR9Blkv27Dhq9XSzSbdQY='
ports:
- 4000:4000
volumes:
- sqlite_data:/app/sqlite
volumes:
sqlite_data:
+187
View File
@@ -0,0 +1,187 @@
## CLI
```
Usage: ./cli [options]
Options:
--create-user=NAME Create a new user with the given name
--list-users List all users
--delete-user=USER_ID Delete a user by ID
--update-parsers Download all required data files
```
## Local Development
### Requirements
- Crystal 1.18+
- Shards package manager
- SQLite3
### Install Dependencies
- linux
```bash
sudo apt-get update && sudo apt-get install -y crystal libssl-dev libsqlite3-dev
```
- macos
```bash
brew tap amberframework/micrate
brew install micrate
```
### Install Shards and Run
```bash
shards run bit
```
- Generate the `X-Api-Key`
```bash
shards run cli -- --create-user=Admin
```
- Run tests
```bash
ENV=test crystal spec
```
## Benchmark
### Run
```
shards build --release --no-debug --progress --stats
shards run benchmark
```
### Output
Chip: Apple M4 Pro. Memory: 24GB
```
1762075350 ~/p/bit> shards build --release --no-debug --progress --stats
shards run benchmark
Dependencies are satisfied
Building: bit
Parse: 00:00:00.000652375 ( 1.17MB)
Semantic (top level): 00:00:00.419246250 ( 163.45MB)
Semantic (new): 00:00:00.001636125 ( 163.45MB)
Semantic (type declarations): 00:00:00.019569792 ( 179.45MB)
Semantic (abstract def check): 00:00:00.009145125 ( 195.45MB)
Semantic (restrictions augmenter): 00:00:00.008421709 ( 195.45MB)
Semantic (ivars initializers): 00:00:00.019696584 ( 211.45MB)
Semantic (cvars initializers): 00:00:00.106829666 ( 211.50MB)
Semantic (main): 00:00:00.649298375 ( 499.88MB)
Semantic (cleanup): 00:00:00.000765250 ( 499.88MB)
Semantic (recursive struct check): 00:00:00.000752250 ( 499.88MB)
Codegen (crystal): 00:00:00.521307417 ( 532.38MB)
Codegen (bc+obj): 00:00:00.143842542 ( 532.38MB)
Codegen (linking): 00:00:00.236228750 ( 532.38MB)
Macro runs:
- /opt/homebrew/Cellar/crystal/1.18.2/share/crystal/src/ecr/process.cr: reused previous compilation (00:00:00.003593375)
Codegen (bc+obj):
- all previous .o files were reused
Building: cli
Parse: 00:00:00.000053291 ( 1.17MB)
Semantic (top level): 00:00:00.323534167 ( 163.45MB)
Semantic (new): 00:00:00.001705500 ( 163.45MB)
Semantic (type declarations): 00:00:00.018311958 ( 179.45MB)
Semantic (abstract def check): 00:00:00.007766750 ( 195.45MB)
Semantic (restrictions augmenter): 00:00:00.005686667 ( 195.45MB)
Semantic (ivars initializers): 00:00:00.011239792 ( 211.45MB)
Semantic (cvars initializers): 00:00:00.100870833 ( 211.50MB)
Semantic (main): 00:00:00.285426750 ( 371.62MB)
Semantic (cleanup): 00:00:00.000369875 ( 371.62MB)
Semantic (recursive struct check): 00:00:00.000570917 ( 371.62MB)
Codegen (crystal): 00:00:00.317534875 ( 387.88MB)
Codegen (bc+obj): 00:00:00.097321417 ( 387.88MB)
Codegen (linking): 00:00:00.095931000 ( 387.88MB)
Codegen (bc+obj):
- all previous .o files were reused
Building: benchmark
Parse: 00:00:00.000228500 ( 1.17MB)
Semantic (top level): 00:00:00.242174458 ( 147.78MB)
Semantic (new): 00:00:00.000863333 ( 147.78MB)
Semantic (type declarations): 00:00:00.011527792 ( 147.78MB)
Semantic (abstract def check): 00:00:00.031242333 ( 147.78MB)
Semantic (restrictions augmenter): 00:00:00.003593583 ( 147.78MB)
Semantic (ivars initializers): 00:00:00.006753667 ( 147.78MB)
Semantic (cvars initializers): 00:00:00.028373834 ( 195.78MB)
Semantic (main): 00:00:00.152039542 ( 243.83MB)
Semantic (cleanup): 00:00:00.000249084 ( 243.83MB)
Semantic (recursive struct check): 00:00:00.000460417 ( 243.83MB)
Codegen (crystal): 00:00:00.075461000 ( 259.83MB)
Codegen (bc+obj): 00:00:04.834914333 ( 259.83MB)
Codegen (linking): 00:00:00.119920416 ( 259.83MB)
Codegen (bc+obj):
- no previous .o files were reused
Dependencies are satisfied
Building: benchmark
Executing: benchmark
Cleaning up benchmark database...
Deleted existing database: ./sqlite/data.benchmark.db
Database cleanup completed.
Running database migrations...
Migrating db, current version: 0, target: 20250319192003
OK 20240512214223_create_links.sql
OK 20240512225208_add_slug_index_to_links.sql
OK 20240513115731_create_users.sql
OK 20240513130054_add_api_key_index_to_users.sql
OK 20240711224103_create_clicks.sql
OK 20240714215409_update_slug_size_links.sql
OK 20250316102350_add_country_to_clicks.sql
OK 20250316111734_replace_unkwown_with_null.sql
OK 20250318072657_replace_slug_index_with_covering_index.sql
OK 20250319192003_convert_all_tables_text_ids_to_integer.sql
Migrations completed successfully.
Seeding benchmark database...
Database seeded successfully.
Starting application: ./bit...
Application output will be saved to: app_output.log
Application started with PID: 11638
Using database: ./sqlite/data.benchmark.db
Checking if server is ready at http://localhost:4001...
.Server is ready!
Fetching links from API...
Selected link: http://localhost:4001/slug9391
Starting benchmark with 100000 requests...
Bombarding http://localhost:4001/slug9391 with 100000 request(s) using 125 connection(s)
100000 / 100000 [============================================================================] 100.00% 11078/s 9s
Done!
Statistics Avg Stdev Max
Reqs/sec 11427.28 8889.68 30270.91
Latency 11.02ms 6.55ms 53.91ms
Latency Distribution
50% 1.85ms
75% 5.37ms
90% 39.36ms
95% 39.87ms
99% 42.66ms
HTTP codes:
1xx - 0, 2xx - 0, 3xx - 100000, 4xx - 0, 5xx - 0
others - 0
Throughput: 3.08MB/s
Benchmark completed successfully.
**** Resource Usage Statistics ****
Measurements: 12
Average CPU Usage: 71.5%
Average Memory Usage: 39.8 MiB
Peak CPU Usage: 100.0%
Peak Memory Usage: 53.41 MiB
**** Files Generated ****
Resource stats: resource_usage.log
Application log: app_output.log
Database: ./sqlite/data.benchmark.db
Stopping application...
Application stopped.
```
View File
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 665 B

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 628 B

-51
View File
@@ -1,51 +0,0 @@
<!-- HTML for static distribution bundle build -->
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Swagger UI</title>
<link rel="stylesheet" type="text/css" href="./swagger-ui.css" >
<link rel="icon" type="image/png" href="./favicon-32x32.png" sizes="32x32" />
<link rel="icon" type="image/png" href="./favicon-16x16.png" sizes="16x16" />
<style>
html
{
box-sizing: border-box;
overflow: -moz-scrollbars-vertical;
overflow-y: scroll;
}
*,
*:before,
*:after
{
box-sizing: inherit;
}
body
{
margin:0;
background: #fafafa;
}
</style>
</head>
<body>
<div id="swagger-ui"></div>
<script src="./swagger-ui-bundle.js" charset="UTF-8"> </script>
<script src="./swagger-ui-standalone-preset.js" charset="UTF-8"> </script>
<script>
window.onload = function() {
// Begin Swagger UI call region
const ui = SwaggerUIBundle({
configUrl: "swagger-config.json",
dom_id: '#swagger-ui'
})
// End Swagger UI call region
window.ui = ui
}
</script>
</body>
</html>
+341
View File
@@ -0,0 +1,341 @@
#!/usr/bin/env crystal
require "http/client"
require "json"
PORT = "4001"
APP_URL = "http://localhost:#{PORT}"
API_URL = "#{APP_URL}/api/links"
API_KEY = "secure_api_key_1"
NUMBER_OF_REQUESTS = 100000
APP_COMMAND = "./bit"
APP_ARGS = [] of String
STATS_FILE = "resource_usage.log"
APP_LOG_FILE = "app_output.log"
DATABASE_URL = "sqlite3://./sqlite/data.benchmark.db?journal_mode=wal&synchronous=normal&foreign_keys=true"
DATABASE_FILE = "./sqlite/data.benchmark.db"
class ResourceMonitor
def initialize(@pid : Int32)
@running = false
@stats = [] of {timestamp: Time, cpu: Float64, memory: Float64}
end
def start
@running = true
@stats.clear
File.write(STATS_FILE, "Timestamp\tCPU(%)\tMemory(MiB)\n")
spawn do
while @running
if stat = capture_stats
File.open(STATS_FILE, "a") do |file|
file.puts "#{stat[:timestamp].to_unix}\t#{stat[:cpu]}\t#{stat[:memory]}"
end
@stats << stat
end
sleep 1.seconds
end
end
end
def stop
@running = false
sleep 1.seconds
end
private def capture_stats
output = IO::Memory.new
process = Process.run(
"ps", ["-p", @pid.to_s, "-o", "%cpu,%mem,rss"],
output: output
)
if process.success?
lines = output.to_s.strip.split("\n")
if lines.size >= 2
data_line = lines[1].strip.split
if data_line.size >= 3
cpu = data_line[0].to_f
# RSS is in KB on macOS, convert to MiB
memory_kb = data_line[2].to_f
memory_mib = memory_kb / 1024.0
return {timestamp: Time.utc, cpu: cpu, memory: memory_mib}
end
end
end
nil
end
def print_summary
return if @stats.empty?
total_cpu = 0.0
total_memory = 0.0
peak_cpu = 0.0
peak_memory = 0.0
@stats.each do |stat|
total_cpu += stat[:cpu]
total_memory += stat[:memory]
peak_cpu = stat[:cpu] if stat[:cpu] > peak_cpu
peak_memory = stat[:memory] if stat[:memory] > peak_memory
end
avg_cpu = total_cpu / @stats.size
avg_memory = total_memory / @stats.size
summary = <<-STATS
**** Resource Usage Statistics ****
Measurements: #{@stats.size}
Average CPU Usage: #{avg_cpu.round(2)}%
Average Memory Usage: #{avg_memory.round(2)} MiB
Peak CPU Usage: #{peak_cpu.round(2)}%
Peak Memory Usage: #{peak_memory.round(2)} MiB
STATS
File.open(STATS_FILE, "a") do |file|
file.puts summary
end
puts summary
end
end
def start_application : Process
puts "Starting application: #{APP_COMMAND}..."
puts "Application output will be saved to: #{APP_LOG_FILE}"
log_file = File.open(APP_LOG_FILE, "w")
process = Process.new(
APP_COMMAND,
APP_ARGS,
output: log_file,
error: log_file,
env: {
"DATABASE_URL" => DATABASE_URL,
"APP_URL" => APP_URL,
"PORT" => PORT,
}
)
puts "Application started with PID: #{process.pid}"
puts "Using database: #{DATABASE_FILE}"
process
end
def stop_application(process : Process)
puts "\nStopping application..."
process.signal(Signal::TERM)
# Give it a few seconds to shut down gracefully
sleep 3.seconds
# Force kill if still running
begin
process.signal(Signal::KILL)
rescue
# Process already terminated
end
puts "Application stopped."
end
def check_dependencies
{"bombardier", "sqlite3", "micrate"}.each do |cmd|
process = Process.run("which", [cmd], output: Process::Redirect::Close)
unless process.success?
puts "Error: #{cmd} is not installed. Please install it to proceed."
case cmd
when "bombardier"
puts " brew install bombardier"
when "sqlite3"
puts " brew install sqlite3"
when "micrate"
puts " shards install"
end
exit(1)
end
end
end
def wait_for_server
puts "Checking if server is ready at #{APP_URL}..."
30.times do
begin
if HTTP::Client.get("#{APP_URL}/api/ping").success?
puts "Server is ready!"
return
end
rescue
# Server not ready yet
end
sleep 1.seconds
print "."
end
puts "\nError: Server is not responding. Please start your application first."
exit(1)
end
def run_benchmark
puts "Fetching links from API..."
response = HTTP::Client.get(
"#{API_URL}?limit=10000",
headers: HTTP::Headers{"X-Api-Key" => API_KEY}
)
unless response.success?
puts "Failed to fetch links. Status: #{response.status_code}"
puts "Make sure your server is running and the API key is correct."
exit(1)
end
data = JSON.parse(response.body)
links = data["data"].as_a.map { |link| link["refer"].as_s }
if links.empty?
puts "No links found. Please seed your database first."
exit(1)
end
random_link = links.sample
puts "Selected link: #{random_link}"
puts "\nStarting benchmark with #{NUMBER_OF_REQUESTS} requests..."
sleep 2.seconds
process = Process.new(
"bombardier",
["-n", NUMBER_OF_REQUESTS.to_s, "-l", "--disableKeepAlives", random_link],
output: Process::Redirect::Inherit,
error: Process::Redirect::Inherit
)
status = process.wait
if status.success?
puts "\nBenchmark completed successfully."
else
puts "\nBombardier failed with error code: #{status.exit_code}"
exit(1)
end
end
def cleanup_database
puts "Cleaning up benchmark database..."
if File.exists?(DATABASE_FILE)
File.delete(DATABASE_FILE)
puts "Deleted existing database: #{DATABASE_FILE}"
end
# Also remove WAL and SHM files if they exist
["#{DATABASE_FILE}-wal", "#{DATABASE_FILE}-shm"].each do |file|
if File.exists?(file)
File.delete(file)
puts "Deleted: #{file}"
end
end
# Ensure sqlite directory exists
Dir.mkdir_p("./sqlite")
puts "Database cleanup completed."
end
def run_migrations
puts "Running database migrations..."
process = Process.run("which", ["micrate"], output: Process::Redirect::Close)
unless process.success?
puts "Error: micrate is not installed. Please install it to proceed."
puts " shards install"
exit(1)
end
process = Process.run(
"micrate",
["up"],
env: {"DATABASE_URL" => DATABASE_URL},
output: Process::Redirect::Inherit,
error: Process::Redirect::Inherit
)
if process.success?
puts "Migrations completed successfully."
else
puts "Error: Migrations failed."
exit(1)
end
end
def seed_database
puts "Seeding benchmark database..."
unless File.exists?("./db/seed.sql")
puts "Warning: ./db/seed.sql not found. Skipping database seeding."
return
end
unless File.exists?(DATABASE_FILE)
puts "Warning: #{DATABASE_FILE} not found. Database may not be initialized."
end
process = Process.run(
"sqlite3",
[DATABASE_FILE],
input: File.open("./db/seed.sql"),
output: Process::Redirect::Inherit,
error: Process::Redirect::Inherit
)
if process.success?
puts "Database seeded successfully."
else
puts "Warning: Database seeding failed. Continuing anyway..."
end
end
def main
check_dependencies
# Setup benchmark database
cleanup_database
run_migrations
seed_database
app_process = start_application
begin
wait_for_server
# Give it a moment to settle
sleep 2.seconds
monitor = ResourceMonitor.new(app_process.pid.to_i32)
monitor.start
run_benchmark
monitor.stop
monitor.print_summary
puts "\n**** Files Generated ****"
puts " Resource stats: #{STATS_FILE}"
puts " Application log: #{APP_LOG_FILE}"
puts " Database: #{DATABASE_FILE}"
ensure
# Always stop the application
stop_application(app_process)
end
end
main
+37
View File
@@ -0,0 +1,37 @@
require "option_parser"
require "../app/services/cli"
OptionParser.parse do |parser|
parser.on("--create-user=NAME", "Create a new user with the given name") do |name|
puts App::Services::Cli.create_user(name)
exit
end
parser.on("--list-users", "List all users") do
puts App::Services::Cli.list_users
exit
end
parser.on("--delete-user=USER_ID", "Delete a user by ID") do |user_id|
puts App::Services::Cli.delete_user(user_id)
exit
end
parser.on("--update-parsers", "Download UA regexes and/or GeoLite2 database") do
puts "=== Starting data files update ==="
App::Services::Cli.update_uap_regexes
App::Services::Cli.update_geolite_db
puts "=== Data files updated successfully ==="
exit
end
if ARGV.empty?
puts "Usage: ./cli [options]"
puts "Options:"
puts " --create-user=NAME Create a new user with the given name"
puts " --list-users List all users"
puts " --delete-user=USER_ID Delete a user by ID"
puts " --update-parsers Download all required data files"
end
end
+58
View File
@@ -0,0 +1,58 @@
version: 2.0
shards:
backtracer:
git: https://github.com/sija/backtracer.cr.git
version: 1.2.4
crecto:
git: https://github.com/fridgerator/crecto.git
version: 0.14.0
db:
git: https://github.com/crystal-lang/crystal-db.git
version: 0.11.0
dotenv:
git: https://github.com/gdotdesign/cr-dotenv.git
version: 1.0.0
exception_page:
git: https://github.com/crystal-loot/exception_page.git
version: 0.5.0
ipaddress:
git: https://github.com/sija/ipaddress.cr.git
version: 0.2.3
kemal:
git: https://github.com/kemalcr/kemal.git
version: 1.7.3
maxminddb:
git: https://github.com/delef/maxminddb.cr.git
version: 1.5.0
micrate:
git: https://github.com/amberframework/micrate.git
version: 0.15.1
mysql:
git: https://github.com/crystal-lang/crystal-mysql.git
version: 0.14.0
pg:
git: https://github.com/will/crystal-pg.git
version: 0.26.0
radix:
git: https://github.com/luislavena/radix.git
version: 0.4.1
spec-kemal:
git: https://github.com/kemalcr/spec-kemal.git
version: 1.0.0
sqlite3:
git: https://github.com/crystal-lang/crystal-sqlite3.git
version: 0.19.0
+37
View File
@@ -0,0 +1,37 @@
name: bit
version: 1.6.0
authors:
- Juan Rodriguez Donado <@sjdonado>
targets:
bit:
main: bit.cr
cli:
main: scripts/cli.cr
benchmark:
main: scripts/benchmark.cr
dependencies:
kemal:
github: kemalcr/kemal
version: 1.7.3
sqlite3:
github: crystal-lang/crystal-sqlite3
crecto:
github: fridgerator/crecto
micrate:
github: amberframework/micrate
version: 0.15.1
maxminddb:
github: delef/maxminddb.cr
development_dependencies:
dotenv:
github: gdotdesign/cr-dotenv
spec-kemal:
github: kemalcr/spec-kemal
crystal: ">= 1.18.2"
license: MIT
+446
View File
@@ -0,0 +1,446 @@
require "../spec_helper"
require "../../app/models/*"
API_KEY = Random::Secure.urlsafe_base64
describe "App::Controllers::Link" do
describe "Create" do
it "should create link" do
test_user = create_test_user()
payload = {"url" => "https://kagi.com"}
post(
"/api/links",
headers: HTTP::Headers{"Content-Type" => "application/json", "X-Api-Key" => test_user.api_key.to_s},
body: payload.to_json
)
parsed_response = Hash(String, Hash(String, String | Int64 | Array(Hash(String, String | Int64)))).from_json(response.body)
parsed_response["data"]["origin"].should eq(payload["url"])
end
it "should return existing link if url already exists" do
test_user = create_test_user()
payload = {"url" => "http://idonthavespotify.donado.co"}
post(
"/api/links",
headers: HTTP::Headers{"Content-Type" => "application/json", "X-Api-Key" => test_user.api_key.to_s},
body: payload.to_json
)
first_response = Hash(String, Hash(String, String | Int64 | Array(Hash(String, String)))).from_json(response.body)
first_response["data"]["origin"].should eq(payload["url"])
post(
"/api/links",
headers: HTTP::Headers{"Content-Type" => "application/json", "X-Api-Key" => test_user.api_key.to_s},
body: payload.to_json
)
second_response = Hash(String, Hash(String, String | Int64 | Array(Hash(String, String)))).from_json(response.body)
second_response["data"]["origin"].should eq(payload["url"])
second_response["data"]["id"].should eq(first_response["data"]["id"])
end
it "should return 400 - url required field" do
test_user = create_test_user()
payload = {"test" => "https://kagi.com"}
post(
"/api/links",
headers: HTTP::Headers{"Content-Type" => "application/json", "X-Api-Key" => test_user.api_key.to_s},
body: payload.to_json
)
expected = {"error" => "url: Required field"}.to_json
response.body.should eq(expected)
end
it "should return 400 - invalid url" do
test_user = create_test_user()
payload = {"url" => "test"}
post(
"/api/links",
headers: HTTP::Headers{"Content-Type" => "application/json", "X-Api-Key" => test_user.api_key.to_s},
body: payload.to_json
)
expected = {"errors" => {"url" => ["is invalid"]}}.to_json
response.body.should eq(expected)
end
it "should return 401 - missing api key" do
payload = {"url" => "https://kagi.com"}
post("/api/links", headers: HTTP::Headers{"Content-Type" => "application/json"}, body: payload.to_json)
expected = {"error" => "Unauthorized access"}.to_json
response.status_code.should eq(401)
response.body.should eq(expected)
end
end
describe "Index" do
it "should redirect to origin domain with forwarded headers" do
link = "https://test.com"
test_user = create_test_user()
test_link = create_test_link(test_user, link)
user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:127.0) Gecko/20100101 Firefox/127.0"
get("/#{test_link.slug}", headers: HTTP::Headers{
"X-Api-Key" => test_user.api_key.to_s,
"User-Agent" => user_agent
})
response.headers["Location"].should eq(link)
response.headers["User-Agent"].should eq(user_agent)
response.headers.has_key?("X-Forwarded-For").should be_true
end
it "should create a new click after redirect with proper information" do
link = "https://sjdonado.com"
test_user = create_test_user()
test_link = create_test_link(test_user, link)
user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:127.0) Gecko/20100101 Firefox/127.0"
referer = "https://example.com/page"
get("/#{test_link.slug}", headers: HTTP::Headers{
"User-Agent" => user_agent,
"Referer" => referer
})
Fiber.yield # replace yield with sleep 5 to debug errors
response.headers["Location"].should eq(link)
# Verify that the click was recorded
updated_test_link = get_test_link(test_link.id.not_nil!)
updated_test_link.clicks.size.should eq(test_link.clicks.size + 1)
# Verify click details
latest_click = updated_test_link.clicks.last
latest_click.user_agent.should eq(user_agent)
latest_click.browser.should eq("Firefox")
latest_click.os.should eq("Mac OS X")
latest_click.referer.should eq("example.com") # Should extract host from the referer
end
it "should create a click with utm_source when no referer is provided" do
link = "https://sjdonado.com"
test_user = create_test_user()
test_link = create_test_link(test_user, link)
# Add utm_source parameter
user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:127.0) Gecko/20100101 Firefox/127.0"
get("/#{test_link.slug}?utm_source=email_campaign", headers: HTTP::Headers{
"User-Agent" => user_agent
})
sleep 0.2.seconds # Wait for async click creation
updated_test_link = get_test_link(test_link.id.not_nil!)
latest_click = updated_test_link.clicks.last
latest_click.referer.should eq("email_campaign")
end
it "should return 404 - link does not exist" do
test_user = create_test_user()
get("/R4kj2")
expected = {"error" => "Resource not found"}.to_json
response.status_code.should eq(404)
response.body.should eq(expected)
end
end
describe "All" do
it "should return all links with pagination" do
links = ["https://sjdonado.com", "sjdonado.com", "sjdonado.com.co"]
test_user = create_test_user()
links.each do |link|
create_test_link(test_user, link)
end
get("/api/links", headers: HTTP::Headers{"X-Api-Key" => test_user.api_key.to_s})
parsed_response = Hash(String, Array(Hash(String, String | Int64)) | Hash(String, Bool | String? | Int64?)).from_json(response.body)
# Check that each link is in the response data
origins = parsed_response["data"].as(Array).map { |link| link["origin"] }
links.each do |link|
origins.should contain(link)
end
parsed_response["pagination"].as(Hash)["has_more"].should be_false
end
it "should respect custom limit parameter" do
test_user = create_test_user()
5.times do |i|
create_test_link(test_user, "https://example.com/#{i}")
end
get("/api/links?limit=2", headers: HTTP::Headers{"X-Api-Key" => test_user.api_key.to_s})
parsed_response = Hash(String, Array(Hash(String, String | Int64)) | Hash(String, Bool | String? | Int64?)).from_json(response.body)
parsed_response["data"].as(Array).size.should eq(2)
parsed_response["pagination"].as(Hash)["has_more"].should be_true
parsed_response["pagination"].as(Hash)["next"].should_not be_nil
end
it "should support cursor-based pagination" do
test_user = create_test_user()
5.times do |i|
create_test_link(test_user, "https://example.com/#{i}")
end
# Get first page
get("/api/links?limit=2", headers: HTTP::Headers{"X-Api-Key" => test_user.api_key.to_s})
first_page = Hash(String, Array(Hash(String, String | Int64)) | Hash(String, Bool | String? | Int64?)).from_json(response.body)
cursor = first_page["pagination"].as(Hash)["next"]
# Get second page using cursor
get("/api/links?limit=2&cursor=#{cursor}", headers: HTTP::Headers{"X-Api-Key" => test_user.api_key.to_s})
second_page = Hash(String, Array(Hash(String, String | Int64)) | Hash(String, Bool | String? | Int64?)).from_json(response.body)
# Ensure different links are returned
first_page_ids = first_page["data"].as(Array).map { |link| link["id"] }
second_page_ids = second_page["data"].as(Array).map { |link| link["id"] }
# Check that no IDs from first page appear in second page
(first_page_ids & second_page_ids).empty?.should be_true
end
it "should return owned links only" do
links = ["https://donado.co", "donado.co", "uninorte.edu.co", "kagi.com"]
test_user = create_test_user()
links[0..2].each do |link|
create_test_link(test_user, link)
end
test_other_user = create_test_user()
create_test_link(test_other_user, links[3])
get("/api/links", headers: HTTP::Headers{"X-Api-Key" => test_user.api_key.to_s})
parsed_response = Hash(String, Array(Hash(String, String | Int64)) | Hash(String, Bool | String? | Int64?)).from_json(response.body)
parsed_response["data"].as(Array).size.should eq(3)
origins = parsed_response["data"].as(Array).map { |link| link["origin"] }
links[0..2].each do |link|
origins.should contain(link)
end
origins.should_not contain(links[3])
end
it "should return 401 - missing api key" do
get "/api/links"
expected = {"error" => "Unauthorized access"}.to_json
response.status_code.should eq(401)
response.body.should eq(expected)
end
end
describe "Get" do
it "should return the specified link with limited click details" do
link = "https://bing.com"
test_user = create_test_user()
test_link = create_test_link(test_user, link)
110.times do
create_test_click(test_link)
end
get("/api/links/#{test_link.id}", headers: HTTP::Headers{"X-Api-Key" => test_user.api_key.to_s})
parsed_response = Hash(String, Hash(String, String | Int64 | Array(Hash(String, String | Int64)))).from_json(response.body)
parsed_response["data"]["origin"].should eq(link)
parsed_response["data"]["clicks"].as(Array).size.should eq(100)
end
it "should return 404 - link does not exist" do
test_user = create_test_user()
get("/api/links/999999", headers: HTTP::Headers{"X-Api-Key" => test_user.api_key.to_s})
expected = {"error" => "Resource not found"}.to_json
response.status_code.should eq(404)
response.body.should eq(expected)
end
it "should return 401 - missing api key" do
get "/api/links/1"
expected = {"error" => "Unauthorized access"}.to_json
response.status_code.should eq(401)
response.body.should eq(expected)
end
end
describe "Clicks" do
it "should return paginated clicks for a link" do
link = "https://example.com"
test_user = create_test_user()
test_link = create_test_link(test_user, link)
5.times do
create_test_click(test_link)
end
get("/api/links/#{test_link.id}/clicks", headers: HTTP::Headers{"X-Api-Key" => test_user.api_key.to_s})
parsed_response = Hash(String, Array(Hash(String, String | Int64)) | Hash(String, Bool | String? | Int64?)).from_json(response.body)
parsed_response["data"].as(Array).size.should eq(5)
parsed_response["pagination"].as(Hash)["has_more"].should be_false
end
it "should respect limit parameter" do
link = "https://example.com"
test_user = create_test_user()
test_link = create_test_link(test_user, link)
10.times do
create_test_click(test_link)
end
get("/api/links/#{test_link.id}/clicks?limit=3", headers: HTTP::Headers{"X-Api-Key" => test_user.api_key.to_s})
parsed_response = Hash(String, Array(Hash(String, String | Int64)) | Hash(String, Bool | String? | Int64?)).from_json(response.body)
parsed_response["data"].as(Array).size.should eq(3)
parsed_response["pagination"].as(Hash)["has_more"].should be_true
parsed_response["pagination"].as(Hash)["next"].should_not be_nil
end
it "should support cursor-based pagination" do
link = "https://example.com"
test_user = create_test_user()
test_link = create_test_link(test_user, link)
10.times do
create_test_click(test_link)
end
# Get first page
get("/api/links/#{test_link.id}/clicks?limit=3", headers: HTTP::Headers{"X-Api-Key" => test_user.api_key.to_s})
first_page = Hash(String, Array(Hash(String, String | Int64)) | Hash(String, Bool | String? | Int64?)).from_json(response.body)
cursor = first_page["pagination"].as(Hash)["next"]
# Get second page using cursor
get("/api/links/#{test_link.id}/clicks?limit=3&cursor=#{cursor}", headers: HTTP::Headers{"X-Api-Key" => test_user.api_key.to_s})
second_page = Hash(String, Array(Hash(String, String | Int64)) | Hash(String, Bool | String? | Int64?)).from_json(response.body)
# Ensure different clicks are returned
first_page_ids = first_page["data"].as(Array).map { |click| click["id"] }
second_page_ids = second_page["data"].as(Array).map { |click| click["id"] }
# Check that no IDs from first page appear in second page
(first_page_ids & second_page_ids).empty?.should be_true
end
it "should return 404 - link does not exist" do
test_user = create_test_user()
get("/api/links/999999/clicks", headers: HTTP::Headers{"X-Api-Key" => test_user.api_key.to_s})
expected = {"error" => "Resource not found"}.to_json
response.status_code.should eq(404)
response.body.should eq(expected)
end
it "should return 401 - missing api key" do
get("/api/links/1/clicks")
expected = {"error" => "Unauthorized access"}.to_json
response.status_code.should eq(401)
response.body.should eq(expected)
end
end
describe "Update" do
it "should update link url" do
link = "https://github.com"
test_user = create_test_user()
test_link = create_test_link(test_user, link)
payload = {"url" => "https://github.com.co"}
put(
"/api/links/#{test_link.id}",
headers: HTTP::Headers{"Content-Type" => "application/json", "X-Api-Key" => test_user.api_key.to_s},
body: payload.to_json
)
parsed_response = Hash(String, Hash(String, String | Int64 | Array(Hash(String, String | Int64)))).from_json(response.body)
parsed_response["data"]["origin"].should eq(payload["url"])
end
it "should return 404 - link does not exist" do
test_user = create_test_user()
payload = {"url" => "https://kagi.com.co"}
put(
"/api/links/999999",
headers: HTTP::Headers{"Content-Type" => "application/json", "X-Api-Key" => test_user.api_key.to_s},
body: payload.to_json
)
expected = {"error" => "Resource not found"}.to_json
response.status_code.should eq(404)
response.body.should eq(expected)
end
it "should return 401 - missing api key" do
payload = {"url" => "https://kagi.com.co"}
put(
"/api/links/1",
headers: HTTP::Headers{"Content-Type" => "application/json"},
body: payload.to_json
)
expected = {"error" => "Unauthorized access"}.to_json
response.status_code.should eq(401)
response.body.should eq(expected)
end
end
describe "Delete" do
it "should delete link url" do
link = "https://news.ycombinator.com"
test_user = create_test_user()
test_link = create_test_link(test_user, link)
delete("/api/links/#{test_link.id}", headers: HTTP::Headers{"X-Api-Key" => test_user.api_key.to_s})
response.status_code.should eq(204)
end
it "should return 404 - link does not exist" do
test_user = create_test_user()
delete("/api/links/999999", headers: HTTP::Headers{"X-Api-Key" => test_user.api_key.to_s})
expected = {"error" => "Resource not found"}.to_json
response.status_code.should eq(404)
response.body.should eq(expected)
end
it "should return 401 - missing api key" do
delete "/api/links/1"
expected = {"error" => "Unauthorized access"}.to_json
response.status_code.should eq(401)
response.body.should eq(expected)
end
end
end
+10
View File
@@ -0,0 +1,10 @@
require "../spec_helper"
describe "App::Controllers::Ping" do
it "should return pong" do
get "/api/ping"
expected = {"data" => "pong"}.to_json
response.body.should eq(expected)
end
end
+62
View File
@@ -0,0 +1,62 @@
require "../spec_helper"
require "../../app/services/cli"
describe "App::Services::Cli" do
it "creates a new user" do
name = "testuser"
output = App::Services::Cli.create_user(name)
output.should contain "New user created: Name: testuser"
end
it "lists all users" do
App::Services::Cli.create_user("user1")
App::Services::Cli.create_user("user2")
output = App::Services::Cli.list_users
output.should contain "Users:"
output.should contain "Name: user1"
output.should contain "Name: user2"
end
it "deletes a user by ID" do
App::Services::Cli.create_user("user_to_delete")
user = App::Lib::Database.all(App::Models::User).first
output = App::Services::Cli.delete_user(user.id)
output.should contain "User with ID #{user.id} deleted successfully"
end
it "handles deletion of non-existent user" do
output = App::Services::Cli.delete_user("non-existent-id")
output.should contain "Failed to delete user"
end
it "sets up an admin user if environment variables are present" do
ENV["ADMIN_NAME"] = "adminuser"
ENV["ADMIN_API_KEY"] = "secure_admin_key"
App::Services::Cli.setup_admin_user
admin_user = App::Lib::Database.all(App::Models::User).find { |u| u.name == "adminuser" }
admin_user.should_not be_nil
admin_user = admin_user.not_nil!
admin_user.api_key.should eq "secure_admin_key"
App::Services::Cli.delete_user(admin_user.id)
end
it "skips admin setup if environment variables are missing" do
ENV.delete("ADMIN_NAME")
ENV.delete("ADMIN_API_KEY")
App::Services::Cli.setup_admin_user
users = App::Lib::Database.all(App::Models::User)
users.none? { |u| u.name == "adminuser" }.should be_true
end
end
+86
View File
@@ -0,0 +1,86 @@
require "file_utils"
require "spec-kemal"
require "micrate"
require "dotenv"
Dotenv.load ".env.#{ENV["ENV"]}"
require "../bit"
Spec.before_suite do
# Delete the SQLite database file if it exists
db_file_path = ENV["DATABASE_URL"].split("sqlite3://").last.split("?").first
if File.exists?(db_file_path)
File.delete(db_file_path)
end
Micrate::DB.connection_url = ENV["DATABASE_URL"]
Micrate::Cli.run_up
Kemal.config.logging = false
end
def create_test_user
user = App::Models::User.new
user.name = "Tester"
user.api_key = Random::Secure.urlsafe_base64()
changeset = App::Lib::Database.insert(user)
if !changeset.valid?
error_messages = changeset.errors.map { |error| "#{error}" }.join(", ")
raise "Test user creation failed #{error_messages}"
end
changeset.instance
end
def create_test_link(user, url)
link = App::Models::Link.new
link.slug = App::Services::SlugService.shorten_url(url, user.id.not_nil!)
link.url = url
link.user = user
changeset = App::Lib::Database.insert(link)
unless changeset.valid?
error_messages = changeset.errors.map { |error| "#{error}" }.join(", ")
raise "Test link creation failed: #{error_messages}"
end
inserted_link = changeset.instance
inserted_link.clicks = [] of App::Models::Click
inserted_link
end
def create_test_click(link)
click = App::Models::Click.new
click.user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:127.0) Gecko/20100101 Firefox/127.0"
click.browser = "Firefox"
click.os = "Mac OS X"
click.referer = "example.com"
click.country = "US"
click.created_at = Time.utc
click.link = link
click.link_id = link.id.not_nil!
changeset = App::Lib::Database.insert(click)
unless changeset.valid?
error_messages = changeset.errors.map { |error| "#{error}" }.join(", ")
raise "Test click creation failed: #{error_messages}"
end
changeset.instance
end
def get_test_link(link_id : Int64)
query = App::Lib::Database::Query.where(id: link_id).limit(1)
link = App::Lib::Database.all(App::Models::Link, query, preload: [:clicks]).first?
raise "Link not found" if link.nil?
link
end
def delete_test_link(link_id : Int64)
App::Lib::Database.raw_exec("DELETE FROM links WHERE id = (?)", link_id) # tempfix: Database.delete does not work
end
View File
-1
View File
@@ -1 +0,0 @@
{"url":"openapi.yaml","deepLinking":true}
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
-4
View File
File diff suppressed because one or more lines are too long