I think this service is already pretty good, but when defending against LLM based attacks, they're a godsend. Current tools don't appear to be too discerning.
Any idea on how to roll-your-own canary token for an unsupported token provider like Cloudflare/Stripe/OpenAI? Do you just make a real Cloudflare/Stripe/OpenAI account, create an API key, and vibecode an alerting system?
I'm kinda surprised that they haven't expanded beyond their existing token providers.
We look at several factors before adding new Canarytoken API key types to canarytokens.org:
1. Can we create/remove keys programmatically at scale cheaply? Most sites aren't geared up to handle tens or hundreds of thousands of API keys tied to a single account. If the per-account limits are low then we have to build infra to manage multiple accounts.
2. Can we defang the keys so that there is no risk to them being discovered?
3. Does the service provide a near real-time audit log to pinpoint when an individual key was used? Ideally the audit log also includes IP information.
4. Does the service provide a mechanism to trigger audit events, or do we need to poll the service for the audit logs?
5. Would most attackers recognise the API key and want to try it?
They make credit cards. Obviously they're not real cards but what sort of financial wizardry is required to make that possible? And given it's vibecoded that means Claude knows how to make fake credit cards now?
If by "fake" you mean ones that pass number validation on web sites, that's easy. I wrote python code to check and generate valid numbers back in the '90s; it's a simple, well known check digit algorithm appended at the end. https://en.wikipedia.org/wiki/Luhn_algorithm.
Vibecoding has seriously eroding the trust we have in public software. Hopefully there's some of badge projects can use to clarify how much Ai was in charge of it.
The first version used a provider that started rate-limiting us pretty heavily. It meant a pretty crappy experience for folks creating the Canarytoken. We decided to pause and look for alternatives.
Canarytokens: Honeypot for critical credentials, get notified when they are used (2015) - https://news.ycombinator.com/item?id=41107209 - July 2024 (42 comments)
Canary Tokens - https://news.ycombinator.com/item?id=34157751 - Dec 2022 (59 comments)
A “Safety Net” for AWS Canarytokens - https://news.ycombinator.com/item?id=33644351 - Nov 2022 (1 comment)
Canarytokens - https://news.ycombinator.com/item?id=31914078 - June 2022 (3 comments)
An insanely easy-to-use honeypot solution that deploys in just four minutes - https://news.ycombinator.com/item?id=30374561 - Feb 2022 (4 comments)
Canarytokens – honeypot tokens that tell you when someone used them - https://news.ycombinator.com/item?id=15778011 - Nov 2017 (1 comment)
You might find some recent work we did on agent deception interesting as well: https://blog.thinkst.com/2026/09/getting-agents-to-tell-on-t...
I'm kinda surprised that they haven't expanded beyond their existing token providers.
1. Can we create/remove keys programmatically at scale cheaply? Most sites aren't geared up to handle tens or hundreds of thousands of API keys tied to a single account. If the per-account limits are low then we have to build infra to manage multiple accounts.
2. Can we defang the keys so that there is no risk to them being discovered?
3. Does the service provide a near real-time audit log to pinpoint when an individual key was used? Ideally the audit log also includes IP information.
4. Does the service provide a mechanism to trigger audit events, or do we need to poll the service for the audit logs?
5. Would most attackers recognise the API key and want to try it?
Most services fail one or more of these.
I feel like a self-hosted version would be better though. Having to phone home to their servers could give the game away.
https://github.com/thinkst/canarytokens
If by "fake" you mean ones that pass number validation on web sites, that's easy. I wrote python code to check and generate valid numbers back in the '90s; it's a simple, well known check digit algorithm appended at the end. https://en.wikipedia.org/wiki/Luhn_algorithm.
PCI audits are fun for everyone
Why is this a given? This service has been around for longer than LLMs have been a thing.
- No LLMs
- LLMs as tools
- LLMs decided some things
- YOLO vibecoded
Also what service do you use to mint endless virtual cc numbers?
We’re using Airwallex now. They’ve been great.
You can read more about the implementation in our blogpost: https://blog.thinkst.com/2024/12/its-baaack-credit-card-cana...
blog posts date back to at least 2015: https://leonjza.github.io/blog/2015/09/10/canarytokens---the...
I've been using it since 2015