For the vibe coders, developers, and honestly anyone who uses AI a lot: don't casually drop API keys, tokens, or sensitive data into your prompts.
The reason is that your prompts and the model's responses can be picked up by LLM observability. Put simply, user prompts and AI responses get tracked for evaluation.
What these tools actually capture
At a lot of AI companies, observability tools are used to see input and output tokens, cost per request, the user's prompt, and the model's response. The goal is debugging, analyzing user experience, and checking how often the AI hands back a hallucinated answer.
I've used one of the open-source options myself: Langfuse. It can be self-hosted, it's free, and it's fairly complete — tracking prompts, responses, token usage, latency, and even output-quality evaluation.
Nothing you send is automatically private
Using it made me realize that whatever we send to an AI shouldn't be treated as private by default. If you put an API key, an access token, a credential, or internal data into a prompt, there's a real risk it gets recorded in the observability system and saved into the company's database.
Use guardrails
So my take is that using AI still needs guardrails and limits:
- Don't paste credentials straight into a prompt.
- Mask sensitive data.
- Use environment variables.
- Check the policy of the tools you're using.
AI is great for moving faster, but don't let it turn into a security leak. Stay careful out there.