Add comprehensive security protections for credentials
Enhanced .gitignore to protect: - Configuration files with secrets (config.yaml, .env files) - Mastodon credential files (*_clientcred.secret, *_usercred.secret) - API keys and tokens (*.key, *.token, *.pem, credentials.json) - Database files (may contain user data) - Backup files (may contain sensitive data) - LLM API key directories (.llm/, .openai/, .anthropic/) Added SECURITY.md documentation covering: - Where secrets are stored - What is/isn't committed to git - Best practices for credential management - Production secret management options - What to do if secrets are accidentally committed - Pre-commit hook examples - Security audit checklist Verified all patterns with test suite - all sensitive files properly ignored. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
31
.gitignore
vendored
31
.gitignore
vendored
@@ -32,21 +32,44 @@ ENV/
|
|||||||
*.swo
|
*.swo
|
||||||
*~
|
*~
|
||||||
|
|
||||||
# Configuration
|
# Configuration with secrets
|
||||||
config/config.yaml
|
config/config.yaml
|
||||||
|
config/config.local.yaml
|
||||||
.env
|
.env
|
||||||
|
.env.local
|
||||||
|
.env.*.local
|
||||||
|
|
||||||
# Database
|
# Mastodon credentials
|
||||||
|
*_clientcred.secret
|
||||||
|
*_usercred.secret
|
||||||
|
govbot_clientcred.secret
|
||||||
|
govbot_usercred.secret
|
||||||
|
|
||||||
|
# API keys and tokens
|
||||||
|
*.key
|
||||||
|
*.token
|
||||||
|
*.pem
|
||||||
|
credentials.json
|
||||||
|
secrets.json
|
||||||
|
api_keys.json
|
||||||
|
|
||||||
|
# Database (contains governance data)
|
||||||
*.db
|
*.db
|
||||||
*.sqlite
|
*.sqlite
|
||||||
*.sqlite3
|
*.sqlite3
|
||||||
|
|
||||||
# Logs
|
# Backups (may contain sensitive data)
|
||||||
|
*.backup
|
||||||
|
*.bak
|
||||||
|
|
||||||
|
# Logs (may contain tokens in debug output)
|
||||||
*.log
|
*.log
|
||||||
|
|
||||||
# AI/LLM
|
# AI/LLM (API keys stored here)
|
||||||
.llm/
|
.llm/
|
||||||
embeddings/
|
embeddings/
|
||||||
|
.openai/
|
||||||
|
.anthropic/
|
||||||
|
|
||||||
# Testing
|
# Testing
|
||||||
.pytest_cache/
|
.pytest_cache/
|
||||||
|
|||||||
366
SECURITY.md
Normal file
366
SECURITY.md
Normal file
@@ -0,0 +1,366 @@
|
|||||||
|
# Security Guide
|
||||||
|
|
||||||
|
This document explains how to handle sensitive data securely when using Govbot.
|
||||||
|
|
||||||
|
## Sensitive Data in Govbot
|
||||||
|
|
||||||
|
Govbot handles several types of sensitive information:
|
||||||
|
|
||||||
|
1. **Platform Credentials**
|
||||||
|
- Mastodon access tokens
|
||||||
|
- Discord bot tokens
|
||||||
|
- Telegram bot tokens
|
||||||
|
- OAuth client secrets
|
||||||
|
|
||||||
|
2. **AI Model API Keys**
|
||||||
|
- OpenAI API keys
|
||||||
|
- Anthropic (Claude) API keys
|
||||||
|
- Other cloud LLM providers
|
||||||
|
|
||||||
|
3. **Governance Data**
|
||||||
|
- Vote records
|
||||||
|
- User actions
|
||||||
|
- Audit logs (in database)
|
||||||
|
|
||||||
|
4. **Instance Configuration**
|
||||||
|
- Admin credentials
|
||||||
|
- Database passwords (if applicable)
|
||||||
|
- Secret keys
|
||||||
|
|
||||||
|
## Where Secrets Are Stored
|
||||||
|
|
||||||
|
### Configuration File
|
||||||
|
|
||||||
|
**Primary location**: `config/config.yaml`
|
||||||
|
|
||||||
|
This file contains all your credentials and should **NEVER be committed to git**.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
platform:
|
||||||
|
type: mastodon
|
||||||
|
mastodon:
|
||||||
|
instance_url: https://your-instance.social
|
||||||
|
access_token: YOUR_SECRET_TOKEN_HERE # ← SECRET!
|
||||||
|
client_secret: YOUR_CLIENT_SECRET # ← SECRET!
|
||||||
|
```
|
||||||
|
|
||||||
|
**Protection**: Already in `.gitignore` (line 36)
|
||||||
|
|
||||||
|
### Environment Variables
|
||||||
|
|
||||||
|
Alternative to config file, you can use environment variables:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export GOVBOT_PLATFORM__MASTODON__ACCESS_TOKEN="your_token"
|
||||||
|
export GOVBOT_PLATFORM__MASTODON__CLIENT_SECRET="your_secret"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Protection**: `.env` files are in `.gitignore` (line 37)
|
||||||
|
|
||||||
|
### LLM API Keys
|
||||||
|
|
||||||
|
The `llm` CLI tool stores API keys in `~/.llm/keys.json`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Set API keys
|
||||||
|
llm keys set openai
|
||||||
|
llm keys set anthropic
|
||||||
|
```
|
||||||
|
|
||||||
|
**Protection**: `.llm/` directory is in `.gitignore` (line 48)
|
||||||
|
|
||||||
|
### Database
|
||||||
|
|
||||||
|
The SQLite database (`govbot.db`) contains:
|
||||||
|
- All governance actions
|
||||||
|
- Vote records
|
||||||
|
- User information
|
||||||
|
- Audit trail
|
||||||
|
|
||||||
|
**Protection**: `*.db` files are in `.gitignore` (line 40)
|
||||||
|
|
||||||
|
## What IS Committed to Git
|
||||||
|
|
||||||
|
These files are safe to commit and are already in the repository:
|
||||||
|
|
||||||
|
✅ `config/config.example.yaml` - Template with placeholder values
|
||||||
|
✅ `constitution.md` - Your governance rules (usually public)
|
||||||
|
✅ `README.md` and other documentation
|
||||||
|
✅ All Python source code
|
||||||
|
✅ `.gitignore` - Protects your secrets
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
### Check What's Ignored
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# See what files git is ignoring
|
||||||
|
git status --ignored
|
||||||
|
|
||||||
|
# Check if a specific file would be ignored
|
||||||
|
git check-ignore -v config/config.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test Before Committing
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# List files that would be committed
|
||||||
|
git status
|
||||||
|
|
||||||
|
# If you see any of these, STOP:
|
||||||
|
# - config/config.yaml
|
||||||
|
# - *.token, *.key, *.secret files
|
||||||
|
# - .env files
|
||||||
|
# - govbot.db
|
||||||
|
```
|
||||||
|
|
||||||
|
## Creating Your Config File
|
||||||
|
|
||||||
|
### Step 1: Copy Template
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp config/config.example.yaml config/config.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
The copied file (`config.yaml`) is automatically ignored by git.
|
||||||
|
|
||||||
|
### Step 2: Add Your Secrets
|
||||||
|
|
||||||
|
Edit `config/config.yaml` with your actual credentials:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Use a text editor
|
||||||
|
nano config/config.yaml
|
||||||
|
|
||||||
|
# Or use environment variable editor
|
||||||
|
export EDITOR=nano
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 3: Verify It's Ignored
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git status
|
||||||
|
# Should NOT show config/config.yaml as untracked
|
||||||
|
```
|
||||||
|
|
||||||
|
## Best Practices
|
||||||
|
|
||||||
|
### 1. Never Hard-Code Secrets
|
||||||
|
|
||||||
|
❌ **Bad** - Hard-coding in source files:
|
||||||
|
```python
|
||||||
|
# DON'T DO THIS!
|
||||||
|
access_token = "abc123secret"
|
||||||
|
```
|
||||||
|
|
||||||
|
✅ **Good** - Load from config:
|
||||||
|
```python
|
||||||
|
access_token = config.platform.mastodon.access_token
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Use Different Secrets for Different Environments
|
||||||
|
|
||||||
|
```
|
||||||
|
config/
|
||||||
|
├── config.example.yaml # Template (committed)
|
||||||
|
├── config.yaml # Production (NOT committed)
|
||||||
|
├── config.dev.yaml # Development (NOT committed)
|
||||||
|
└── config.test.yaml # Testing (NOT committed)
|
||||||
|
```
|
||||||
|
|
||||||
|
Load different configs:
|
||||||
|
```bash
|
||||||
|
python -m src.govbot --config config/config.dev.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Rotate Tokens Periodically
|
||||||
|
|
||||||
|
- Change access tokens every 3-6 months
|
||||||
|
- Rotate immediately if compromised
|
||||||
|
- Use separate tokens for testing vs. production
|
||||||
|
|
||||||
|
### 4. Limit Token Scopes
|
||||||
|
|
||||||
|
Only grant the permissions you need:
|
||||||
|
|
||||||
|
**Mastodon**:
|
||||||
|
- ✅ `read`, `write` - Basic operation
|
||||||
|
- ✅ `admin:read`, `admin:write` - Only if bot needs admin powers
|
||||||
|
- ❌ Don't grant scopes you don't use
|
||||||
|
|
||||||
|
**OpenAI/Anthropic**:
|
||||||
|
- Set usage limits on API keys
|
||||||
|
- Use separate keys for development vs. production
|
||||||
|
|
||||||
|
### 5. Secure Your Environment
|
||||||
|
|
||||||
|
**Development machine**:
|
||||||
|
```bash
|
||||||
|
# Set restrictive permissions on config
|
||||||
|
chmod 600 config/config.yaml
|
||||||
|
|
||||||
|
# Don't share your config directory
|
||||||
|
```
|
||||||
|
|
||||||
|
**Production server**:
|
||||||
|
```bash
|
||||||
|
# Use a service account
|
||||||
|
sudo useradd -r -s /bin/false govbot
|
||||||
|
|
||||||
|
# Restrict file access
|
||||||
|
chown govbot:govbot config/config.yaml
|
||||||
|
chmod 400 config/config.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6. Use Secret Management (Production)
|
||||||
|
|
||||||
|
For production deployments, consider:
|
||||||
|
|
||||||
|
**Docker Secrets**:
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
govbot:
|
||||||
|
secrets:
|
||||||
|
- mastodon_token
|
||||||
|
- openai_key
|
||||||
|
```
|
||||||
|
|
||||||
|
**Kubernetes Secrets**:
|
||||||
|
```bash
|
||||||
|
kubectl create secret generic govbot-secrets \
|
||||||
|
--from-literal=mastodon-token=your_token
|
||||||
|
```
|
||||||
|
|
||||||
|
**Cloud Provider Secret Managers**:
|
||||||
|
- AWS Secrets Manager
|
||||||
|
- GCP Secret Manager
|
||||||
|
- Azure Key Vault
|
||||||
|
- HashiCorp Vault
|
||||||
|
|
||||||
|
**Environment variables in systemd**:
|
||||||
|
```ini
|
||||||
|
[Service]
|
||||||
|
Environment="GOVBOT_PLATFORM__MASTODON__ACCESS_TOKEN=your_token"
|
||||||
|
EnvironmentFile=/etc/govbot/secrets.env
|
||||||
|
```
|
||||||
|
|
||||||
|
## What If I Accidentally Commit a Secret?
|
||||||
|
|
||||||
|
### If You Haven't Pushed Yet
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Remove the file from git (but keep it locally)
|
||||||
|
git rm --cached config/config.yaml
|
||||||
|
|
||||||
|
# Amend the commit
|
||||||
|
git commit --amend
|
||||||
|
|
||||||
|
# Verify it's gone
|
||||||
|
git show
|
||||||
|
```
|
||||||
|
|
||||||
|
### If You Already Pushed
|
||||||
|
|
||||||
|
**You must:**
|
||||||
|
|
||||||
|
1. **Rotate the secret immediately** - Consider it compromised
|
||||||
|
- Generate new Mastodon access token
|
||||||
|
- Create new API keys
|
||||||
|
- Update your local config
|
||||||
|
|
||||||
|
2. **Remove from git history** (advanced):
|
||||||
|
```bash
|
||||||
|
# Use BFG Repo-Cleaner or git-filter-branch
|
||||||
|
# See: https://rtyley.github.io/bfg-repo-cleaner/
|
||||||
|
|
||||||
|
bfg --replace-text passwords.txt
|
||||||
|
git reflog expire --expire=now --all
|
||||||
|
git gc --prune=now --aggressive
|
||||||
|
git push --force
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Notify collaborators** to pull fresh history
|
||||||
|
|
||||||
|
4. **Check access logs** for unauthorized use
|
||||||
|
|
||||||
|
## Secure Development Workflow
|
||||||
|
|
||||||
|
### Daily Development
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Pull latest code
|
||||||
|
git pull
|
||||||
|
|
||||||
|
# 2. Ensure config exists (one time)
|
||||||
|
if [ ! -f config/config.yaml ]; then
|
||||||
|
cp config/config.example.yaml config/config.yaml
|
||||||
|
echo "⚠️ Edit config/config.yaml with your credentials"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 3. Run the bot
|
||||||
|
python -m src.govbot
|
||||||
|
```
|
||||||
|
|
||||||
|
### Before Every Commit
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check what you're committing
|
||||||
|
git status
|
||||||
|
git diff --cached
|
||||||
|
|
||||||
|
# Verify no secrets
|
||||||
|
git diff --cached | grep -i "token\|secret\|password\|key"
|
||||||
|
|
||||||
|
# If anything matches, DON'T COMMIT
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pre-commit Hook (Optional)
|
||||||
|
|
||||||
|
Create `.git/hooks/pre-commit`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# Check for potential secrets
|
||||||
|
if git diff --cached | grep -iE "(token|secret|password|api[_-]?key)" | grep -v "example"; then
|
||||||
|
echo "⚠️ WARNING: Potential secret detected in commit!"
|
||||||
|
echo "Review your changes carefully."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
```
|
||||||
|
|
||||||
|
Make it executable:
|
||||||
|
```bash
|
||||||
|
chmod +x .git/hooks/pre-commit
|
||||||
|
```
|
||||||
|
|
||||||
|
## Audit Checklist
|
||||||
|
|
||||||
|
Run this before going to production:
|
||||||
|
|
||||||
|
- [ ] `config/config.yaml` is in `.gitignore`
|
||||||
|
- [ ] No secrets in git history: `git log -p | grep -i "token"`
|
||||||
|
- [ ] Config file has restrictive permissions: `ls -la config/`
|
||||||
|
- [ ] `.env` files are ignored
|
||||||
|
- [ ] Database is not committed
|
||||||
|
- [ ] API keys are rotated from development
|
||||||
|
- [ ] Tokens have minimal required scopes
|
||||||
|
- [ ] Production uses separate credentials from dev
|
||||||
|
- [ ] Backup strategy excludes secrets or encrypts them
|
||||||
|
- [ ] Team knows not to commit secrets
|
||||||
|
|
||||||
|
## Resources
|
||||||
|
|
||||||
|
- [GitHub's Secret Scanning](https://docs.github.com/en/code-security/secret-scanning)
|
||||||
|
- [OWASP Secrets Management Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html)
|
||||||
|
- [12 Factor App: Config](https://12factor.net/config)
|
||||||
|
|
||||||
|
## Questions?
|
||||||
|
|
||||||
|
If you suspect a security issue:
|
||||||
|
1. Rotate affected credentials immediately
|
||||||
|
2. Check access logs
|
||||||
|
3. Open a security issue (mark as security vulnerability)
|
||||||
|
|
||||||
|
**Remember**: It's better to be paranoid about secrets than to have them leaked!
|
||||||
Reference in New Issue
Block a user