Add comprehensive security protections for credentials
Enhanced .gitignore to protect: - Configuration files with secrets (config.yaml, .env files) - Mastodon credential files (*_clientcred.secret, *_usercred.secret) - API keys and tokens (*.key, *.token, *.pem, credentials.json) - Database files (may contain user data) - Backup files (may contain sensitive data) - LLM API key directories (.llm/, .openai/, .anthropic/) Added SECURITY.md documentation covering: - Where secrets are stored - What is/isn't committed to git - Best practices for credential management - Production secret management options - What to do if secrets are accidentally committed - Pre-commit hook examples - Security audit checklist Verified all patterns with test suite - all sensitive files properly ignored. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Nathan Schneider
31
.gitignore
vendored
31
.gitignore
vendored
@@ -32,21 +32,44 @@ ENV/
|
||||
*.swo
|
||||
*~
|
||||
|
||||
# Configuration
|
||||
# Configuration with secrets
|
||||
config/config.yaml
|
||||
config/config.local.yaml
|
||||
.env
|
||||
.env.local
|
||||
.env.*.local
|
||||
|
||||
# Database
|
||||
# Mastodon credentials
|
||||
*_clientcred.secret
|
||||
*_usercred.secret
|
||||
govbot_clientcred.secret
|
||||
govbot_usercred.secret
|
||||
|
||||
# API keys and tokens
|
||||
*.key
|
||||
*.token
|
||||
*.pem
|
||||
credentials.json
|
||||
secrets.json
|
||||
api_keys.json
|
||||
|
||||
# Database (contains governance data)
|
||||
*.db
|
||||
*.sqlite
|
||||
*.sqlite3
|
||||
|
||||
# Logs
|
||||
# Backups (may contain sensitive data)
|
||||
*.backup
|
||||
*.bak
|
||||
|
||||
# Logs (may contain tokens in debug output)
|
||||
*.log
|
||||
|
||||
# AI/LLM
|
||||
# AI/LLM (API keys stored here)
|
||||
.llm/
|
||||
embeddings/
|
||||
.openai/
|
||||
.anthropic/
|
||||
|
||||
# Testing
|
||||
.pytest_cache/
|
||||
|
||||
366
SECURITY.md
Normal file
366
SECURITY.md
Normal file
@@ -0,0 +1,366 @@
|
||||
# Security Guide
|
||||
|
||||
This document explains how to handle sensitive data securely when using Govbot.
|
||||
|
||||
## Sensitive Data in Govbot
|
||||
|
||||
Govbot handles several types of sensitive information:
|
||||
|
||||
1. **Platform Credentials**
|
||||
- Mastodon access tokens
|
||||
- Discord bot tokens
|
||||
- Telegram bot tokens
|
||||
- OAuth client secrets
|
||||
|
||||
2. **AI Model API Keys**
|
||||
- OpenAI API keys
|
||||
- Anthropic (Claude) API keys
|
||||
- Other cloud LLM providers
|
||||
|
||||
3. **Governance Data**
|
||||
- Vote records
|
||||
- User actions
|
||||
- Audit logs (in database)
|
||||
|
||||
4. **Instance Configuration**
|
||||
- Admin credentials
|
||||
- Database passwords (if applicable)
|
||||
- Secret keys
|
||||
|
||||
## Where Secrets Are Stored
|
||||
|
||||
### Configuration File
|
||||
|
||||
**Primary location**: `config/config.yaml`
|
||||
|
||||
This file contains all your credentials and should **NEVER be committed to git**.
|
||||
|
||||
```yaml
|
||||
platform:
|
||||
type: mastodon
|
||||
mastodon:
|
||||
instance_url: https://your-instance.social
|
||||
access_token: YOUR_SECRET_TOKEN_HERE # ← SECRET!
|
||||
client_secret: YOUR_CLIENT_SECRET # ← SECRET!
|
||||
```
|
||||
|
||||
**Protection**: Already in `.gitignore` (line 36)
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Alternative to config file, you can use environment variables:
|
||||
|
||||
```bash
|
||||
export GOVBOT_PLATFORM__MASTODON__ACCESS_TOKEN="your_token"
|
||||
export GOVBOT_PLATFORM__MASTODON__CLIENT_SECRET="your_secret"
|
||||
```
|
||||
|
||||
**Protection**: `.env` files are in `.gitignore` (line 37)
|
||||
|
||||
### LLM API Keys
|
||||
|
||||
The `llm` CLI tool stores API keys in `~/.llm/keys.json`
|
||||
|
||||
```bash
|
||||
# Set API keys
|
||||
llm keys set openai
|
||||
llm keys set anthropic
|
||||
```
|
||||
|
||||
**Protection**: `.llm/` directory is in `.gitignore` (line 48)
|
||||
|
||||
### Database
|
||||
|
||||
The SQLite database (`govbot.db`) contains:
|
||||
- All governance actions
|
||||
- Vote records
|
||||
- User information
|
||||
- Audit trail
|
||||
|
||||
**Protection**: `*.db` files are in `.gitignore` (line 40)
|
||||
|
||||
## What IS Committed to Git
|
||||
|
||||
These files are safe to commit and are already in the repository:
|
||||
|
||||
✅ `config/config.example.yaml` - Template with placeholder values
|
||||
✅ `constitution.md` - Your governance rules (usually public)
|
||||
✅ `README.md` and other documentation
|
||||
✅ All Python source code
|
||||
✅ `.gitignore` - Protects your secrets
|
||||
|
||||
## Verification
|
||||
|
||||
### Check What's Ignored
|
||||
|
||||
```bash
|
||||
# See what files git is ignoring
|
||||
git status --ignored
|
||||
|
||||
# Check if a specific file would be ignored
|
||||
git check-ignore -v config/config.yaml
|
||||
```
|
||||
|
||||
### Test Before Committing
|
||||
|
||||
```bash
|
||||
# List files that would be committed
|
||||
git status
|
||||
|
||||
# If you see any of these, STOP:
|
||||
# - config/config.yaml
|
||||
# - *.token, *.key, *.secret files
|
||||
# - .env files
|
||||
# - govbot.db
|
||||
```
|
||||
|
||||
## Creating Your Config File
|
||||
|
||||
### Step 1: Copy Template
|
||||
|
||||
```bash
|
||||
cp config/config.example.yaml config/config.yaml
|
||||
```
|
||||
|
||||
The copied file (`config.yaml`) is automatically ignored by git.
|
||||
|
||||
### Step 2: Add Your Secrets
|
||||
|
||||
Edit `config/config.yaml` with your actual credentials:
|
||||
|
||||
```bash
|
||||
# Use a text editor
|
||||
nano config/config.yaml
|
||||
|
||||
# Or use environment variable editor
|
||||
export EDITOR=nano
|
||||
```
|
||||
|
||||
### Step 3: Verify It's Ignored
|
||||
|
||||
```bash
|
||||
git status
|
||||
# Should NOT show config/config.yaml as untracked
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Never Hard-Code Secrets
|
||||
|
||||
❌ **Bad** - Hard-coding in source files:
|
||||
```python
|
||||
# DON'T DO THIS!
|
||||
access_token = "abc123secret"
|
||||
```
|
||||
|
||||
✅ **Good** - Load from config:
|
||||
```python
|
||||
access_token = config.platform.mastodon.access_token
|
||||
```
|
||||
|
||||
### 2. Use Different Secrets for Different Environments
|
||||
|
||||
```
|
||||
config/
|
||||
├── config.example.yaml # Template (committed)
|
||||
├── config.yaml # Production (NOT committed)
|
||||
├── config.dev.yaml # Development (NOT committed)
|
||||
└── config.test.yaml # Testing (NOT committed)
|
||||
```
|
||||
|
||||
Load different configs:
|
||||
```bash
|
||||
python -m src.govbot --config config/config.dev.yaml
|
||||
```
|
||||
|
||||
### 3. Rotate Tokens Periodically
|
||||
|
||||
- Change access tokens every 3-6 months
|
||||
- Rotate immediately if compromised
|
||||
- Use separate tokens for testing vs. production
|
||||
|
||||
### 4. Limit Token Scopes
|
||||
|
||||
Only grant the permissions you need:
|
||||
|
||||
**Mastodon**:
|
||||
- ✅ `read`, `write` - Basic operation
|
||||
- ✅ `admin:read`, `admin:write` - Only if bot needs admin powers
|
||||
- ❌ Don't grant scopes you don't use
|
||||
|
||||
**OpenAI/Anthropic**:
|
||||
- Set usage limits on API keys
|
||||
- Use separate keys for development vs. production
|
||||
|
||||
### 5. Secure Your Environment
|
||||
|
||||
**Development machine**:
|
||||
```bash
|
||||
# Set restrictive permissions on config
|
||||
chmod 600 config/config.yaml
|
||||
|
||||
# Don't share your config directory
|
||||
```
|
||||
|
||||
**Production server**:
|
||||
```bash
|
||||
# Use a service account
|
||||
sudo useradd -r -s /bin/false govbot
|
||||
|
||||
# Restrict file access
|
||||
chown govbot:govbot config/config.yaml
|
||||
chmod 400 config/config.yaml
|
||||
```
|
||||
|
||||
### 6. Use Secret Management (Production)
|
||||
|
||||
For production deployments, consider:
|
||||
|
||||
**Docker Secrets**:
|
||||
```yaml
|
||||
services:
|
||||
govbot:
|
||||
secrets:
|
||||
- mastodon_token
|
||||
- openai_key
|
||||
```
|
||||
|
||||
**Kubernetes Secrets**:
|
||||
```bash
|
||||
kubectl create secret generic govbot-secrets \
|
||||
--from-literal=mastodon-token=your_token
|
||||
```
|
||||
|
||||
**Cloud Provider Secret Managers**:
|
||||
- AWS Secrets Manager
|
||||
- GCP Secret Manager
|
||||
- Azure Key Vault
|
||||
- HashiCorp Vault
|
||||
|
||||
**Environment variables in systemd**:
|
||||
```ini
|
||||
[Service]
|
||||
Environment="GOVBOT_PLATFORM__MASTODON__ACCESS_TOKEN=your_token"
|
||||
EnvironmentFile=/etc/govbot/secrets.env
|
||||
```
|
||||
|
||||
## What If I Accidentally Commit a Secret?
|
||||
|
||||
### If You Haven't Pushed Yet
|
||||
|
||||
```bash
|
||||
# Remove the file from git (but keep it locally)
|
||||
git rm --cached config/config.yaml
|
||||
|
||||
# Amend the commit
|
||||
git commit --amend
|
||||
|
||||
# Verify it's gone
|
||||
git show
|
||||
```
|
||||
|
||||
### If You Already Pushed
|
||||
|
||||
**You must:**
|
||||
|
||||
1. **Rotate the secret immediately** - Consider it compromised
|
||||
- Generate new Mastodon access token
|
||||
- Create new API keys
|
||||
- Update your local config
|
||||
|
||||
2. **Remove from git history** (advanced):
|
||||
```bash
|
||||
# Use BFG Repo-Cleaner or git-filter-branch
|
||||
# See: https://rtyley.github.io/bfg-repo-cleaner/
|
||||
|
||||
bfg --replace-text passwords.txt
|
||||
git reflog expire --expire=now --all
|
||||
git gc --prune=now --aggressive
|
||||
git push --force
|
||||
```
|
||||
|
||||
3. **Notify collaborators** to pull fresh history
|
||||
|
||||
4. **Check access logs** for unauthorized use
|
||||
|
||||
## Secure Development Workflow
|
||||
|
||||
### Daily Development
|
||||
|
||||
```bash
|
||||
# 1. Pull latest code
|
||||
git pull
|
||||
|
||||
# 2. Ensure config exists (one time)
|
||||
if [ ! -f config/config.yaml ]; then
|
||||
cp config/config.example.yaml config/config.yaml
|
||||
echo "⚠️ Edit config/config.yaml with your credentials"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 3. Run the bot
|
||||
python -m src.govbot
|
||||
```
|
||||
|
||||
### Before Every Commit
|
||||
|
||||
```bash
|
||||
# Check what you're committing
|
||||
git status
|
||||
git diff --cached
|
||||
|
||||
# Verify no secrets
|
||||
git diff --cached | grep -i "token\|secret\|password\|key"
|
||||
|
||||
# If anything matches, DON'T COMMIT
|
||||
```
|
||||
|
||||
### Pre-commit Hook (Optional)
|
||||
|
||||
Create `.git/hooks/pre-commit`:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
# Check for potential secrets
|
||||
if git diff --cached | grep -iE "(token|secret|password|api[_-]?key)" | grep -v "example"; then
|
||||
echo "⚠️ WARNING: Potential secret detected in commit!"
|
||||
echo "Review your changes carefully."
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
Make it executable:
|
||||
```bash
|
||||
chmod +x .git/hooks/pre-commit
|
||||
```
|
||||
|
||||
## Audit Checklist
|
||||
|
||||
Run this before going to production:
|
||||
|
||||
- [ ] `config/config.yaml` is in `.gitignore`
|
||||
- [ ] No secrets in git history: `git log -p | grep -i "token"`
|
||||
- [ ] Config file has restrictive permissions: `ls -la config/`
|
||||
- [ ] `.env` files are ignored
|
||||
- [ ] Database is not committed
|
||||
- [ ] API keys are rotated from development
|
||||
- [ ] Tokens have minimal required scopes
|
||||
- [ ] Production uses separate credentials from dev
|
||||
- [ ] Backup strategy excludes secrets or encrypts them
|
||||
- [ ] Team knows not to commit secrets
|
||||
|
||||
## Resources
|
||||
|
||||
- [GitHub's Secret Scanning](https://docs.github.com/en/code-security/secret-scanning)
|
||||
- [OWASP Secrets Management Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html)
|
||||
- [12 Factor App: Config](https://12factor.net/config)
|
||||
|
||||
## Questions?
|
||||
|
||||
If you suspect a security issue:
|
||||
1. Rotate affected credentials immediately
|
||||
2. Check access logs
|
||||
3. Open a security issue (mark as security vulnerability)
|
||||
|
||||
**Remember**: It's better to be paranoid about secrets than to have them leaked!
|
||||
Reference in New Issue
Block a user