ilia/homelab-notes
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add SearXNG LXC notes and LLM prompt for service documentation #3

Merged
ilia merged 2 commits from searxng into main 2026-01-07 12:22:33 -05:00
Conversation 0 Commits 2 Files Changed 2 +469
2 changed files with 469 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
README.md
View File

@ -21,3 +21,22 @@ Personal notes for my homelab: build logs, configs, and operational runbooks for
## Status ## Status
This is an actively evolving notebook. Some sections may be incomplete or “work in progress”. This is an actively evolving notebook. Some sections may be incomplete or “work in progress”.
### Using an LLM to draft new service notes
When adding a new container/VM/service, use this prompt with an LLM to generate a first draft of notes (similar style to the rest of this repo):
> You are helping me maintain my homelab-notes repo.
> I am adding or modifying ONE service/container/VM.
> Read the snippets I paste from my terminal (docker-compose, configs, IPs, curl outputs, etc.) and then produce a single Markdown file in the same style as other *-notes.md in this repo:
>
> - Start with a short description of what the service is and where it runs.
> - Capture design/architecture decisions (networking, reverse proxy, storage, security).
> - Document IPs, hostnames, ports, container names, and paths that are specific to my environment.
> - Include trimmed config examples (docker-compose, Caddy/nginx, app config) with comments where helpful.
> - Add a small "Deployment steps" section summarizing the exact commands I used (cd, docker compose up, systemctl, etc.).
> - Add brief "Troubleshooting notes" if we fixed any issues (e.g., wrong port binding, DNS, reverse proxy issues).
>
> Do NOT invent services or values that I did not show you.
> Only use the information I provide plus obvious inferences (e.g., 0.0.0.0:8080 means accessible on LAN).
> Output ONLY the Markdown for the new note file, no explanation around it.

450
proxmox1/searxng-lxc-notes.md Normal file
View File

@ -0,0 +1,450 @@
# SearXNG LXC Notes
Self-hosted SearXNG instance running in a Debian LXC, fronted by external Caddy VM, using Docker for SearXNG + Valkey/Redis.
---
## Design
- SearXNG runs in a Debian LXC (10.0.10.70) with Docker; only SearXNG and Valkey live in this container.
- Caddy runs on a separate VM and is the single external reverse proxy for the homelab, including `search.levkin.ca`.
- The SearXNG Docker setup uses "bring your own reverse proxy" mode from the official searxng-docker repo, so internal Caddy in the stack is removed.
---
## Network and URLs
- **LXC IP**: 10.0.10.70 (Proxmox container)
- **SearXNG container**: Listens on `0.0.0.0:8080` on the LXC, published via Docker ports: `- "8080:8080"`
- **Public URL**: `https://search.levkin.ca` via Caddy VM reverse proxy to `10.0.10.70:8080`
---
## Initial Setup
### 1. Create and Configure LXC Container
In Proxmox:
```bash
# Create Debian LXC container
# - Template: debian-12-standard
# - Storage: local-lvm
# - Network: Bridge vmbr0, IP 10.0.10.70/24
# - Resources: 2GB RAM, 1 CPU core minimum
```
### 2. Install Docker in LXC
SSH into the LXC container:
```bash
# Update system
apt update && apt upgrade -y
# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh
# Add current user to docker group (if needed)
usermod -aG docker $USER
# Verify installation
docker --version
docker compose version
```
### 3. Clone SearXNG Docker Repository
```bash
# Create directory
mkdir -p /opt/searxng-docker
cd /opt/searxng-docker
# Clone the official repository
git clone https://github.com/searxng/searxng-docker.git .
# Or download and extract if git is not available
```
---
## Docker Compose Configuration
**Location**: `/opt/searxng-docker/docker-compose.yaml` inside the LXC.
### Key Points
- Only two services: `redis` (Valkey) and `searxng`
- Internal Caddy service and volumes from the upstream searxng-docker example are removed
- SearXNG base URL is set via env so generated links use the public domain
### Complete docker-compose.yaml
```yaml
services:
redis:
container_name: redis
image: docker.io/valkey/valkey:8-alpine
command: valkey-server --save 30 1 --loglevel warning
restart: unless-stopped
networks:
- searxng
volumes:
- valkey-data2:/data
healthcheck:
test: ["CMD", "valkey-cli", "ping"]
interval: 10s
timeout: 3s
retries: 3
searxng:
container_name: searxng
image: docker.io/searxng/searxng:latest
restart: unless-stopped
networks:
- searxng
ports:
- "8080:8080" # Must NOT be bound to 127.0.0.1, Caddy VM needs LAN access
volumes:
- ./searxng:/etc/searxng:rw
- searxng-data:/var/cache/searxng:rw
environment:
- SEARXNG_BASE_URL=https://search.levkin.ca/
depends_on:
redis:
condition: service_healthy
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:8080/"]
interval: 30s
timeout: 10s
retries: 3
networks:
searxng:
driver: bridge
volumes:
valkey-data2:
searxng-data:
```
### Important Notes
- **Port binding**: Binding to `127.0.0.1:8080:8080` broke access from the Caddy VM; use `8080:8080` so it listens on `10.0.10.70`
- **Volumes**: Match the official documentation: config under `/etc/searxng`, cache under `/var/cache/searxng`
- **Health checks**: Added to ensure services are ready before accepting traffic
---
## SearXNG Configuration
### Settings File
Location: `/opt/searxng-docker/searxng/settings.yml`
Key configuration options:
```yaml
server:
secret_key: "CHANGE_ME_TO_SOMETHING_RANDOM" # Generate with: openssl rand -hex 32
base_url: "https://search.levkin.ca/"
bind_address: "0.0.0.0"
port: 8080
search:
formats:
- html
- json
- csv
- rss
engines:
# Enable/disable specific engines as needed
- name: google
- name: bing
- name: duckduckgo
- name: wikipedia
# Add more engines as desired
general:
instance_name: "SearXNG"
enable_metrics: false # Set to true if you want metrics endpoint
ui:
theme: simple # Options: simple, oscar, courgette
infinite_scroll: true
results_on_new_tab: false
outgoing:
max_request_timeout: 3.0
pool_connections: 100
useragent_suffix: "SearXNG"
redis:
url: "redis://redis:6379/0"
```
### Generate Secret Key
```bash
openssl rand -hex 32
```
Update the `secret_key` in `settings.yml` with the generated value.
---
## Deployment Steps
1. **SSH to LXC** and navigate to the directory:
```bash
cd /opt/searxng-docker
```
2. **Edit docker-compose.yaml** to:
- Remove the caddy service and related volumes
- Ensure searxng has `ports: - "8080:8080"` and the correct `SEARXNG_BASE_URL`
3. **Configure settings.yml** (if customizing):
```bash
# Copy default settings if not present
cp searxng/settings.yml.example searxng/settings.yml
# Edit settings.yml
nano searxng/settings.yml
```
4. **Start/restart stack**:
```bash
docker compose up -d
docker compose ps # Confirm searxng -> 0.0.0.0:8080->8080
```
5. **Check logs**:
```bash
docker compose logs -f searxng
docker compose logs -f redis
```
6. **Quick local test** from another host on LAN:
```bash
curl -v http://10.0.10.70:8080/ | head
```
Expected: HTML for SearXNG front page, status 200 from server granian.
---
## Caddy VM Configuration
Caddy runs on a separate VM and handles HTTPS + public DNS.
### Site Block
```caddy
search.levkin.ca {
encode gzip
reverse_proxy 10.0.10.70:8080 {
header_up Host {host}
header_up X-Real-IP {remote}
header_up X-Forwarded-For {remote}
header_up X-Forwarded-Proto {scheme}
}
}
```
### Notes
- Caddy handles TLS certificates and HTTPS for `search.levkin.ca`
- Upstream is plain HTTP on LAN; change IP/port here if the LXC moves
- This follows the "bring your own reverse proxy" approach recommended by the SearXNG Docker docs
- Headers are forwarded to preserve client information
---
## Firewall Configuration
### LXC Container
Ensure port 8080 is accessible from the Caddy VM:
```bash
# If using ufw
ufw allow from 10.0.10.0/24 to any port 8080
# Or if using iptables directly
iptables -A INPUT -s 10.0.10.0/24 -p tcp --dport 8080 -j ACCEPT
```
### Proxmox Host
Ensure the Proxmox firewall allows traffic between VMs/LXCs on the same network.
---
## Security Considerations
1. **Secret Key**: Always use a strong, randomly generated secret key in `settings.yml`
2. **Network Isolation**: SearXNG only needs to be accessible from the Caddy VM, not the entire network
3. **Updates**: Regularly update Docker images:
```bash
cd /opt/searxng-docker
docker compose pull
docker compose up -d
```
4. **Logging**: Consider disabling or limiting logging for privacy:
```yaml
# In settings.yml
general:
enable_metrics: false
```
5. **Rate Limiting**: Configure rate limiting in Caddy if needed to prevent abuse
---
## Backup and Restore
### Backup
```bash
# Backup configuration
tar -czf searxng-backup-$(date +%Y%m%d).tar.gz \
/opt/searxng-docker/searxng \
/opt/searxng-docker/docker-compose.yaml
# Backup volumes (optional, for cache)
docker run --rm -v searxng-docker_searxng-data:/data -v $(pwd):/backup \
alpine tar czf /backup/searxng-data-$(date +%Y%m%d).tar.gz /data
```
### Restore
```bash
# Extract configuration
tar -xzf searxng-backup-YYYYMMDD.tar.gz -C /
# Restore volumes (if needed)
docker run --rm -v searxng-docker_searxng-data:/data -v $(pwd):/backup \
alpine tar xzf /backup/searxng-data-YYYYMMDD.tar.gz -C /
```
---
## Monitoring and Maintenance
### Check Container Status
```bash
docker compose ps
docker compose logs --tail=50 searxng
```
### View Resource Usage
```bash
docker stats searxng redis
```
### Update Containers
```bash
cd /opt/searxng-docker
docker compose pull
docker compose up -d
docker compose image prune -f # Remove old images
```
### Clear Cache
```bash
docker compose exec searxng rm -rf /var/cache/searxng/*
docker compose restart searxng
```
---
## Troubleshooting
### Container Won't Start
```bash
# Check logs
docker compose logs searxng
# Verify configuration
docker compose config
# Test connectivity
docker compose exec searxng ping -c 3 redis
```
### Can't Access from Caddy VM
1. Verify port binding: `docker compose ps` should show `0.0.0.0:8080->8080/tcp`
2. Test from Caddy VM: `curl http://10.0.10.70:8080/`
3. Check firewall rules on LXC
4. Verify network connectivity: `ping 10.0.10.70` from Caddy VM
### SearXNG Returns Errors
1. Check Redis connectivity: `docker compose exec searxng ping redis`
2. Verify settings.yml syntax: `docker compose exec searxng cat /etc/searxng/settings.yml`
3. Check logs for specific errors: `docker compose logs searxng | grep -i error`
### Performance Issues
1. Monitor resource usage: `docker stats`
2. Check Redis performance: `docker compose exec redis valkey-cli info stats`
3. Consider increasing LXC resources if needed
4. Review search engine timeouts in settings.yml
---
## Privacy and Usage Notes
- **SearXNG is a metasearch engine**: It queries multiple search providers and aggregates results, instead of you talking to each provider directly
- **Self-hosting means**:
- No profiling or tracking by the instance; logs can be disabled or tightly controlled
- You no longer have to trust a random public SearXNG operator with your search queries
- **Intended use in this homelab**:
- Default search engine in browsers via `https://search.levkin.ca`
- Potential backend for LLM tools or scripts that need web search via SearXNG's APIs
---
## API Usage
SearXNG provides a JSON API for programmatic access:
```bash
# Example search query
curl "https://search.levkin.ca/search?q=example&format=json" | jq
# RSS feed
curl "https://search.levkin.ca/search?q=example&format=rss"
```
Useful for automation, LLM integrations, or custom search tools.
---
## Resource Limits
Recommended LXC container resources:
- **CPU**: 1-2 cores
- **RAM**: 2-4 GB
- **Disk**: 10-20 GB (for cache and logs)
- **Network**: Standard bridge connection
Adjust based on usage patterns and concurrent users.