ilia
bdbf09a9ac
✅ TICKET-006: Wake-word Detection Service - Implemented wake-word detection using openWakeWord - HTTP/WebSocket server on port 8002 - Real-time detection with configurable threshold - Event emission for ASR integration - Location: home-voice-agent/wake-word/ ✅ TICKET-010: ASR Service - Implemented ASR using faster-whisper - HTTP endpoint for file transcription - WebSocket endpoint for streaming transcription - Support for multiple audio formats - Auto language detection - GPU acceleration support - Location: home-voice-agent/asr/ ✅ TICKET-014: TTS Service - Implemented TTS using Piper - HTTP endpoint for text-to-speech synthesis - Low-latency processing (< 500ms) - Multiple voice support - WAV audio output - Location: home-voice-agent/tts/ ✅ TICKET-047: Updated Hardware Purchases - Marked Pi5 kit, SSD, microphone, and speakers as purchased - Updated progress log with purchase status 📚 Documentation: - Added VOICE_SERVICES_README.md with complete testing guide - Each service includes README.md with usage instructions - All services ready for Pi5 deployment 🧪 Testing: - Created test files for each service - All imports validated - FastAPI apps created successfully - Code passes syntax validation 🚀 Ready for: - Pi5 deployment - End-to-end voice flow testing - Integration with MCP server Files Added: - wake-word/detector.py - wake-word/server.py - wake-word/requirements.txt - wake-word/README.md - wake-word/test_detector.py - asr/service.py - asr/server.py - asr/requirements.txt - asr/README.md - asr/test_service.py - tts/service.py - tts/server.py - tts/requirements.txt - tts/README.md - tts/test_service.py - VOICE_SERVICES_README.md Files Modified: - tickets/done/TICKET-047_hardware-purchases.md Files Moved: - tickets/backlog/TICKET-006_prototype-wake-word-node.md → tickets/done/ - tickets/backlog/TICKET-010_streaming-asr-service.md → tickets/done/ - tickets/backlog/TICKET-014_tts-service.md → tickets/done/
133 lines
3.8 KiB
Markdown
133 lines
3.8 KiB
Markdown
# Integration Design Documents
|
|
|
|
Design documents for optional integrations (email, calendar, smart home).
|
|
|
|
## Overview
|
|
|
|
These integrations are marked as "optional" and can be implemented after MVP. They require:
|
|
- External API access (with privacy considerations)
|
|
- Confirmation flows (high-risk actions)
|
|
- Boundary enforcement (work vs family separation)
|
|
|
|
## Email Integration (TICKET-036)
|
|
|
|
### Design Considerations
|
|
|
|
**Privacy**:
|
|
- Email access requires explicit user consent
|
|
- Consider local email server (IMAP/SMTP) vs cloud APIs
|
|
- Family agent should NOT access work email
|
|
|
|
**Confirmation Required**:
|
|
- Sending emails is CRITICAL risk
|
|
- Always require explicit confirmation
|
|
- Show email preview before sending
|
|
|
|
**Tools**:
|
|
- `list_recent_emails` - List recent emails (read-only)
|
|
- `read_email` - Read specific email
|
|
- `draft_email` - Create draft (no send)
|
|
- `send_email` - Send email (requires confirmation token)
|
|
|
|
**Implementation**:
|
|
- Use IMAP for reading (local email server)
|
|
- Use SMTP for sending (with authentication)
|
|
- Or use email API (Gmail, Outlook) with OAuth
|
|
|
|
## Calendar Integration (TICKET-037)
|
|
|
|
### Design Considerations
|
|
|
|
**Privacy**:
|
|
- Calendar access requires explicit user consent
|
|
- Separate calendars for work vs family
|
|
- Family agent should NOT access work calendar
|
|
|
|
**Confirmation Required**:
|
|
- Creating/modifying/deleting events is HIGH risk
|
|
- Always require explicit confirmation
|
|
- Show event details before confirming
|
|
|
|
**Tools**:
|
|
- `list_events` - List upcoming events
|
|
- `get_event` - Get event details
|
|
- `create_event` - Create event (requires confirmation)
|
|
- `update_event` - Update event (requires confirmation)
|
|
- `delete_event` - Delete event (requires confirmation)
|
|
|
|
**Implementation**:
|
|
- Use CalDAV for local calendar server
|
|
- Or use calendar API (Google Calendar, Outlook) with OAuth
|
|
- Support iCal format
|
|
|
|
## Smart Home Integration (TICKET-038)
|
|
|
|
### Design Considerations
|
|
|
|
**Privacy**:
|
|
- Smart home control is HIGH risk
|
|
- Require explicit confirmation for all actions
|
|
- Log all smart home actions
|
|
|
|
**Confirmation Required**:
|
|
- All smart home actions are CRITICAL risk
|
|
- Always require explicit confirmation
|
|
- Show action details before confirming
|
|
|
|
**Tools**:
|
|
- `list_devices` - List available devices
|
|
- `get_device_status` - Get device status
|
|
- `toggle_device` - Toggle device on/off (requires confirmation)
|
|
- `set_scene` - Set smart home scene (requires confirmation)
|
|
- `adjust_thermostat` - Adjust temperature (requires confirmation)
|
|
|
|
**Implementation**:
|
|
- Use Home Assistant API (if available)
|
|
- Or use device-specific APIs (Philips Hue, etc.)
|
|
- Abstract interface for multiple platforms
|
|
|
|
## Common Patterns
|
|
|
|
### Confirmation Flow
|
|
|
|
All high-risk integrations follow this pattern:
|
|
|
|
1. **Agent proposes action**: "I'll send an email to..."
|
|
2. **User confirms**: "Yes" or "No"
|
|
3. **Confirmation token generated**: Signed token with action details
|
|
4. **Tool validates token**: Before executing
|
|
5. **Action logged**: All actions logged for audit
|
|
|
|
### Boundary Enforcement
|
|
|
|
- **Family Agent**: Can only access family email/calendar
|
|
- **Work Agent**: Can access work email/calendar
|
|
- **Smart Home**: Both can access, but with confirmation
|
|
|
|
### Error Handling
|
|
|
|
- Network errors: Retry with backoff
|
|
- Authentication errors: Re-authenticate
|
|
- Permission errors: Log and notify user
|
|
|
|
## Implementation Priority
|
|
|
|
1. **Smart Home** (if Home Assistant available) - Most useful
|
|
2. **Calendar** - Useful for reminders and scheduling
|
|
3. **Email** - Less critical, can use web interface
|
|
|
|
## Security Considerations
|
|
|
|
- **OAuth Tokens**: Store securely, never in code
|
|
- **API Keys**: Use environment variables
|
|
- **Rate Limiting**: Respect API rate limits
|
|
- **Audit Logging**: Log all actions
|
|
- **Token Expiration**: Handle expired tokens gracefully
|
|
|
|
## Future Enhancements
|
|
|
|
- Voice confirmation ("Yes, send it")
|
|
- Batch operations
|
|
- Templates for common actions
|
|
- Integration with memory system (remember preferences)
|