✅ TICKET-006: Wake-word Detection Service - Implemented wake-word detection using openWakeWord - HTTP/WebSocket server on port 8002 - Real-time detection with configurable threshold - Event emission for ASR integration - Location: home-voice-agent/wake-word/ ✅ TICKET-010: ASR Service - Implemented ASR using faster-whisper - HTTP endpoint for file transcription - WebSocket endpoint for streaming transcription - Support for multiple audio formats - Auto language detection - GPU acceleration support - Location: home-voice-agent/asr/ ✅ TICKET-014: TTS Service - Implemented TTS using Piper - HTTP endpoint for text-to-speech synthesis - Low-latency processing (< 500ms) - Multiple voice support - WAV audio output - Location: home-voice-agent/tts/ ✅ TICKET-047: Updated Hardware Purchases - Marked Pi5 kit, SSD, microphone, and speakers as purchased - Updated progress log with purchase status 📚 Documentation: - Added VOICE_SERVICES_README.md with complete testing guide - Each service includes README.md with usage instructions - All services ready for Pi5 deployment 🧪 Testing: - Created test files for each service - All imports validated - FastAPI apps created successfully - Code passes syntax validation 🚀 Ready for: - Pi5 deployment - End-to-end voice flow testing - Integration with MCP server Files Added: - wake-word/detector.py - wake-word/server.py - wake-word/requirements.txt - wake-word/README.md - wake-word/test_detector.py - asr/service.py - asr/server.py - asr/requirements.txt - asr/README.md - asr/test_service.py - tts/service.py - tts/server.py - tts/requirements.txt - tts/README.md - tts/test_service.py - VOICE_SERVICES_README.md Files Modified: - tickets/done/TICKET-047_hardware-purchases.md Files Moved: - tickets/backlog/TICKET-006_prototype-wake-word-node.md → tickets/done/ - tickets/backlog/TICKET-010_streaming-asr-service.md → tickets/done/ - tickets/backlog/TICKET-014_tts-service.md → tickets/done/
3.8 KiB
3.8 KiB
Integration Design Documents
Design documents for optional integrations (email, calendar, smart home).
Overview
These integrations are marked as "optional" and can be implemented after MVP. They require:
- External API access (with privacy considerations)
- Confirmation flows (high-risk actions)
- Boundary enforcement (work vs family separation)
Email Integration (TICKET-036)
Design Considerations
Privacy:
- Email access requires explicit user consent
- Consider local email server (IMAP/SMTP) vs cloud APIs
- Family agent should NOT access work email
Confirmation Required:
- Sending emails is CRITICAL risk
- Always require explicit confirmation
- Show email preview before sending
Tools:
list_recent_emails- List recent emails (read-only)read_email- Read specific emaildraft_email- Create draft (no send)send_email- Send email (requires confirmation token)
Implementation:
- Use IMAP for reading (local email server)
- Use SMTP for sending (with authentication)
- Or use email API (Gmail, Outlook) with OAuth
Calendar Integration (TICKET-037)
Design Considerations
Privacy:
- Calendar access requires explicit user consent
- Separate calendars for work vs family
- Family agent should NOT access work calendar
Confirmation Required:
- Creating/modifying/deleting events is HIGH risk
- Always require explicit confirmation
- Show event details before confirming
Tools:
list_events- List upcoming eventsget_event- Get event detailscreate_event- Create event (requires confirmation)update_event- Update event (requires confirmation)delete_event- Delete event (requires confirmation)
Implementation:
- Use CalDAV for local calendar server
- Or use calendar API (Google Calendar, Outlook) with OAuth
- Support iCal format
Smart Home Integration (TICKET-038)
Design Considerations
Privacy:
- Smart home control is HIGH risk
- Require explicit confirmation for all actions
- Log all smart home actions
Confirmation Required:
- All smart home actions are CRITICAL risk
- Always require explicit confirmation
- Show action details before confirming
Tools:
list_devices- List available devicesget_device_status- Get device statustoggle_device- Toggle device on/off (requires confirmation)set_scene- Set smart home scene (requires confirmation)adjust_thermostat- Adjust temperature (requires confirmation)
Implementation:
- Use Home Assistant API (if available)
- Or use device-specific APIs (Philips Hue, etc.)
- Abstract interface for multiple platforms
Common Patterns
Confirmation Flow
All high-risk integrations follow this pattern:
- Agent proposes action: "I'll send an email to..."
- User confirms: "Yes" or "No"
- Confirmation token generated: Signed token with action details
- Tool validates token: Before executing
- Action logged: All actions logged for audit
Boundary Enforcement
- Family Agent: Can only access family email/calendar
- Work Agent: Can access work email/calendar
- Smart Home: Both can access, but with confirmation
Error Handling
- Network errors: Retry with backoff
- Authentication errors: Re-authenticate
- Permission errors: Log and notify user
Implementation Priority
- Smart Home (if Home Assistant available) - Most useful
- Calendar - Useful for reminders and scheduling
- Email - Less critical, can use web interface
Security Considerations
- OAuth Tokens: Store securely, never in code
- API Keys: Use environment variables
- Rate Limiting: Respect API rate limits
- Audit Logging: Log all actions
- Token Expiration: Handle expired tokens gracefully
Future Enhancements
- Voice confirmation ("Yes, send it")
- Batch operations
- Templates for common actions
- Integration with memory system (remember preferences)