ilia bdbf09a9ac feat: Implement voice I/O services (TICKET-006, TICKET-010, TICKET-014)
 TICKET-006: Wake-word Detection Service
- Implemented wake-word detection using openWakeWord
- HTTP/WebSocket server on port 8002
- Real-time detection with configurable threshold
- Event emission for ASR integration
- Location: home-voice-agent/wake-word/

 TICKET-010: ASR Service
- Implemented ASR using faster-whisper
- HTTP endpoint for file transcription
- WebSocket endpoint for streaming transcription
- Support for multiple audio formats
- Auto language detection
- GPU acceleration support
- Location: home-voice-agent/asr/

 TICKET-014: TTS Service
- Implemented TTS using Piper
- HTTP endpoint for text-to-speech synthesis
- Low-latency processing (< 500ms)
- Multiple voice support
- WAV audio output
- Location: home-voice-agent/tts/

 TICKET-047: Updated Hardware Purchases
- Marked Pi5 kit, SSD, microphone, and speakers as purchased
- Updated progress log with purchase status

📚 Documentation:
- Added VOICE_SERVICES_README.md with complete testing guide
- Each service includes README.md with usage instructions
- All services ready for Pi5 deployment

🧪 Testing:
- Created test files for each service
- All imports validated
- FastAPI apps created successfully
- Code passes syntax validation

🚀 Ready for:
- Pi5 deployment
- End-to-end voice flow testing
- Integration with MCP server

Files Added:
- wake-word/detector.py
- wake-word/server.py
- wake-word/requirements.txt
- wake-word/README.md
- wake-word/test_detector.py
- asr/service.py
- asr/server.py
- asr/requirements.txt
- asr/README.md
- asr/test_service.py
- tts/service.py
- tts/server.py
- tts/requirements.txt
- tts/README.md
- tts/test_service.py
- VOICE_SERVICES_README.md

Files Modified:
- tickets/done/TICKET-047_hardware-purchases.md

Files Moved:
- tickets/backlog/TICKET-006_prototype-wake-word-node.md → tickets/done/
- tickets/backlog/TICKET-010_streaming-asr-service.md → tickets/done/
- tickets/backlog/TICKET-014_tts-service.md → tickets/done/
2026-01-12 22:22:38 -05:00

3.5 KiB

Confirmation Flows

Confirmation system for high-risk actions to ensure user consent before executing sensitive operations.

Features

  • Risk Classification: Categorizes tools by risk level (LOW, MEDIUM, HIGH, CRITICAL)
  • Confirmation Tokens: Signed tokens for validated confirmations
  • Token Validation: Verifies tokens match intended actions
  • Expiration: Tokens expire after 5 minutes for security

Usage

Checking if Confirmation is Required

from safety.confirmations.flow import get_flow

flow = get_flow()

# Check if confirmation needed
requires, message = flow.check_confirmation_required(
    tool_name="send_email",
    to="user@example.com",
    subject="Important"
)

if requires:
    print(f"Confirmation needed: {message}")

Processing Confirmation Request

# Agent proposes action
response = flow.process_confirmation_request(
    tool_name="send_email",
    parameters={
        "to": "user@example.com",
        "subject": "Important",
        "body": "Message content"
    },
    session_id="session-123",
    user_id="user-1"
)

if response["confirmation_required"]:
    # Present message to user
    user_confirmed = ask_user(response["message"])
    
    if user_confirmed:
        # Validate token before executing
        is_valid, error = flow.validate_confirmation(
            token=response["token"],
            tool_name="send_email",
            parameters=response["parameters"]
        )
        
        if is_valid:
            # Execute tool
            execute_tool("send_email", **parameters)

In MCP Tools

Tools should check for confirmation tokens:

from safety.confirmations.flow import get_flow

flow = get_flow()

def execute(self, agent_type: str, confirmation_token: Optional[str] = None, **kwargs):
    # Check if confirmation required
    requires, message = flow.check_confirmation_required(self.name, **kwargs)
    
    if requires:
        if not confirmation_token:
            raise ValueError(f"Confirmation required: {message}")
        
        # Validate token
        is_valid, error = flow.validate_confirmation(
            confirmation_token,
            self.name,
            kwargs
        )
        
        if not is_valid:
            raise ValueError(f"Invalid confirmation: {error}")
    
    # Execute tool...

Risk Levels

LOW Risk

  • No confirmation needed
  • Examples: get_current_time, list_tasks, read_note

MEDIUM Risk

  • Optional confirmation
  • Examples: update_task_status, append_to_note, create_reminder

HIGH Risk

  • Confirmation required
  • Examples: send_email, create_calendar_event, set_smart_home_scene

CRITICAL Risk

  • Explicit confirmation required
  • Examples: send_email, delete_calendar_event, set_smart_home_scene

Token Security

  • Tokens are signed with HMAC-SHA256
  • Tokens expire after 5 minutes
  • Tokens are tied to specific tool and parameters
  • Secret key is stored securely in data/.confirmation_secret

Integration

With LLM

The LLM should:

  1. Detect high-risk tool calls
  2. Request confirmation from user
  3. Include token in tool call
  4. Tool validates token before execution

With Clients

Clients should:

  1. Present confirmation message to user
  2. Collect user response (Yes/No)
  3. Include token in tool call if confirmed
  4. Handle rejection gracefully

Future Enhancements

  • Voice confirmation support
  • Multi-step confirmations for critical actions
  • Confirmation history and audit log
  • Custom confirmation messages per tool
  • Rate limiting on confirmation requests