# Video Viewing Analysis

## Overview
This document analyzes how videos can be viewed in the PunimTag Viewer application, now that the `photos` table has a `media_type` field that can be either "image" or "video".

## Current Architecture

### Database Schema
- **Photo Model**: Currently missing `media_type` field in Prisma schema (needs to be added)
- **Database**: Has `media_type` field at line 44 (Text, Default: "image", Values: "image" or "video", Indexed)

### Current Photo Display Flow

1. **PhotoGrid Component** (`components/PhotoGrid.tsx`)
   - Displays photos in a grid using Next.js `Image` component
   - Uses `getImageSrc()` to determine image source (URL vs API proxy)
   - Handles face detection hover tooltips
   - Clicking opens photo in viewer

2. **PhotoViewerClient Component** (`components/PhotoViewerClient.tsx`)
   - Full-screen photo viewer with zoom, pan, navigation
   - Uses Next.js `Image` component exclusively
   - Supports slideshow, face identification, reporting
   - Keyboard navigation (arrows, zoom, escape)

3. **Image API Route** (`app/api/photos/[id]/image/route.ts`)
   - Serves images from file system or URLs
   - Applies watermarks for non-logged-in users
   - Uses Sharp for image processing
   - Returns image buffers with appropriate content types

4. **Data Flow**
   - Photos queried via Prisma from `photos` table
   - Includes faces, tags, and related data
   - Serialized and sent to frontend
   - Path can be HTTP/HTTPS URL or file system path

## Required Changes for Video Support

### 1. Database Schema Update

**File**: `prisma/schema.prisma`

Add `media_type` field to Photo model:
```prisma
model Photo {
  id         Int      @id @default(autoincrement())
  path       String   @unique
  filename   String
  mediaType  String   @default("image") @map("media_type") // "image" or "video"
  dateAdded  DateTime @default(now()) @map("date_added")
  dateTaken  DateTime? @map("date_taken") @db.Date
  processed  Boolean  @default(false)
  // ... rest of fields
}
```

**Action Required**: Run `npx prisma db pull` to sync schema, then `npx prisma generate`

### 2. PhotoGrid Component Updates

**File**: `components/PhotoGrid.tsx`

**Changes Needed**:
- Detect `media_type` field from photo data
- For videos: Render video thumbnail or `<video>` element with `poster` attribute
- For images: Keep existing `Image` component
- Add video play icon overlay on thumbnails
- Handle video click to open in viewer (same as images)

**Key Considerations**:
- Video thumbnails: Use first frame or generate poster image
- Grid layout: Videos should maintain aspect ratio like images
- Hover effects: May need different behavior for videos
- Face detection: Only applies to images (videos may have faces but detection is image-based)

### 3. PhotoViewerClient Component Updates

**File**: `components/PhotoViewerClient.tsx`

**Changes Needed**:
- Conditional rendering: `<Image>` for images, `<video>` for videos
- Video player controls: Native HTML5 video controls or custom controls
- Video features:
  - Play/pause
  - Seek bar
  - Volume control
  - Fullscreen
  - Playback speed
- Zoom/pan: May not apply to videos (or only to video frame, not playback)
- Slideshow: Videos should auto-play when in slideshow mode
- Navigation: Previous/next should work with mixed media types

**Key Considerations**:
- Video autoplay: Browser policies may prevent autoplay
- Video loading: Larger files, need loading states
- Video formats: Support common formats (MP4, WebM, MOV)
- Video metadata: Duration, resolution, codec info
- Face detection: Disable for videos (or handle differently)

### 4. API Route Updates

**File**: `app/api/photos/[id]/image/route.ts`

**Option A: Rename to `/api/photos/[id]/media/route.ts`**
- Handle both images and videos
- Return appropriate content type based on `media_type`
- For videos: Stream video file directly
- For images: Keep existing logic

**Option B: Create separate route `/api/photos/[id]/video/route.ts`**
- Dedicated video serving endpoint
- Stream video files
- Support range requests for video seeking
- Handle video formats and codecs

**Recommended**: Option A (unified media route) for consistency

**Key Considerations**:
- Video streaming: Support HTTP range requests (206 Partial Content)
- Video formats: Detect and serve appropriate MIME types
- Watermarks: May need video watermarking (more complex than images)
- Caching: Different cache strategies for videos vs images
- File size: Videos are larger, need efficient streaming

### 5. Video Thumbnail Generation

**Options**:
1. **Generate on upload**: Extract first frame during upload process
2. **Generate on demand**: Extract first frame when needed (slower)
3. **Use video poster**: HTML5 video `poster` attribute
4. **Store thumbnail path**: Add `thumbnail_path` field to Photo model

**Recommended**: Use HTML5 video `poster` attribute initially, generate thumbnails on demand if needed

### 6. TypeScript Type Updates

**Files**: All files using Photo type

**Changes Needed**:
- Update Photo interface to include `mediaType: string`
- Create type guards: `isVideo(photo: Photo): boolean`
- Update PhotoWithDetails, PhotoWithPeople interfaces

### 7. Query Updates

**Files**: `lib/queries.ts`, `app/api/search/route.ts`, etc.

**Changes Needed**:
- Include `mediaType` in Prisma queries
- Filter by media type if needed (future feature)
- Ensure `mediaType` is serialized and sent to frontend

## Technical Considerations

### Video Formats Support
- **MP4 (H.264)**: Widely supported, recommended
- **WebM**: Good browser support, smaller files
- **MOV**: Apple format, may need conversion
- **Codec**: H.264 for compatibility

### Performance Considerations
- **Lazy loading**: Videos should load on demand, not all at once
- **Thumbnails**: Use lightweight thumbnails in grid
- **Streaming**: Implement proper video streaming with range requests
- **Caching**: Cache video thumbnails, not full videos
- **CDN**: Consider CDN for video delivery if many videos

### Browser Compatibility
- **HTML5 Video**: Modern browsers support `<video>` element
- **Autoplay policies**: Browsers restrict autoplay, need user interaction
- **Mobile**: Ensure video controls work on touch devices
- **Bandwidth**: Consider adaptive bitrate streaming for large videos

### User Experience
- **Loading states**: Show loading indicators for videos
- **Error handling**: Handle video load errors gracefully
- **Playback controls**: Intuitive controls for play/pause/seek
- **Mixed media**: Smooth transitions between images and videos in slideshow
- **Keyboard shortcuts**: Space for play/pause, arrows for navigation

### Security Considerations
- **File validation**: Ensure video files are safe
- **Path validation**: Prevent path traversal attacks
- **Content type**: Verify actual file type matches `media_type`
- **Access control**: Same permissions as images

## Implementation Approach

### Phase 1: Basic Video Display
1. Update Prisma schema with `mediaType` field
2. Update PhotoGrid to detect and render videos
3. Update PhotoViewerClient to show video player
4. Create/update API route to serve videos
5. Test with sample videos

### Phase 2: Enhanced Features
1. Video thumbnails/poster images
2. Custom video controls
3. Video metadata display
4. Improved loading states
5. Error handling

### Phase 3: Optimization
1. Video streaming optimization
2. Thumbnail generation
3. Caching strategies
4. Performance monitoring

## Files That Need Changes

### Core Components
- `components/PhotoGrid.tsx` - Grid display with video support
- `components/PhotoViewerClient.tsx` - Video player in viewer
- `prisma/schema.prisma` - Add mediaType field

### API Routes
- `app/api/photos/[id]/image/route.ts` - Rename/update to handle videos
- OR `app/api/photos/[id]/video/route.ts` - New video route

### Utilities
- `lib/queries.ts` - Include mediaType in queries
- `lib/serialize.ts` - Serialize mediaType field

### Types
- All files importing Photo type from Prisma

## Testing Considerations

1. **Mixed media**: Test grid with both images and videos
2. **Video formats**: Test MP4, WebM, MOV
3. **Video sizes**: Test small and large video files
4. **Network**: Test with slow connections
5. **Mobile**: Test on mobile devices
6. **Browser**: Test on Chrome, Firefox, Safari, Edge
7. **Slideshow**: Test slideshow with mixed media
8. **Navigation**: Test prev/next with videos
9. **Watermarks**: Test watermark behavior (if applicable to videos)

## Open Questions

1. **Video watermarks**: Should videos have watermarks? (More complex than images)
2. **Face detection**: Should videos support face detection? (Requires frame extraction)
3. **Video editing**: Any video editing features needed?
4. **Video metadata**: Display duration, resolution, codec?
5. **Video upload**: How are videos uploaded? (Same as images?)
6. **Video processing**: Any server-side video processing needed?
7. **Thumbnail storage**: Where to store video thumbnails?
8. **Video transcoding**: Need to transcode videos to different formats?

## Conclusion

Videos can be viewed by:
1. Adding `mediaType` field to Prisma schema
2. Conditionally rendering `<video>` elements instead of `<Image>` for videos
3. Creating/updating API route to serve video files with proper streaming
4. Updating grid and viewer components to handle both media types
5. Implementing video-specific features (controls, playback, etc.)

The architecture is well-positioned to support videos with minimal changes, as the existing photo viewing infrastructure can be extended to handle videos.







