ilia
7c633a02ed
- Added .cursorrules for project guidelines and context - Created README.md for project overview and goals - Established ARCHITECTURE.md for architectural documentation - Set up tickets directory with initial ticket management files - Included .gitignore to manage ignored files and directories This commit lays the foundation for the Atlas project, ensuring a clear structure for development and collaboration.
50 lines
1.3 KiB
Markdown
50 lines
1.3 KiB
Markdown
# Ticket: Survey Candidate Open-Weight Models
|
|
|
|
## Ticket Information
|
|
|
|
- **ID**: TICKET-017
|
|
- **Title**: Survey Candidate Open-Weight Models
|
|
- **Type**: Research
|
|
- **Priority**: High
|
|
- **Status**: Backlog
|
|
- **Track**: LLM Infra
|
|
- **Milestone**: Milestone 1 - Survey & Architecture
|
|
- **Created**: 2024-01-XX
|
|
|
|
## Description
|
|
|
|
Survey and evaluate open-weight LLM models:
|
|
- 8-14B and 30B quantized options for RTX 4080 (Q4-Q6 variants)
|
|
- Small models for RTX 1050 (family agent)
|
|
- Evaluate coding/research capabilities for work agent
|
|
- Evaluate instruction-following for family agent
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [ ] Model comparison matrix created
|
|
- [ ] 4080 model candidates identified (8-14B, 30B quantized)
|
|
- [ ] 1050 model candidates identified (small, efficient)
|
|
- [ ] Evaluation criteria documented
|
|
- [ ] Recommendations documented
|
|
|
|
## Technical Details
|
|
|
|
Models to evaluate:
|
|
- 4080: Llama 3 8B/70B, Mistral 7B, Qwen, etc.
|
|
- 1050: TinyLlama, Phi-2, smaller quantized models
|
|
- Quantization: Q4, Q5, Q6, Q8
|
|
- Function calling support required
|
|
|
|
## Dependencies
|
|
|
|
- TICKET-004 (architecture) - helpful context
|
|
|
|
## Related Files
|
|
|
|
- `docs/LLM_MODEL_SURVEY.md` (to be created)
|
|
- `ARCHITECTURE.md`
|
|
|
|
## Notes
|
|
|
|
Can start in parallel with wake-word and clients. Depends on high-level architecture doc.
|