Add comprehensive setup guide for pulling and running repository
This commit is contained in:
parent
216c9f5039
commit
2f8205150f
239
SETUP.md
Normal file
239
SETUP.md
Normal file
@ -0,0 +1,239 @@
|
||||
# Nanobot Setup Guide
|
||||
|
||||
This guide will help you set up nanobot on a fresh system, pulling from the repository and configuring it to use Ollama and AirLLM with Llama models.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.10 or higher
|
||||
- Git
|
||||
- (Optional) CUDA-capable GPU for AirLLM (recommended for better performance)
|
||||
|
||||
## Step 1: Clone the Repository
|
||||
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
cd nanobot
|
||||
```
|
||||
|
||||
If you're using a specific branch (e.g., the cleanup branch):
|
||||
```bash
|
||||
git checkout feature/cleanup-providers-llama-only
|
||||
```
|
||||
|
||||
## Step 2: Create Virtual Environment
|
||||
|
||||
```bash
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||
```
|
||||
|
||||
## Step 3: Install Dependencies
|
||||
|
||||
```bash
|
||||
pip install --upgrade pip
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
If you plan to use AirLLM, also install:
|
||||
```bash
|
||||
pip install airllm bitsandbytes
|
||||
```
|
||||
|
||||
## Step 4: Choose Your Provider Setup
|
||||
|
||||
You have two main options:
|
||||
|
||||
### Option A: Use Ollama (Easiest, No Tokens Needed)
|
||||
|
||||
1. **Install Ollama** (if not already installed):
|
||||
```bash
|
||||
# Linux/Mac
|
||||
curl -fsSL https://ollama.ai/install.sh | sh
|
||||
|
||||
# Or download from: https://ollama.ai
|
||||
```
|
||||
|
||||
2. **Pull a Llama model**:
|
||||
```bash
|
||||
ollama pull llama3.2:latest
|
||||
```
|
||||
|
||||
3. **Configure nanobot**:
|
||||
```bash
|
||||
mkdir -p ~/.nanobot
|
||||
cat > ~/.nanobot/config.json << 'EOF'
|
||||
{
|
||||
"providers": {
|
||||
"ollama": {
|
||||
"apiKey": "dummy",
|
||||
"apiBase": "http://localhost:11434/v1"
|
||||
}
|
||||
},
|
||||
"agents": {
|
||||
"defaults": {
|
||||
"model": "llama3.2:latest"
|
||||
}
|
||||
}
|
||||
}
|
||||
EOF
|
||||
chmod 600 ~/.nanobot/config.json
|
||||
```
|
||||
|
||||
### Option B: Use AirLLM (Direct Local Inference, No HTTP Server)
|
||||
|
||||
1. **Get Hugging Face Token** (one-time, for downloading gated models):
|
||||
- Go to: https://huggingface.co/settings/tokens
|
||||
- Create a new token with "Read" permission
|
||||
- Copy the token (starts with `hf_`)
|
||||
|
||||
2. **Accept Llama License**:
|
||||
- Go to: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
|
||||
- Click "Agree and access repository"
|
||||
- Accept the license terms
|
||||
|
||||
3. **Download Llama Model** (one-time):
|
||||
```bash
|
||||
# Install huggingface_hub if needed
|
||||
pip install huggingface_hub
|
||||
|
||||
# Download model to local directory
|
||||
huggingface-cli download meta-llama/Llama-3.2-3B-Instruct \
|
||||
--local-dir ~/.local/models/llama3.2-3b-instruct \
|
||||
--token YOUR_HF_TOKEN_HERE
|
||||
```
|
||||
|
||||
4. **Configure nanobot**:
|
||||
```bash
|
||||
mkdir -p ~/.nanobot
|
||||
cat > ~/.nanobot/config.json << 'EOF'
|
||||
{
|
||||
"providers": {
|
||||
"airllm": {
|
||||
"apiKey": "/home/YOUR_USERNAME/.local/models/llama3.2-3b-instruct",
|
||||
"apiBase": null,
|
||||
"extraHeaders": {}
|
||||
}
|
||||
},
|
||||
"agents": {
|
||||
"defaults": {
|
||||
"model": "/home/YOUR_USERNAME/.local/models/llama3.2-3b-instruct"
|
||||
}
|
||||
}
|
||||
}
|
||||
EOF
|
||||
chmod 600 ~/.nanobot/config.json
|
||||
```
|
||||
|
||||
**Important**: Replace `YOUR_USERNAME` with your actual username, or use `~/.local/models/llama3.2-3b-instruct` (the `~` will be expanded).
|
||||
|
||||
## Step 5: Test the Setup
|
||||
|
||||
```bash
|
||||
nanobot agent -m "Hello, what is 2+5?"
|
||||
```
|
||||
|
||||
You should see a response from the model. If you get errors, see the Troubleshooting section below.
|
||||
|
||||
## Step 6: (Optional) Use Setup Script
|
||||
|
||||
Instead of manual configuration, you can use the provided setup script:
|
||||
|
||||
```bash
|
||||
python3 setup_llama_airllm.py
|
||||
```
|
||||
|
||||
This script will:
|
||||
- Guide you through model selection
|
||||
- Help you configure the Hugging Face token
|
||||
- Set up the config file automatically
|
||||
|
||||
## Configuration File Location
|
||||
|
||||
- **Path**: `~/.nanobot/config.json`
|
||||
- **Permissions**: Should be `600` (read/write for owner only)
|
||||
- **Backup**: Always backup before editing!
|
||||
|
||||
## Available Providers
|
||||
|
||||
After setup, nanobot supports:
|
||||
|
||||
- **Ollama**: Local OpenAI-compatible server (no tokens needed)
|
||||
- **AirLLM**: Direct local model inference (no HTTP server, no tokens after download)
|
||||
- **vLLM**: Local OpenAI-compatible server (for advanced users)
|
||||
- **DeepSeek**: API or local models (for future use)
|
||||
|
||||
## Recommended Models
|
||||
|
||||
### For Ollama:
|
||||
- `llama3.2:latest` - Fast, minimal memory (recommended)
|
||||
- `llama3.1:8b` - Good balance
|
||||
- `llama3.1:70b` - Best quality (needs more GPU)
|
||||
|
||||
### For AirLLM:
|
||||
- `meta-llama/Llama-3.2-3B-Instruct` - Fast, minimal memory (recommended)
|
||||
- `meta-llama/Llama-3.1-8B-Instruct` - Good balance
|
||||
- Local path: `~/.local/models/llama3.2-3b-instruct` (after download)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Model not found" error (AirLLM)
|
||||
- Make sure you've accepted the Llama license on Hugging Face
|
||||
- Verify your HF token has read permissions
|
||||
- Check that the model path in config is correct
|
||||
- Ensure the model files are downloaded (check `~/.local/models/llama3.2-3b-instruct/`)
|
||||
|
||||
### "Connection refused" error (Ollama)
|
||||
- Make sure Ollama is running: `ollama serve`
|
||||
- Check that Ollama is listening on port 11434: `curl http://localhost:11434/api/tags`
|
||||
- Verify the model is pulled: `ollama list`
|
||||
|
||||
### "Out of memory" error (AirLLM)
|
||||
- Try a smaller model (Llama-3.2-3B-Instruct instead of 8B)
|
||||
- Use compression: set `apiBase` to `"4bit"` or `"8bit"` in the airllm config
|
||||
- Close other GPU-intensive applications
|
||||
|
||||
### "No API key configured" error
|
||||
- For Ollama: Use `"dummy"` as apiKey (it's not actually used)
|
||||
- For AirLLM: No API key needed for local paths, but you need the model files downloaded
|
||||
|
||||
### Import errors
|
||||
- Make sure virtual environment is activated
|
||||
- Reinstall dependencies: `pip install -e .`
|
||||
- For AirLLM: `pip install airllm bitsandbytes`
|
||||
|
||||
## Using Local Model Paths (No Tokens After Download)
|
||||
|
||||
Once you've downloaded a model locally with AirLLM, you can use it forever without any tokens:
|
||||
|
||||
```json
|
||||
{
|
||||
"providers": {
|
||||
"airllm": {
|
||||
"apiKey": "/path/to/your/local/model"
|
||||
}
|
||||
},
|
||||
"agents": {
|
||||
"defaults": {
|
||||
"model": "/path/to/your/local/model"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The model path should point to a directory containing:
|
||||
- `config.json`
|
||||
- `tokenizer.json` (or `tokenizer_config.json`)
|
||||
- Model weights (`model.safetensors` or `pytorch_model.bin`)
|
||||
|
||||
## Next Steps
|
||||
|
||||
- Read the main README.md for usage examples
|
||||
- Check `nanobot --help` for available commands
|
||||
- Explore the workspace features: `nanobot workspace create myproject`
|
||||
|
||||
## Getting Help
|
||||
|
||||
- Check the repository issues
|
||||
- Review the code comments
|
||||
- Test with a simple query first: `nanobot agent -m "Hello"`
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user