nanobot/SETUP.md

# Nanobot Setup Guide

This guide will help you set up nanobot on a fresh system, pulling from the repository and configuring it to use Ollama and AirLLM with Llama models.

## Prerequisites

- Python 3.10 or higher
- Git
- (Optional) CUDA-capable GPU for AirLLM (recommended for better performance)

## Step 1: Clone the Repository

```bash
git clone <repository-url>
cd nanobot
```

If you're using a specific branch (e.g., the cleanup branch):
```bash
git checkout feature/cleanup-providers-llama-only
```

## Step 2: Create Virtual Environment

```bash
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
```

## Step 3: Install Dependencies

```bash
pip install --upgrade pip
pip install -e .
```

If you plan to use AirLLM, also install:
```bash
pip install airllm bitsandbytes
```

## Step 4: Choose Your Provider Setup

You have two main options:

### Option A: Use Ollama (Easiest, No Tokens Needed)

1. **Install Ollama** (if not already installed):
   ```bash
   # Linux/Mac
   curl -fsSL https://ollama.ai/install.sh | sh
   
   # Or download from: https://ollama.ai
   ```

2. **Pull a Llama model**:
   ```bash
   ollama pull llama3.2:latest
   ```

3. **Configure nanobot**:
   ```bash
   mkdir -p ~/.nanobot
   cat > ~/.nanobot/config.json << 'EOF'
   {
     "providers": {
       "ollama": {
         "apiKey": "dummy",
         "apiBase": "http://localhost:11434/v1"
       }
     },
     "agents": {
       "defaults": {
         "model": "llama3.2:latest"
       }
     }
   }
   EOF
   chmod 600 ~/.nanobot/config.json
   ```

### Option B: Use AirLLM (Direct Local Inference, No HTTP Server)

1. **Get Hugging Face Token** (one-time, for downloading gated models):
   - Go to: https://huggingface.co/settings/tokens
   - Create a new token with "Read" permission
   - Copy the token (starts with `hf_`)

2. **Accept Llama License**:
   - Go to: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct
   - Click "Agree and access repository"
   - Accept the license terms

3. **Download Llama Model** (one-time):
   ```bash
   # Install huggingface_hub if needed
   pip install huggingface_hub
   
   # Download model to local directory
   huggingface-cli download meta-llama/Llama-3.2-3B-Instruct \
     --local-dir ~/.local/models/llama3.2-3b-instruct \
     --token YOUR_HF_TOKEN_HERE
   ```

4. **Configure nanobot**:
   ```bash
   mkdir -p ~/.nanobot
   cat > ~/.nanobot/config.json << 'EOF'
   {
     "providers": {
       "airllm": {
         "apiKey": "/home/YOUR_USERNAME/.local/models/llama3.2-3b-instruct",
         "apiBase": null,
         "extraHeaders": {}
       }
     },
     "agents": {
       "defaults": {
         "model": "/home/YOUR_USERNAME/.local/models/llama3.2-3b-instruct"
       }
     }
   }
   EOF
   chmod 600 ~/.nanobot/config.json
   ```
   
   **Important**: Replace `YOUR_USERNAME` with your actual username, or use `~/.local/models/llama3.2-3b-instruct` (the `~` will be expanded).

## Step 5: Test the Setup

```bash
nanobot agent -m "Hello, what is 2+5?"
```

You should see a response from the model. If you get errors, see the Troubleshooting section below.

## Step 6: (Optional) Use Setup Script

Instead of manual configuration, you can use the provided setup script:

```bash
python3 setup_llama_airllm.py
```

This script will:
- Guide you through model selection
- Help you configure the Hugging Face token
- Set up the config file automatically

## Configuration File Location

- **Path**: `~/.nanobot/config.json`
- **Permissions**: Should be `600` (read/write for owner only)
- **Backup**: Always backup before editing!

## Available Providers

After setup, nanobot supports:

- **Ollama**: Local OpenAI-compatible server (no tokens needed)
- **AirLLM**: Direct local model inference (no HTTP server, no tokens after download)
- **vLLM**: Local OpenAI-compatible server (for advanced users)
- **DeepSeek**: API or local models (for future use)

## Recommended Models

### For Ollama:
- `llama3.2:latest` - Fast, minimal memory (recommended)
- `llama3.1:8b` - Good balance
- `llama3.1:70b` - Best quality (needs more GPU)

### For AirLLM:
- `meta-llama/Llama-3.2-3B-Instruct` - Fast, minimal memory (recommended)
- `meta-llama/Llama-3.1-8B-Instruct` - Good balance
- Local path: `~/.local/models/llama3.2-3b-instruct` (after download)

## Troubleshooting

### "Model not found" error (AirLLM)
- Make sure you've accepted the Llama license on Hugging Face
- Verify your HF token has read permissions
- Check that the model path in config is correct
- Ensure the model files are downloaded (check `~/.local/models/llama3.2-3b-instruct/`)

### "Connection refused" error (Ollama)
- Make sure Ollama is running: `ollama serve`
- Check that Ollama is listening on port 11434: `curl http://localhost:11434/api/tags`
- Verify the model is pulled: `ollama list`

### "Out of memory" error (AirLLM)
- Try a smaller model (Llama-3.2-3B-Instruct instead of 8B)
- Use compression: set `apiBase` to `"4bit"` or `"8bit"` in the airllm config
- Close other GPU-intensive applications

### "No API key configured" error
- For Ollama: Use `"dummy"` as apiKey (it's not actually used)
- For AirLLM: No API key needed for local paths, but you need the model files downloaded

### Import errors
- Make sure virtual environment is activated
- Reinstall dependencies: `pip install -e .`
- For AirLLM: `pip install airllm bitsandbytes`

## Using Local Model Paths (No Tokens After Download)

Once you've downloaded a model locally with AirLLM, you can use it forever without any tokens:

```json
{
  "providers": {
    "airllm": {
      "apiKey": "/path/to/your/local/model"
    }
  },
  "agents": {
    "defaults": {
      "model": "/path/to/your/local/model"
    }
  }
}
```

The model path should point to a directory containing:
- `config.json`
- `tokenizer.json` (or `tokenizer_config.json`)
- Model weights (`model.safetensors` or `pytorch_model.bin`)

## Next Steps

- Read the main README.md for usage examples
- Check `nanobot --help` for available commands
- Explore the workspace features: `nanobot workspace create myproject`

## Getting Help

- Check the repository issues
- Review the code comments
- Test with a simple query first: `nanobot agent -m "Hello"`