From 2f8205150fcaff33711abdef75ee605863eb3f3a Mon Sep 17 00:00:00 2001 From: Tanya Date: Tue, 17 Feb 2026 14:24:53 -0500 Subject: [PATCH] Add comprehensive setup guide for pulling and running repository --- SETUP.md | 239 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 239 insertions(+) create mode 100644 SETUP.md diff --git a/SETUP.md b/SETUP.md new file mode 100644 index 0000000..81f481d --- /dev/null +++ b/SETUP.md @@ -0,0 +1,239 @@ +# Nanobot Setup Guide + +This guide will help you set up nanobot on a fresh system, pulling from the repository and configuring it to use Ollama and AirLLM with Llama models. + +## Prerequisites + +- Python 3.10 or higher +- Git +- (Optional) CUDA-capable GPU for AirLLM (recommended for better performance) + +## Step 1: Clone the Repository + +```bash +git clone +cd nanobot +``` + +If you're using a specific branch (e.g., the cleanup branch): +```bash +git checkout feature/cleanup-providers-llama-only +``` + +## Step 2: Create Virtual Environment + +```bash +python3 -m venv venv +source venv/bin/activate # On Windows: venv\Scripts\activate +``` + +## Step 3: Install Dependencies + +```bash +pip install --upgrade pip +pip install -e . +``` + +If you plan to use AirLLM, also install: +```bash +pip install airllm bitsandbytes +``` + +## Step 4: Choose Your Provider Setup + +You have two main options: + +### Option A: Use Ollama (Easiest, No Tokens Needed) + +1. **Install Ollama** (if not already installed): + ```bash + # Linux/Mac + curl -fsSL https://ollama.ai/install.sh | sh + + # Or download from: https://ollama.ai + ``` + +2. **Pull a Llama model**: + ```bash + ollama pull llama3.2:latest + ``` + +3. **Configure nanobot**: + ```bash + mkdir -p ~/.nanobot + cat > ~/.nanobot/config.json << 'EOF' + { + "providers": { + "ollama": { + "apiKey": "dummy", + "apiBase": "http://localhost:11434/v1" + } + }, + "agents": { + "defaults": { + "model": "llama3.2:latest" + } + } + } + EOF + chmod 600 ~/.nanobot/config.json + ``` + +### Option B: Use AirLLM (Direct Local Inference, No HTTP Server) + +1. **Get Hugging Face Token** (one-time, for downloading gated models): + - Go to: https://huggingface.co/settings/tokens + - Create a new token with "Read" permission + - Copy the token (starts with `hf_`) + +2. **Accept Llama License**: + - Go to: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct + - Click "Agree and access repository" + - Accept the license terms + +3. **Download Llama Model** (one-time): + ```bash + # Install huggingface_hub if needed + pip install huggingface_hub + + # Download model to local directory + huggingface-cli download meta-llama/Llama-3.2-3B-Instruct \ + --local-dir ~/.local/models/llama3.2-3b-instruct \ + --token YOUR_HF_TOKEN_HERE + ``` + +4. **Configure nanobot**: + ```bash + mkdir -p ~/.nanobot + cat > ~/.nanobot/config.json << 'EOF' + { + "providers": { + "airllm": { + "apiKey": "/home/YOUR_USERNAME/.local/models/llama3.2-3b-instruct", + "apiBase": null, + "extraHeaders": {} + } + }, + "agents": { + "defaults": { + "model": "/home/YOUR_USERNAME/.local/models/llama3.2-3b-instruct" + } + } + } + EOF + chmod 600 ~/.nanobot/config.json + ``` + + **Important**: Replace `YOUR_USERNAME` with your actual username, or use `~/.local/models/llama3.2-3b-instruct` (the `~` will be expanded). + +## Step 5: Test the Setup + +```bash +nanobot agent -m "Hello, what is 2+5?" +``` + +You should see a response from the model. If you get errors, see the Troubleshooting section below. + +## Step 6: (Optional) Use Setup Script + +Instead of manual configuration, you can use the provided setup script: + +```bash +python3 setup_llama_airllm.py +``` + +This script will: +- Guide you through model selection +- Help you configure the Hugging Face token +- Set up the config file automatically + +## Configuration File Location + +- **Path**: `~/.nanobot/config.json` +- **Permissions**: Should be `600` (read/write for owner only) +- **Backup**: Always backup before editing! + +## Available Providers + +After setup, nanobot supports: + +- **Ollama**: Local OpenAI-compatible server (no tokens needed) +- **AirLLM**: Direct local model inference (no HTTP server, no tokens after download) +- **vLLM**: Local OpenAI-compatible server (for advanced users) +- **DeepSeek**: API or local models (for future use) + +## Recommended Models + +### For Ollama: +- `llama3.2:latest` - Fast, minimal memory (recommended) +- `llama3.1:8b` - Good balance +- `llama3.1:70b` - Best quality (needs more GPU) + +### For AirLLM: +- `meta-llama/Llama-3.2-3B-Instruct` - Fast, minimal memory (recommended) +- `meta-llama/Llama-3.1-8B-Instruct` - Good balance +- Local path: `~/.local/models/llama3.2-3b-instruct` (after download) + +## Troubleshooting + +### "Model not found" error (AirLLM) +- Make sure you've accepted the Llama license on Hugging Face +- Verify your HF token has read permissions +- Check that the model path in config is correct +- Ensure the model files are downloaded (check `~/.local/models/llama3.2-3b-instruct/`) + +### "Connection refused" error (Ollama) +- Make sure Ollama is running: `ollama serve` +- Check that Ollama is listening on port 11434: `curl http://localhost:11434/api/tags` +- Verify the model is pulled: `ollama list` + +### "Out of memory" error (AirLLM) +- Try a smaller model (Llama-3.2-3B-Instruct instead of 8B) +- Use compression: set `apiBase` to `"4bit"` or `"8bit"` in the airllm config +- Close other GPU-intensive applications + +### "No API key configured" error +- For Ollama: Use `"dummy"` as apiKey (it's not actually used) +- For AirLLM: No API key needed for local paths, but you need the model files downloaded + +### Import errors +- Make sure virtual environment is activated +- Reinstall dependencies: `pip install -e .` +- For AirLLM: `pip install airllm bitsandbytes` + +## Using Local Model Paths (No Tokens After Download) + +Once you've downloaded a model locally with AirLLM, you can use it forever without any tokens: + +```json +{ + "providers": { + "airllm": { + "apiKey": "/path/to/your/local/model" + } + }, + "agents": { + "defaults": { + "model": "/path/to/your/local/model" + } + } +} +``` + +The model path should point to a directory containing: +- `config.json` +- `tokenizer.json` (or `tokenizer_config.json`) +- Model weights (`model.safetensors` or `pytorch_model.bin`) + +## Next Steps + +- Read the main README.md for usage examples +- Check `nanobot --help` for available commands +- Explore the workspace features: `nanobot workspace create myproject` + +## Getting Help + +- Check the repository issues +- Review the code comments +- Test with a simple query first: `nanobot agent -m "Hello"` +