Guided path

Build a Small Local AI Workbench

A short path for trying local models, comparing hosted options, and adding one repeatable eval.

Outcome

Leave with one local model test, one hosted comparison, and a tiny eval you can rerun.

This rabbit hole keeps the AI experiment small: one local baseline, one hosted comparison, and one repeatable check.

Best for

  • Developers or AI-curious makers who want a small model workflow without buying infrastructure first.
  • Teams comparing local and hosted models before changing a real product workflow.

Before you start

  • A machine that can run at least one small local model or a willingness to use a hosted fallback.
  • One concrete prompt or task that you can judge without building a full benchmark suite.

What you leave with

  • One local baseline result and one hosted comparison for the same prompt.
  • A tiny repeatable eval that captures expected behavior before switching models.

Common pitfalls

  • Treating a pleasing demo response as evidence that the model is reliable.
  • Comparing models with different prompts, context, or success criteria.

Follow-up moves

  • Save the prompt, model names, provider, date, and observed failure cases in one note.
  • Repeat the same eval after changing model size, provider, or system instructions.

Route

  1. 01

    Start with a small local model

    Use Ollama Library to choose a model that your machine can run comfortably.

    Ollama Library
  2. 02

    Compare against hosted models

    Use OpenRouter Models to compare context, price signals, and provider availability.

    OpenRouter Models
  3. 03

    Turn one prompt into a test

    Use promptfoo to capture three inputs and one behavior expectation before changing models.

    promptfoo

Tools and sources in this path

Ollama LibraryOpen Ollama Library and start here: Pick a small model first and confirm it runs comfortably on your machine.OpenRouter ModelsOpen OpenRouter Models and start here: Filter by capability and inspect a few model cards side by side.promptfooOpen promptfoo and start here: Write three representative inputs and one clear expected behavior.

Next useful action

Open the first source, keep one note from each step, and stop when you have one repeatable result.