Local LLM / Local LLM model selection
Local LLM: Reranker selection
Choose a reranker using relevance labels and the target hardware latency budget.
Environment and inputs: Local model selection across Ollama, llama.cpp and Transformers; capabilities depend on model revision and runtime.
When to use it
A model that looks strong on general conversation may fail a specific extraction, language or tool-use requirement. Compare approved artifacts using the same task fixtures and include memory, latency and licensing information in the selection record.
Check the result
Compare ranking gains on the candidate set actually produced by your retriever.
Save and compare results
Save the test input, configuration and result. Fix the failing boundary, repeat the same test and compare it with the saved working configuration.
Scope and limits
These exercises have not been tested on your systems. Check feature support and configuration names against the installed release and linked documentation.