awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
Andyyyy64 avatar

Andyyyy64/whichllm

0
View on GitHub↗

Whichllm

whichllm is a command-line tool that detects a machine's GPU, memory, and memory bandwidth, then recommends open large language models that will actually run on it. Candidates are ordered by benchmark quality and estimated generation speed rather than parameter count, so the list reflects what the hardware can handle.

Ranking relies on benchmark data merged onto one scale, with older model generations demoted and weaker matches, such as same-family interpolation or uploader claims, discounted. Models that would generate below a minimum speed have their scores scaled down, so a slightly weaker but responsive model can outrank a stronger, sluggish one. Scoring can also be tailored to a task such as coding, math, or vision. Fit is decided by adding model weights, a per-layer cache read from the model configuration, and runtime buffers. Speed is estimated from memory bandwidth, with a confidence flag on each estimate. For large mixture-of-experts models, the tool can keep attention and shared weights on the GPU and place expert layers in system RAM.

Beyond recommending, the tool can download and open a chat with the top-ranked or a named model, using recommended launch flags when a compatible runtime is installed. It detects models already in a local library and opens them instead of downloading again. A hardware simulation mode evaluates which models would run on a chosen GPU or memory setup without owning it, and can also override misdetected hardware limits.

By default it reads a prebuilt, daily-rebuilt, quality-checked index of models, configs, and benchmark scores for a fast start, with an option to fetch live data instead. Rankings are checked with regression tests against frozen model data across many hardware profiles, and speed estimates are compared with published real-world runs.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI

Features

  • GPU Engine Recommenders - Detects a machine's GPU, memory, and bandwidth, then ranks open language models that will run on it by benchmark quality and estimated speed.
  • Generation Speed Estimators - Estimates tokens per second from memory bandwidth by splitting weight reads between GPU memory and system RAM, flagging confidence in each estimate.
  • Recency-Weighted Benchmark Merging - Merges benchmark sources onto one scale, demotes older model generations, and discounts weaker matches such as same-family interpolation or uploader claims.
  • Memory Fit Checks - Adds model weights, a per-layer cache read from the model configuration, and runtime buffers to decide whether a model fits in available memory.
  • LLM Benchmarking - Ranks local language models using recency-weighted benchmark data instead of parameter count, with penalties for models too slow to use.
  • Local Model Launchers - Download the top-ranked or a named model and open a chat, using recommended launch flags when a compatible runtime is installed.
  • Local LLM Installers - Fetches a top-ranked or chosen open model, reuses installed ones, and opens a chat with recommended runtime flags from the command line.
  • Expert Offloading to System Memory - Keeps attention and shared weights on the GPU and places expert layers in system RAM so large sparse models run on small cards.
  • Task-Specific Model Ranking - Scores candidate models on benchmarks specific to a chosen task such as coding, math, or vision instead of one general quality score.
  • Hardware Simulations - Evaluates which models would run on a chosen GPU or memory setup without owning it, and can override misdetected hardware limits.
  • Speed-Adjusted Score Penalties - Scales down scores for models that would generate below a minimum speed, so a slightly weaker but responsive model can outrank a stronger, sluggish one.
6,828 stars·378 forks·Python·MIT·3 views

Star history

Star history chart for andyyyy64/whichllmStar history chart for andyyyy64/whichllm

How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.

Frequently asked questions

What does andyyyy64/whichllm do?

whichllm is a command-line tool that detects a machine's GPU, memory, and memory bandwidth, then recommends open large language models that will actually run on it. Candidates are ordered by benchmark quality and estimated generation speed rather than parameter count, so the list reflects what the hardware can handle.

What are the main features of andyyyy64/whichllm?

The main features of andyyyy64/whichllm are: GPU Engine Recommenders, Generation Speed Estimators, Recency-Weighted Benchmark Merging, Memory Fit Checks, LLM Benchmarking, Local Model Launchers, Local LLM Installers, Expert Offloading to System Memory.

Which projects share features with andyyyy64/whichllm?

Projects with overlapping indexed features include: snowkylin/tensorflow-handbook — This project is a comprehensive educational resource and tutorial handbook for building, training, and deploying… open-jarvis/openjarvis — OpenJarvis is a local-first personal AI assistant and agent framework. It runs language models on a user's own laptop,… internlm/opencompass — OpenCompass is a comprehensive evaluation platform, benchmarking suite, and distributed model evaluator designed to… fchollet/arc-agi — This project is a standardized set of abstraction and reasoning problems designed for benchmarking the ability of… open-compass/opencompass — OpenCompass is an open-source framework for standardized benchmarking of large language models. It provides a… zjunlp/easyedit — EasyEdit is a framework and toolkit designed for updating, inserting, or erasing specific factual information within…

Projects sharing features with Whichllm

These projects share indexed features with Whichllm. Shared tags can include platform or build tooling; verify the primary use case before treating a result as a replacement.
  • snowkylin/tensorflow-handbooksnowkylin avatar

    snowkylin/tensorflow-handbook

    3,921View on GitHub↗

    This project is a comprehensive educational resource and tutorial handbook for building, training, and deploying machine learning models using TensorFlow 2. It serves as a structured learning guide covering core deep learning concepts, including neural network architectures, automatic differentiation, and tensor operations. The handbook provides technical guidance on optimizing execution efficiency through GPU memory management, distributed training, and model quantization. It also includes detailed manuals for constructing high-performance data pipelines and exporting models for production s

    Jupyter Notebook
    View on GitHub↗3,921
  • open-jarvis/openjarvisopen-jarvis avatar

    open-jarvis/OpenJarvis

    13,786View on GitHub↗

    OpenJarvis is a local-first personal AI assistant and agent framework. It runs language models on a user's own laptop, workstation, or server, keeps agent memory and state on the device, and treats cloud services as optional extras. Agents reason over multiple steps, call tools, search documents, and can run on demand, on a cron schedule, or continuously in the background. Core functionality stays available offline. A unified inference layer connects local runtimes and optional cloud providers behind one interface. It detects the installed GPU vendor, model, and memory, then recommends a suit

    Python
    View on GitHub↗13,786
  • fchollet/arc-agifchollet avatar

    fchollet/ARC-AGI

    4,787View on GitHub↗

    This project is a standardized set of abstraction and reasoning problems designed for benchmarking the ability of artificial intelligence models to learn new rules. It functions as a fluid intelligence test and a reasoning benchmark, utilizing a collection of grid-based puzzles and a program synthesis dataset to evaluate how agents generate algorithms from examples. The project focuses on measuring general fluid intelligence and the capacity for zero-shot generalization, testing whether a system can apply learned logic to unseen problems without relying on task-specific training. It provides

    JavaScriptartificial-intelligenceintelligence-testingprogram-synthesis
    View on GitHub↗4,787
  • internlm/opencompassInternLM avatar

    InternLM/opencompass

    7,096View on GitHub↗

    OpenCompass is a comprehensive evaluation platform, benchmarking suite, and distributed model evaluator designed to measure the performance and accuracy of large language models. It provides a framework for benchmarking both open-source and API-based models against diverse datasets using standardized metrics and reproducible pipelines. The project features an automated judging framework that uses language models as judges to score and verify the quality of generated text. It includes a performance leaderboard system for comparing the relative capabilities of various models across industry-sta

    Python
    View on GitHub↗7,096
Compare all 21 related projects→