• Comprehensive49
    link
    fedilink
    arrow-up
    1
    ·
    5 days ago

    Why should I use this instead of Ollama? Ollama is considered the local AI standard and is supported by a ton of other open-source software. For example, you can connect Ollama with the Smart Connections plugin for Obsidian, which lets you chat with and analyze your Obsidian notes.

      • Comprehensive49
        link
        fedilink
        arrow-up
        1
        ·
        edit-2
        4 days ago

        You can run multimodal models like LLaVA and LLaMA on Ollama as well.

        The AI models are coded and used in a way that makes them basically platform-agnostic, so the specific platform (Ollama, LocalAI, vLLM, llama.cpp, etc.) you run them with ends up being irrelevant.

        Because of that, the only reasons to use one platform over another are if it’s best for your specific use case (depends), it’s the best supported (Ollama by far), or if it has the best performance (vLLM seems to win right now).