In Jan app LLM model performance.

[shrisha108]

Hello,
I'm running llama.cpp on a linux headless server in my local network. First of all I love Jan app, very nice and neat, bunch of very useful settings, MCP servers, creating assistants for different use cases, etc, so using it as daily driver. And yesterday I updated llama.cpp to the latest and out of curiosity tried the build in llama.cpp chat. And to my surprise it is actually giving me almost twice t\s of what I usually getting in a Jan. Same question was asked to same LLM model in same warm-up situation. I use no assistant in Jan to match the llama.cpp native chat. So llama.cpp chat gives 52 t\s and Jan 28 t\s. Tried few times. Could it be because in Jan I'm using it as OpenAI compatible provider? Would be great to match the native chat :)
Anyway please let me know if I just have to adjust some settings I don't know about or it is expected from Jan app according to App architecture.
Thank you.

[qnixsynapse]

Goto settings check for K type and V type and switch them back from q8 to f16.

[shrisha108]

Goto settings check for K type and V type and switch them back from q8 to f16.

Thank you for your answer. I'm running llama.cpp on a headless server in my local network, not build-in one in Jan. Just added to Jan as OpenAI compatible provider. And yes KV cache is f16 in both cases.