Most people are using something in the llama family for inference. Llama server ...

		sosodev 43 days ago \| parent \| context \| favorite \| on: Tinybox – A powerful computer for deep learning Most people are using something in the llama family for inference. Llama server is my go to. Unsloth guides describe how to configure inference for your model of choice.