| ▲ | prabhanjana_c 7 hours ago | |
Very interesting. I am always curious on llama.cpp and vllm. All the best for samosa inference engine. One reason I am not running local models on my Mac now a days, is My Mac book pro is getting heated more. Otherwise would love to run couple (at least 1 to 3) of local models continuously. | ||
| ▲ | dwa3592 5 minutes ago | parent [-] | |
>>One reason I am not running local models on my Mac now a days, is My Mac book pro is getting heated more Samosa was optimized for keeping mac's temp in check. This is one of the important reasons for creating this - no excessive wear/tear to the machine. https://github.com/deepanwadhwa/samosa-chat#the-three-princi... maybe give it a try and let me know if your macbook is getting heated while running this? | ||