Thanks for posting. I have the same quite positive experience for LLM’s up to 3b parameters, running reasonably on my RPi 5 8GB . I so hope that the Hailo 10H generative AI accelerator will be made available for the RPi 5 via NVMe soon. https://hailo.ai/products/ai-accelerato ... on-module/
Statistics: Posted by WouterO — Sun Feb 16, 2025 11:23 am