Hardware

Large Language Models Speed Test

Large Language Models Speed Test

Let's test the LLMs' speed on GPU vs CPU

Comparing prediction speed of several versions of LLMs: llama3 (Meta/Facebook), phi3 (Microsoft), gemma (Google), mistral(open source) on CPU and GPU.