AI News · #efficient-inference

GeekNews · 2026-07-10 제목번역

Show HN: 느린 컴퓨터에서 GLM 5.2 실행하기

Show HN: 느린 컴퓨터에서 GLM 5.2 실행하기 | GeekNews

<ul> <li>colibrì는 <strong>GLM-5.2 744B MoE</strong>를 소비자용 머신의 약 25GB RAM에서 실행하도록 만든 순수 C 엔진이며, 라우팅된 expert를 디스크에서 스트리밍해 GPU 없이 동작함</li> <li>핵심 구조는 dense 부분 약 17B 파라미터를 <strong>int4…

#efficient-inference #edge-computing #glm-5 #consumer-hardware #moe-model #pure-c

Hacker News (front) · 2026-07-09 제목번역

Colibri: 소비자 머신의 25GB RAM에서 GLM-5.2 실행 (순수 C, 의존성 없음)

GitHub - JustVugg/colibri: Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

<p>A few days ago I found myself trying out GLM 5.2 and was really positively impressed. The capabilities and security I was getting from this LLM are similar to those I've gotten …

#efficient-inference #edge-computing #glm-5 #moe-inference #consumer-hardware #c-implementation

Sebastian Raschka · 2025-06-17 제목번역

LLM의 KV 캐시 이해와 처음부터 구현하기

Understanding and Coding the KV Cache in LLMs from Scratch

KV caches are one of the most critical techniques for efficient inference in LLMs in production.

#large-language-models #kv-cache #llm-inference #efficient-inference #transformer-optimization #neural-network-caching