AI News · #attention-mechanisms

GeekNews · 2026-06-24 제목번역

Unlimited OCR — Baidu의 원샷 장문 파싱 모델

Unlimited OCR — Baidu의 원샷 장문 파싱 모델 | GeekNews

<ul> <li>DeepSeek OCR를 기반으로 디코더의 모든 어텐션을 교체해, 수십 페이지 문서를 한 번의 순전파(forward pass) 로 전사하는 E2E OCR 모델</li> <li>핵심은 참조 슬라이딩 윈도우 어텐션(R-SWA)</str…

#deepseek #document-processing #long-context #attention-mechanisms #optical-character-recognition

Sebastian Raschka · 2026-05-16 제목번역

Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attention

From Gemma 4 to DeepSeek V4, How New Open-Weight LLMs Are Reducing Long-Context Costs

#long-context #open-weight-models #llm-architectures #attention-mechanisms #kv-sharing #compressed-attention

Sebastian Raschka · 2026-03-22 제목번역

A Visual Guide to Attention Variants in Modern LLMs

From MHA and GQA to MLA, sparse attention, and hybrid architectures

#large-language-models #machine-learning #transformer-architecture #sparse-attention #attention-mechanisms #group-query-attention