Ideas & society2021 · shared on Hacker News
Google breaks the trillion-parameter ceiling with the Switch Transformer
Explore Google breaks the trillion-parameter ceiling with the Switch Transformer, its original source and Hacker News discussions. Discover community perspectives and save it to your reading list.
GO BEYOND THE ABSTRACT
The conversation around the idea.
Original Hacker News threads. Different perspectives, questions, and things worth thinking about.
Showing up to 100 related threads. A paper may appear in a comment rather than the original submission.
Links found in the discussion
Up to 50 recent source links, in HN item order.
Ye0m ·
GPT-4
499 points201 comments
Yselimonder ·
Megatron-Turing NLG 530B, the World’s Largest Generative Language Model
116 points99 comments
Ytaytus ·
GPT-4 is getting worse over time, not better
88 points55 comments
Yjayalammar ·
The Illustrated Retrieval Transformer
75 points13 comments
Ylnyan ·
Program Synthesis with Large Language Models
63 points20 comments
Ygroar ·
Google breaks the trillion-parameter ceiling with the Switch Transformer
48 points4 comments
Ysteveharing1 ·
Granite 4.1: IBM's 8B Model Matching 32B MoE
27 points5 comments
Ytompark ·
Decomposing Language Models into Understandable Components
13 points1 comments
Ygeorgehill ·
GPT4 is 8 x 220B params = 1.7T params
12 points0 comments
Yhardmaru ·
Switch Transformers: Scaling to Trillion Parameter Models with Sparsity
3 points0 comments
Yjbesomi ·
Switch Transformers: Scaling to Trillion Parameter Models with Simple Sparsity
2 points0 comments
Yturingbook ·
Switch Transformers: Trillion Parameter Models from Googleent Sparsity
1 points0 comments
Y317070 ·
Switch Transformers Scaling to Trillion Parameter Models
1 points0 comments
Ydanielmorozoff ·
Google Brain’s Switch Transformer Language Model Packs 1.6T Parameters
1 points1 comments