sync layer norms#272

Merged

stas00 merged 14 commits intomainfrom

layer-norm-auto-sync

Jul 4, 2022

Commits on Mar 25, 2022

sync layer norms

thomasw21
authored and
stas00
committed
all_reduce is an in_place operation
thomasw21
committed

Commits on Apr 6, 2022

Commits on May 4, 2022

Merge remote-tracking branch 'origin/main' into layer-norm-auto-sync
thomasw21
committed

Commits on May 18, 2022

add eval dataloader deadlock workaround
stas00
committed

Commits on May 24, 2022

revert generator sync
stas00
committed

Commits on Jun 3, 2022

Merge remote-tracking branch 'origin/main' into layer-norm-auto-sync
stas00
committed

Commits on Jun 29, 2022

Merge remote-tracking branch 'origin/main' into layer-norm-auto-sync
stas00
committed