I think there's some plagiarism at play here - https://www.quora.com/What-is-gradient-Accumulation-in-deep-learning-1.