Remove unnecessary guild redownloading (#383)

Created by: js6pak

Summary

Fixes #383 (closed)

Details

Repair #383 (closed), while not breaking #275 (closed), removing unnecessary guild downloads that only slow down the entire request. Even Kiritsu doesn't know why he wrote it now.