The provider binaries are in the hundreds of MBs and by defaulta are redownloaded into each integration tests' directory, this is a waste of bandwidth, time, and space.
TF supports specifying a provider cache directory [0] which will avoid redownloading them and storing them in multiple places, which should speed up the job time by a bit.
I was having trouble getting it to work with TF 0.12 so I bumped us up to the latest 0.13 beta (stable is scheduled for ~1 month from now)
[0] https://www.terraform.io/docs/configuration/providers.html#provider-plugin-cache