When we trigger the staging pipelines, this pipeline has a backstop reference step which makes reference snapshots before deploying the new version to staging. Sometimes a pipeline needs to run more than once (e.g. an intermittent issue). When that happens the reference image is taken again, however if the pipeline previously had already deployed the new image, the reference snapshot is not from the previous version anymore. This is quite risky since we have the AUTO-PROCEED logic that deploys to production if backstop detects no differences. So if that second run contains regressions that were also in the first run, they will go unnoticed straight to production.

Possible approaches:

• Revert staging to the previous version each time to take the snapshot. Seems like it could increase pipeline times, but maybe it wouldn't: the backstop history job runs parallel with build-staging and takes at least 5 minutes less than that one. So if we already have the image of the previous version, we could probably deploy it within that time.
• Create the reference image on the first run only, then on subsequent runs re-use that one.
• Perhaps we could create the reference image from production instead. This option would require that we ensure they have the same data before making the new snapshot.