Uploaded image for project: 'Planet4'
  1. Planet4
  2. PLANET-5408

Backstop reference image should also be the previous version when re-running the pipeline

    XporterXMLWordPrintable

    Details

    • Type: Bug
    • Status: CLOSED
    • Priority: Must have
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None
    • Section:
      CI/CD
    • Track:
      Development

      Description

      When we trigger the staging pipelines, this pipeline has a backstop reference step which makes reference snapshots before deploying the new version to staging. Sometimes a pipeline needs to run more than once (e.g. an intermittent issue). When that happens the reference image is taken again, however if the pipeline previously had already deployed the new image, the reference snapshot is not from the previous version anymore. This is quite risky since we have the AUTO-PROCEED logic that deploys to production if backstop detects no differences. So if that second run contains regressions that were also in the first run, they will go unnoticed straight to production.

      Possible approaches:

      • Revert staging to the previous version each time to take the snapshot. Seems like it could increase pipeline times, but maybe it wouldn't: the backstop history job runs parallel with build-staging and takes at least 5 minutes less than that one. So if we already have the image of the previous version, we could probably deploy it within that time.
      • Create the reference image on the first run only, then on subsequent runs re-use that one.
      • Perhaps we could create the reference image from production instead. This option would require that we ensure they have the same data before making the new snapshot.

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                Assignee:
                Unassigned
                Reporter:
                pvincent Pieter Vincent
                Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved: