Data360 DQ+

 View Only
  • 1.  Compare Pipelines in different environments

    Posted 04-28-2021 07:43

    It would be an awesome feature to be able to compare the same pipeline in 2 different environments - highlighting the differences.  This would assist in stopping unintended changes to be moved up to production.  I looked into creating this using WinMerge.  It showed promise but I would need to write something custom to loop through each file of an exported solution. 



  • 2.  RE: Compare Pipelines in different environments

    Employee
    Posted 04-28-2021 08:32

    The easiest way to extract this data would be via the API console with the following queries. The first query below pulls all environments, pipelines, paths and stages. The second query pulls a specific environment's pipelines, paths and stages.

    The outputs of these queries will give a cleaner view to the stage types and names for comparison outside the product. 

    All environments, pipelines, paths and stages

    query {
    envs {
    edges {
    node {
    name
    pipelines {
    edges {
    node {
    name
    paths {
    name
    stages {
    edges {
    node {
    stageType
    name
    }
    }
    }
    }
    }
    }
    }
    }
    }
    }
    }

    A single environments, pipelines, paths and stages

    Note that the environment name will need to be manually updated:

    query {
    pipelines(envNameOrId: "YOUR_ENV_NAME") {
    edges {
    node {
    name
    paths {
    name
    stages {
    edges {
    node {
    stageType
    name
    }
    }
    }
    }
    }
    }
    }
    }


  • 3.  RE: Compare Pipelines in different environments

    Posted 04-28-2021 12:04

    Thanks for the quick reply.  But this doesn't give me what I'm looking for.  I want a way to highlight the difference between two versions of the same solution.  



  • 4.  RE: Compare Pipelines in different environments

    Employee
    Posted 04-28-2021 13:51

    It doesn't give differences directly, but running it twice (once per environment) provides the ability to compare the differences in output. It's easier for external tools to compare the output of the API call than the contents of an unzipped export.

    The examples in my last note only give the high-level definitions, such as differences in the arrangement or names of pipelines, paths and stages, such as "Item X" wasn't promoted to a higher environment yet. Are you looking to also compare the stage definitions too, such as nodes within an Analysis, fields within Data Store, etc.? If so, that's also possible by adding one more "definition" field to the API call, such as the examples below:

    All environments, pipelines, paths and stages

    query {
    envs {
    edges {
    node {
    name
    pipelines {
    edges {
    node {
    name
    paths {
    name
    stages {
    edges {
    node {
    stageType
    name
    definition {
    def
    displayName
    envId
    lastUpdateTimestamp
    latestVersion
    stageId
    stageType
    tenantId
    uid
    version
    }
    }
    }
    }
    }
    }
    }
    }
    }
    }
    }
    }

    A single environments, pipelines, paths and stages

    Note that the environment name will need to be manually updated:

    query {
    pipelines(envNameOrId: "YOUR_ENV_NAME") {
    edges {
    node {
    name
    paths {
    name
    stages {
    edges {
    node {
    stageType
    name
    definition {
    def
    displayName
    envId
    lastUpdateTimestamp
    latestVersion
    stageId
    stageType
    tenantId
    uid
    version
    }
    }
    }
    }
    }
    }
    }
    }
    }