Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add check_components preflight to detect metadata component conflicts (#3837) #3837

Merged
merged 19 commits into from
Oct 9, 2024

Conversation

vsbharath
Copy link
Contributor

@vsbharath vsbharath commented Sep 30, 2024

Summuray

CheckComponents task is designed to check for existing metadata components in a Salesforce org before deployment, helping to prevent deployment failures due to conflicts with existing components.

Whats Included

  1. New CheckComponents Task
  • Functionality:
    Collects metadata components from specified paths or tasks within a plan or flow.
    Check if these components exist in the target Salesforce org using the Metadata API.

  • Options:

paths: List of paths to include in the check (optional).
plan_name: Name of the current plan (optional).
flow_name: Name of the current flow (optional).
  1. Unit Tests for CheckComponents

Sample Plan and Flow Integration:

plans:
  install:
    steps:
      0:
        task: check_components
        options:
          paths: 'force-app'
          plan_name: 'install'
      2:
        flow: deploy_metadata

flows:
  homehealth_metadata_setup:
    steps:
      0:
        task: check_components
        options:
          paths: 'src'
          flow_name: 'homehealth_metadata_setup'
      2:
        flow: dependencies
      3:
        flow: deploy_hh_metadata

@vsbharath vsbharath requested a review from a team as a code owner September 30, 2024 22:32
@jstvz jstvz added the internal Pull requests from IPDE team label Oct 1, 2024
Copy link
Contributor

@jstvz jstvz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @vsbharath! This will be a nice tool for pre-deployment checks. However, testing against EDA revealed some issues.

When running cci task run check_components --plan_name install --org dev, the task fails to detect deploy paths, even though they exist in the customer_org flow (e.g., unpackaged/pre and unpackaged/post). It incorrectly reports "No deploy paths found," indicating a problem with detecting paths in nested flows.

The main issue seems to be with the conditional block—it's hard to follow and doesn't correctly decompose plans and tasks. I recommend reviewing the _freeze_steps method from the metadeploy task. Lantz has already started extracting this functionality in his PR.

For example, both deploy_pre and deploy_post handle groups of paths and should be decomposed into individual deploy calls. If you're using the List[dict] returned by _freeze_steps, make sure to detect the task_class class path to handle this properly.

Another suggestion is to merge all package.xml files into one. For projects like NPSP with many deploy calls, this would be much faster. I'm not sure if we've discussed this before as a v2.

cumulusci/tasks/salesforce/check_components.py Outdated Show resolved Hide resolved
cumulusci/tasks/salesforce/check_components.py Outdated Show resolved Hide resolved
cumulusci/cumulusci.yml Outdated Show resolved Hide resolved
cumulusci/tasks/salesforce/check_components.py Outdated Show resolved Hide resolved
cumulusci/tasks/salesforce/check_components.py Outdated Show resolved Hide resolved
Co-authored-by: James Estevez <[email protected]>
@vsbharath
Copy link
Contributor Author

Thanks @vsbharath! This will be a nice tool for pre-deployment checks. However, testing against EDA revealed some issues.

When running cci task run check_components --plan_name install --org dev, the task fails to detect deploy paths, even though they exist in the customer_org flow (e.g., unpackaged/pre and unpackaged/post). It incorrectly reports "No deploy paths found," indicating a problem with detecting paths in nested flows.

The main issue seems to be with the conditional block—it's hard to follow and doesn't correctly decompose plans and tasks. I recommend reviewing the _freeze_steps method from the metadeploy task. Lantz has already started extracting this functionality in his PR.

For example, both deploy_pre and deploy_post handle groups of paths and should be decomposed into individual deploy calls. If you're using the List[dict] returned by _freeze_steps, make sure to detect the task_class class path to handle this properly.

Another suggestion is to merge all package.xml files into one. For projects like NPSP with many deploy calls, this would be much faster. I'm not sure if we've discussed this before as a v2.

I will take a look at it. It still not resolving the steps from the source. Will address these and get back

@vsbharath vsbharath requested a review from jstvz October 4, 2024 07:32
@vsbharath
Copy link
Contributor Author

Thanks @jstvz for taking the time to review. I have addressed all the comments. Please review and let me know if any changes are required.

@vsbharath
Copy link
Contributor Author

Thanks @jstvz for taking the time to review. I have addressed all the comments. Please review and let me know if any changes are required.

Apologies for more changes. Added few more tests to utils and pushed the changes for review

jstvz
jstvz previously approved these changes Oct 8, 2024
Copy link
Contributor

@jstvz jstvz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Additional tweaks suggested below.

cumulusci/core/utils.py Outdated Show resolved Hide resolved
cumulusci/core/utils.py Outdated Show resolved Hide resolved
cumulusci/core/tests/test_utils.py Outdated Show resolved Hide resolved
cumulusci/tasks/salesforce/check_components.py Outdated Show resolved Hide resolved
cumulusci/tasks/salesforce/check_components.py Outdated Show resolved Hide resolved
@vsbharath
Copy link
Contributor Author

@jstvz Addressed all the suggestions. Need your approval one more time.

@vsbharath vsbharath requested a review from jstvz October 9, 2024 01:10
@jstvz jstvz changed the title @W-16780066 - New task to Check components in the org from deploy paths Add check_components preflight to detect metadata component conflicts (#3837) Oct 9, 2024
@jstvz jstvz merged commit 39ed860 into main Oct 9, 2024
21 of 23 checks passed
@jstvz jstvz deleted the feature/check_components branch October 9, 2024 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
internal Pull requests from IPDE team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants