Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix comparision for new model/body #4631

Merged
merged 4 commits into from
Jan 31, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
### Fixes
- Projects created using `dbt init` now have the correct `seeds` directory created (instead of `data`) ([#4588](https:/dbt-labs/dbt-core/issues/4588), [#4599](https:/dbt-labs/dbt-core/pull/4589))
- Don't require a profile for dbt deps and clean commands ([#4554](https:/dbt-labs/dbt-core/issues/4554), [#4610](https:/dbt-labs/dbt-core/pull/4610))
- Select modified.body works correctly when new model added([#4570](https:/dbt-labs/dbt-core/issues/4570), [#4631](https:/dbt-labs/dbt-core/pull/4631))

## dbt-core 1.0.1 (January 03, 2022)

Expand Down
67 changes: 30 additions & 37 deletions core/dbt/graph/selector_methods.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import abc
from itertools import chain
from pathlib import Path
from typing import Set, List, Dict, Iterator, Tuple, Any, Union, Type, Optional
from typing import Set, List, Dict, Iterator, Tuple, Any, Union, Type, Optional, Callable

from dbt.dataclass_schema import StrEnum

Expand Down Expand Up @@ -478,42 +478,28 @@ def check_macros_modified(self, node):
previous_macros = []
return self.recursively_check_macros_modified(node, previous_macros)

def check_modified(self, old: Optional[SelectorTarget], new: SelectorTarget) -> bool:
# TODO check modifed_content and check_modified macro seems a bit redundent
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gshank do you know what's the main difference between these two functions? seems a little bit confusing here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the way you're doing this is fine. The check_modified method (state:modified for users) is meant to be the superset of all checks. Renaming this so it clearly maps to same_contents (the umbrella check) makes sense

def check_modified_content(self, old: Optional[SelectorTarget], new: SelectorTarget) -> bool:
different_contents = not new.same_contents(old) # type: ignore
upstream_macro_change = self.check_macros_modified(new)
return different_contents or upstream_macro_change

def check_modified_body(self, old: Optional[SelectorTarget], new: SelectorTarget) -> bool:
if hasattr(new, "same_body"):
return not new.same_body(old) # type: ignore
else:
return False

def check_modified_configs(self, old: Optional[SelectorTarget], new: SelectorTarget) -> bool:
if hasattr(new, "same_config"):
return not new.same_config(old) # type: ignore
else:
return False

def check_modified_persisted_descriptions(
self, old: Optional[SelectorTarget], new: SelectorTarget
) -> bool:
if hasattr(new, "same_persisted_description"):
return not new.same_persisted_description(old) # type: ignore
else:
return False

def check_modified_relation(
self, old: Optional[SelectorTarget], new: SelectorTarget
) -> bool:
if hasattr(new, "same_database_representation"):
return not new.same_database_representation(old) # type: ignore
else:
return False

def check_modified_macros(self, _, new: SelectorTarget) -> bool:
return self.check_macros_modified(new)

@staticmethod
def check_modified_factory(
compare_method: str
) -> Callable[[Optional[SelectorTarget], SelectorTarget], bool]:
# get a function that compares two selector target based on compare method provided
def check_modified_things(old: Optional[SelectorTarget], new: SelectorTarget) -> bool:
if hasattr(new, compare_method):
# when old body does not exist or old and new are not the same
return not old or not getattr(new, compare_method)(old) # type: ignore
else:
return False
return check_modified_things

def check_new(self, old: Optional[SelectorTarget], new: SelectorTarget) -> bool:
return old is None

Expand All @@ -527,14 +513,21 @@ def search(

state_checks = {
# it's new if there is no old version
'new': lambda old, _: old is None,
'new':
lambda old, _: old is None,
# use methods defined above to compare properties of old + new
'modified': self.check_modified,
'modified.body': self.check_modified_body,
'modified.configs': self.check_modified_configs,
'modified.persisted_descriptions': self.check_modified_persisted_descriptions,
'modified.relation': self.check_modified_relation,
'modified.macros': self.check_modified_macros,
'modified':
self.check_modified_content,
'modified.body':
self.check_modified_factory('same_body'),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know if you think we should go change the underlying function names instead of using the factory here! they are used somewhere but I think it is not too much

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the factory. It avoids duplicating the code in the functions.

'modified.configs':
self.check_modified_factory('same_config'),
'modified.persisted_descriptions':
self.check_modified_factory('same_persisted_description'),
'modified.relation':
self.check_modified_factory('same_database_representation'),
'modified.macros':
self.check_modified_macros,
}
if selector in state_checks:
checker = state_checks[selector]
Expand Down
2 changes: 2 additions & 0 deletions test/unit/test_graph_selector_methods.py
Original file line number Diff line number Diff line change
Expand Up @@ -835,6 +835,8 @@ def test_select_state_added_model(manifest, previous_state):
manifest, method, 'modified') == {'another_model'}
assert search_manifest_using_method(
manifest, method, 'new') == {'another_model'}
assert search_manifest_using_method(
manifest, method, 'modified.body') == {'another_model'}


def test_select_state_changed_model_sql(manifest, previous_state, view_model):
Expand Down