Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DocGenerator - Search UI integration #1691

Merged
merged 8 commits into from
Aug 9, 2021
Merged

DocGenerator - Search UI integration #1691

merged 8 commits into from
Aug 9, 2021

Conversation

MiguelazoDS
Copy link
Member

Related issue
#1681

Description

As part of the epic #1600 it was required that:

  • The JSON output generated after parsing the DocStrings in the test files was indexed into ElasticSearch.
  • The Search-UI app should build a request related to the information indexed in ElasticSearch.
  • The Search-UI app should parse the response to display relevant data and create filters.
  • The DocGenerator tool was able to index the data and launch the UI by using parameter options.

The previous three first items were mostly covered in #1666 and continued in #1681.

Dod

1
2
3
4
5
6

@MiguelazoDS MiguelazoDS linked an issue Aug 6, 2021 that may be closed by this pull request
Integrate SearchUI with DocGenerator #1681
Closed
@MiguelazoDS MiguelazoDS self-assigned this Aug 6, 2021
palaciosjeremias
palaciosjeremias reviewed Aug 6, 2021
View reviewed changes
docs/DocGenerator/DocGenerator.py Outdated
@@ -221,6 +223,14 @@ def start_logging(folder, debug_level=logging.INFO):
elif args.sanity:
sanity = Sanity(Config(CONFIG_PATH))
sanity.run()
elif args.index_name:
indexData=IndexData(args.index_name, Config(CONFIG_PATH))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code style: Use white spaces around "="

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

docs/DocGenerator/DocGenerator.py Outdated
indexData=IndexData(args.index_name, Config(CONFIG_PATH))
indexData.run()
elif args.launch_app:
indexData=IndexData(args.launch_app, Config(CONFIG_PATH))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

docs/DocGenerator/lib/IndexData.py Outdated
@@ -0,0 +1,81 @@
"""
brief: Wazuh DocGenerator config parser.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modify this brief with the description of the module.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

docs/DocGenerator/lib/IndexData.py Outdated
brief: Recursively finds all the files that match with the regex provided.
"""
doc_files = []
r = re.compile(self.regex)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the regex isn´t supposed to change, it is recommended to compile it in the init method, so if this method is used more than once, the regex wouldn´t be recompiled each time.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

docs/DocGenerator/lib/IndexData.py Outdated
"""
def __init__(self, index, config):
self.conf = config
self.path = self.conf.documentation_path
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this assignment required? It will be more clear to directly use self.conf.documentation_path

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

docs/DocGenerator/lib/IndexData.py Outdated

def remove_index(self):
"""
brief: It Deletes an index.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

docs/DocGenerator/lib/IndexData.py Outdated
self.path = self.conf.documentation_path
self.index = index
self.regex = ".*json"
self.files = self.get_files()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is required to conserve all the files in the class? We can only obtain them during read_files_content

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

docs/DocGenerator/lib/IndexData.py Outdated

def run(self):
"""
brief: This calls all the methods of the Class. Finally, it uses the index name and the documents
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rephrase this like: Collects all the documentation files and makes a request to the BULK API to index the new data.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

docs/DocGenerator/lib/IndexData.py Outdated
self.read_files_content()
if self.test_connection():
self.remove_index()
print("Indexing data...\n")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add this to the log output

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

docs/DocGenerator/lib/IndexData.py Outdated
print("Indexing data...\n")
helpers.bulk(self.es, self.output, index=self.index)
out=json.dumps(self.es.cluster.health(wait_for_status='yellow', request_timeout=1), indent=4)
print(out)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@MiguelazoDS MiguelazoDS force-pushed the dev-1681-docgenerator-search_ui-integration branch 2 times, most recently from d6fc354 to 3e8f41e Compare August 6, 2021 21:00
@MiguelazoDS MiguelazoDS force-pushed the dev-1681-docgenerator-search_ui-integration branch from 3e8f41e to 45c8911 Compare August 6, 2021 21:07
palaciosjeremias
palaciosjeremias approved these changes Aug 9, 2021
View reviewed changes
Copy link
Contributor

@palaciosjeremias palaciosjeremias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@palaciosjeremias palaciosjeremias merged commit 9fa1d31 into dev-1600-docgenerator-UI Aug 9, 2021
@palaciosjeremias palaciosjeremias deleted the dev-1681-docgenerator-search_ui-integration branch August 9, 2021 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@palaciosjeremias palaciosjeremias palaciosjeremias approved these changes

Assignees

@MiguelazoDS MiguelazoDS

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Integrate SearchUI with DocGenerator
2 participants
@MiguelazoDS @palaciosjeremias