Changes to state manager and redis datastore schema (#1489)

* Added the TurbiniaRequest hashed object in Redis * Converted TurbiniaTasks to hash objects in Redis * Made get_request_data more efficient * Made get_requests_summary more efficient * Fixed some issues in request_status * Remove GCP dependencies (#1440) * Remove gcp dependencies * Update dockerfiles * Update dockerfiles * Update gcp error reporting * Updates to formatting * Add unit test * Update unit test * Clean up * Update unit test * Update error reporting * Update file * Update config template * Catch exception * Updates * fix lint * Lint fixes * Updates * Updates * Various updates and fixes * Updates * --- (#1483) updated-dependencies: - dependency-name: requests dependency-type: indirect dependency-group: pip - dependency-name: requests dependency-type: indirect dependency-group: pip ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Updates * Update typo * Updates to unit tests * Updates to unit tests and linter fixes * Update table width UI * Bump the pip group across 2 directories with 1 update (#1487) Bumps the pip group with 1 update in the / directory: [requests](https:/psf/requests). Bumps the pip group with 1 update in the /turbinia/api/cli directory: [requests](https:/psf/requests). Updates `requests` from 2.32.0 to 2.32.3 - [Release notes](https:/psf/requests/releases) - [Changelog](https:/psf/requests/blob/main/HISTORY.md) - [Commits](psf/requests@v2.32.0...v2.32.3) Updates `requests` from 2.32.0 to 2.32.3 - [Release notes](https:/psf/requests/releases) - [Changelog](https:/psf/requests/blob/main/HISTORY.md) - [Commits](psf/requests@v2.32.0...v2.32.3) --- updated-dependencies: - dependency-name: requests dependency-type: indirect dependency-group: pip - dependency-name: requests dependency-type: indirect dependency-group: pip ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix small UI bug * Minor typos/errors * Set celery task limits and fix worker timeout (#1486) * Set celery task limits and fix worker timeout * Remove solo pool and change concurrency=1 * Add soft/hard limit buffers * Fix server timeout * Late import for psutil * small fixes, tests * fix run tests * test format string * Fix process_result test * fix execute test * revert unnecessary config changes * Handle soft timeout exception in execute() * update timeout message * Yaaaaaaaapf * Review fixes * Updates and yapf fix * Minor updates * Update docstrings * Updates to evidence_size * Updates * Change log level for message * Lint * bug fixes * Address review comments * Fix docstrings * Minor UI update --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Igor Rodrigues <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Aaron Peterson <[email protected]>
google · Jun 13, 2024 · 0588367 · 0588367
1 parent b162614
commit 0588367
Show file tree

Hide file tree

Showing 140 changed files with 2,147 additions and 1,807 deletions.
diff --git a/.pylintrc b/.pylintrc
@@ -102,7 +102,6 @@ disable=abstract-method,
  no-init, # added
  no-member,
  no-name-in-module,
- no-self-use,
  nonzero-method,
  oct-method,
  old-division,

diff --git a/.style.yapf b/.style.yapf
@@ -4,6 +4,7 @@
 #
 [style]
 based_on_style = yapf
+COLUMN_LIMIT = 80
 COALESCE_BRACKETS = True
 SPLIT_BEFORE_FIRST_ARGUMENT = True
 SPLIT_PENALTY_AFTER_OPENING_BRACKET = 0

diff --git a/docs/conf.py b/docs/conf.py
@@ -14,8 +14,6 @@
 # import sys
 # sys.path.insert(0, os.path.abspath('.'))
 
-from __future__ import unicode_literals
-
 from recommonmark.transform import AutoStructify
 from docutils import nodes, transforms
 

diff --git a/poetry.lock b/poetry.lock
diff --git a/pyproject.toml b/pyproject.toml
@@ -67,4 +67,4 @@ ignore_patterns = [
 
 [build-system]
 requires = ["poetry-core"]
-build-backend = "poetry.core.masonry.api"
+build-backend = "poetry.core.masonry.api"
diff --git a/test_data/state_manager_test_data.json b/test_data/state_manager_test_data.json
@@ -24,7 +24,6 @@
  "sketch_id": null,
  "group_name": "",
  "reason": "",
- "all_args": "turbinia/turbiniactl.py rawdisk -l /workspaces/turbinia/test_data/artifact_disk.dd",
  "group_id": "55ce6e98dc154e73990b24f0c79ab07e",
  "requester": "root"
  }
@@ -70,7 +69,6 @@
  "sketch_id": null,
  "group_name": "",
  "reason": "",
- "all_args": "turbinia/turbiniactl.py rawdisk -l /workspaces/turbinia/test_data/artifact_disk.dd",
  "group_id": "55ce6e98dc154e73990b24f0c79ab07e",
  "requester": "root"
  }
@@ -115,7 +113,6 @@
  "sketch_id": null,
  "group_name": "",
  "reason": "",
- "all_args": "turbinia/turbiniactl.py rawdisk -l /workspaces/turbinia/test_data/artifact_disk.dd",
  "group_id": "55ce6e98dc154e73990b24f0c79ab07e",
  "requester": "root"
  }

diff --git a/tools/turbinia_config_parser.py b/tools/turbinia_config_parser.py
@@ -14,7 +14,6 @@
 # # limitations under the License.
 """Turbinia Config Parser."""
 
-from __future__ import unicode_literals
 from __future__ import print_function
 
 import imp

diff --git a/tools/turbinia_job_graph.py b/tools/turbinia_job_graph.py
@@ -16,7 +16,6 @@
 """Graph to visualise job/evidence relationships."""
 
 from __future__ import print_function
-from __future__ import unicode_literals
 
 import argparse
 import graphviz