optional-component secure-data-proxy + related configs in cowbird/magpie/weaver #283
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… secure-data-proxy component
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac/48/Result : failure BIRDHOUSE_DEPLOY_BRANCH : secure-data-proxy DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-36.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1153/Result : failure BIRDHOUSE_DEPLOY_BRANCH : secure-data-proxy DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-36.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1156/Result : failure BIRDHOUSE_DEPLOY_BRANCH : secure-data-proxy DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-36.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac/51/Result : failure BIRDHOUSE_DEPLOY_BRANCH : secure-data-proxy DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-36.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1192/Result : failure BIRDHOUSE_DEPLOY_BRANCH : secure-data-proxy DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
|
run tests |
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1202/Result : failure BIRDHOUSE_DEPLOY_BRANCH : secure-data-proxy DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-118.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1204/Result : failure BIRDHOUSE_DEPLOY_BRANCH : secure-data-proxy DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-91.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
matprov
reviewed
Feb 7, 2023
…solution by default when secure-data-proxy component is not enabled
|
2 builds are expected after this comment ⬇️ (comments by test suite itself to appear eventually...)
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1221/Result : failure BIRDHOUSE_DEPLOY_BRANCH : secure-data-proxy DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-36.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/952/NOTEBOOK TEST RESULTS |
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1223/Result : failure BIRDHOUSE_DEPLOY_BRANCH : secure-data-proxy DACCS_CONFIGS_BRANCH : secure-data-proxy PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-36.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/954/NOTEBOOK TEST RESULTS |
|
@tlvu |
tlvu
reviewed
Feb 10, 2023
tlvu
approved these changes
Feb 10, 2023
There was a problem hiding this comment.
Magpie DB upgrade seems to be fine, only 1 DB upgrade:
[2023-02-10 01:29:43,140] INFO [MainThread][alembic.runtime.migration] Running upgrade 0c6269f410cd -> 5e5acc33adce, Case Insensitive Email Constraint
However this new Magpie version seems to have a bunch of warnings, anything to be worried about?
/opt/local/src/magpie/magpie/models.py:935: SAWarning: Enum 'length' argument is currently ignored unless native_enum is specified as False, including for DDL that renders VARCHAR in any case. This may change in a future release.
operation = sa.Column(sa.Enum(TokenOperation, name=TokenOperation.__name__, length=32), nullable=False)
/opt/local/src/magpie/magpie/db.py:44: SAWarning: relationship 'Group.users_dynamic' will copy column groups.id to column users_groups.group_id, which conflicts with relationship(s): 'Group.users' (copies groups.id to users_groups.group_id), 'User.groups' (copies groups.id to users_groups.group_id). If this
is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the co
lumns that should be written towards. To silence this warning, add the parameter 'overlaps="groups,users"' to the 'Group.users_dynamic' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
configure_mappers()
/opt/local/src/magpie/magpie/db.py:44: SAWarning: relationship 'Group.users_dynamic' will copy column users.id to column users_groups.user_id, which conflicts with relationship(s): 'Group.users' (copies users.id to users_groups.user_id), 'User.groups' (copies users.id to users_groups.user_id). If this is no
t the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns
that should be written towards. To silence this warning, add the parameter 'overlaps="groups,users"' to the 'Group.users_dynamic' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
configure_mappers()
/opt/local/src/magpie/magpie/db.py:44: SAWarning: relationship 'Group.resources_dynamic' will copy column groups.id to column resources.owner_group_id, which conflicts with relationship(s): 'Group.resources' (copies groups.id to resources.owner_group_id), 'Resource.owner_group' (copies groups.id to resource
s.owner_group_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can
be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="owner_group,resources"' to the 'Group.resources_dynamic' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
configure_mappers()
/opt/local/src/magpie/magpie/db.py:44: SAWarning: relationship 'Resource.groups' will copy column resources.resource_id to column groups_resources_permissions.resource_id, which conflicts with relationship(s): 'Resource.group_permissions' (copies resources.resource_id to groups_resources_permissions.resourc
e_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to
isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="group_permissions"' to the 'Resource.groups' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
configure_mappers()
/opt/local/src/magpie/magpie/db.py:44: SAWarning: relationship 'Resource.groups' will copy column groups.id to column groups_resources_permissions.group_id, which conflicts with relationship(s): 'Group.resource_permissions' (copies groups.id to groups_resources_permissions.group_id), 'GroupResourcePermissio
n.groups' (copies groups.id to groups_resources_permissions.group_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are par
tially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="groups,resource_permissions"' to the 'Resource.groups' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
configure_mappers()
/opt/local/src/magpie/magpie/db.py:44: SAWarning: relationship 'Resource.users' will copy column resources.resource_id to column users_resources_permissions.resource_id, which conflicts with relationship(s): 'Resource.user_permissions' (copies resources.resource_id to users_resources_permissions.resource_id
). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isol
ate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="user_permissions"' to the 'Resource.users' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
configure_mappers()
/opt/local/src/magpie/magpie/db.py:44: SAWarning: relationship 'User.groups_dynamic' will copy column users.id to column users_groups.user_id, which conflicts with relationship(s): 'Group.users' (copies users.id to users_groups.user_id), 'Group.users_dynamic' (copies users.id to users_groups.user_id), 'User
.groups' (copies users.id to users_groups.user_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping,
the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="groups,users,users_dynamic"' to the 'User.groups_dynamic' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
configure_mappers()
/opt/local/src/magpie/magpie/db.py:44: SAWarning: relationship 'User.groups_dynamic' will copy column groups.id to column users_groups.group_id, which conflicts with relationship(s): 'Group.users' (copies groups.id to users_groups.group_id), 'Group.users_dynamic' (copies groups.id to users_groups.group_id),
'User.groups' (copies groups.id to users_groups.group_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially over
lapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="groups,users,users_dynamic"' to the 'User.groups_dynamic' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
configure_mappers()
/opt/local/src/magpie/magpie/db.py:44: SAWarning: relationship 'User.resource_permissions' will copy column users.id to column users_resources_permissions.user_id, which conflicts with relationship(s): 'Resource.users' (copies users.id to users_resources_permissions.user_id). If this is not the intention, c
onsider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be wri
tten towards. To silence this warning, add the parameter 'overlaps="users"' to the 'User.resource_permissions' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
configure_mappers()
|
Please remove |
Ah I see why, in |
Those are produced in preparation of SQLAalchemy 2.0. |
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1229/Result : failure BIRDHOUSE_DEPLOY_BRANCH : secure-data-proxy DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-91.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/957/NOTEBOOK TEST RESULTS |
|
@tlvu Just so we are in sync, I'll bump this to 1.23.0 now and merge. |
github-actions
bot
added
ci/operations
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1230/Result : failure BIRDHOUSE_DEPLOY_BRANCH : secure-data-proxy DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-91.rdext.crim.ca Infrastructure deployment failed. Instance has not been destroyed. @matprov |
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1231/Result : failure BIRDHOUSE_DEPLOY_BRANCH : secure-data-proxy DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-92.rdext.crim.ca Infrastructure deployment failed. Instance has not been destroyed. @matprov |
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1235/Result : failure BIRDHOUSE_DEPLOY_BRANCH : secure-data-proxy DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// Infrastructure deployment failed. Instance has not been destroyed. @matprov |
|
Sorry for these annoying e2e test results messages. They are due to the fact that the branch is deleted since merged. I'm testing something to avoid posting these useless notifications when branch gets deleted. |
fmigneault
added a commit
that referenced
this pull request
Apr 20, 2023
## Overview
Please include a summary of the changes and which issues are fixed.
Please also include relevant motivation and context.
List any dependencies that are required for this change.
## Breaking changes
- CanarieAPI: update to `0.7.1`.
- The Docker running `CanarieAPI` is now using Python 3 (since `0.4.x` tags).
Configurations need to be updated if any specific Python 2 definitions were used.
See [2to3](https://docs.python.org/3/library/2to3.html) to help migrate configurations automatically if necessary.
- Update the [CanarieAPI configuration](birdhouse/config/canarie-api/docker_configuration.py.template) to use
Python 3.x executable code.
## Changes
- CanarieAPI: update to `0.7.1`.
- The server node now provides a generic ``server`` configuration for the current ``platform`` definition.
- Added multiple missing docuementation references for all the services included within `CanarieAPI` configurations.
- With new `CanarieAPI` version, a slightly improved UI with more service details are provided for the active server:
- Add optional variables witht defaults to define reference Docker image version tags.
Following optional variables are defined by default. These are used as reference in the respective Docker compose
service definition of these components, as well as in their `CanarieAPI` configuration to retrieve the release time
of the tag, and refer to relevant URL references as needed.
- `CATALOG_VERSION`
- `FINCH_VERSION`
- `FLYINGPIGEON_VERSION`
- `GEOSERVER_VERSION`
- `HUMMINGBIRD_VERSION`
- `MALLEEFOWL_VERSION`
- `RAVEN_VERSION`
## Fixes:
- CanarieAPI: update to `0.7.1`.
- Fixes an `AttributeError` raised due to misconfiguration of the Web Application with Flask 2.x definitions
(relates to [Ouranosinc/CanarieAPI#10](Ouranosinc/CanarieAPI#10)).
- Skip over `0.4.x`, `0.5.x`, `0.6.x` versions to avoid issue related to `cron` job monitoring and log parser
command failures in order to collect configured service statistics and statuses
(see also [Ouranosinc/CanarieAPI#14](Ouranosinc/CanarieAPI#14)).
- Weaver: update CanarieAPI monitoring definitions
- Move monitoring of public endpoint under [optional-components/canarie-api-full-monitoring][canarie-monitor].
- Add monitoring of private endpoint by default when using Weaver component.
- Cowbird: update CanarieAPI monitoring definitions
- Add monitoring of public endpoint under [optional-components/canarie-api-full-monitoring][canarie-monitor].
- Add public Magpie permission on Cowbird entrypoint only to allow its monitoring.
## Additional Information
Resolves the following log error.
```
proxy | [2023-01-31 19:37:01 +0000] [37] [DEBUG] GET /canarie/
proxy | [2023-01-31 19:37:01,708] [37] [INFO] app_object : Disconnecting from database
proxy | [2023-01-31 19:37:01,709] [37] [DEBUG] app_object : Using db filename : /opt/local/src/CanarieAPI/stats.db
proxy | [2023-01-31 19:37:01 +0000] [37] [DEBUG] Closing connection.
proxy | [2023-01-31 19:37:02 +0000] [37] [DEBUG] GET /canarie/background.jpg
proxy | [2023-01-31 19:37:02,176] [37] [INFO] app_object : Disconnecting from database
proxy | [2023-01-31 19:37:02,176] [37] [DEBUG] app_object : Using db filename : /opt/local/src/CanarieAPI/stats.db
proxy | [2023-01-31 19:37:02 +0000] [37] [ERROR] Error handling request /canarie/background.jpg
proxy | Traceback (most recent call last):
proxy | File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/async.py", line 56, in handle
proxy | self.handle_request(listener_name, req, client, addr)
proxy | File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/ggevent.py", line 152, in handle_request
proxy | super(GeventWorker, self).handle_request(*args)
proxy | File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/async.py", line 107, in handle_request
proxy | respiter = self.wsgi(environ, resp.start_response)
proxy | File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1997, in __call__
proxy | return self.wsgi_app(environ, start_response)
proxy | File "/opt/local/src/CanarieAPI/canarieapi/reverse_proxied.py", line 33, in __call__
proxy | return self.app(environ, start_response)
proxy | File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1985, in wsgi_app
proxy | response = self.handle_exception(e)
proxy | File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1532, in handle_exception
proxy | handler = self._find_error_handler(InternalServerError())
proxy | File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1449, in _find_error_handler
proxy | .get(code))
proxy | File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1440, in find_handler
proxy | handler = handler_map.get(cls)
proxy | AttributeError: 'function' object has no attribute 'get'
```
The above problem lead to unresponsive CanarieAPI, which in turn caused the platform to fail responding with successful monitoring statuses for requests toward the configured services, which in turn, caused the stack to never completely boot. When the stack failed to boot, the *End2End Test Results* (example: #283 (comment)) could not run due to the unresponsive instance, which is the cause of the incomplete output:
`````
Tests URL :
NOTEBOOK TEST RESULTS
````
</code>
``
`````
# To Do (in follow-up PRs)
- remove deprecated configs deleted in #287
- remove deprecated configs deleted in #291
- remove deprecated configs deleted in #292
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Multiple updates to support secured WPS outputs.
Changes
Non-breaking changes
secure-data-proxy: add new
secure-data-proxyoptional component.When enabled, this component will enforce authentication and authorization to be resolved against the
/wpsoutputsendpoint prior to accessing the results produced by WPS executions. A Magpie service named
secure-data-proxyiscreated to define the resource and permission hierarchy of directories and files the users and groups can access.
When disabled, the original behavior to provide open access to
/wpsoutputsis employed.A variable named
SECURE_DATA_PROXY_AUTH_INCLUDEis dynamically assigned based on the activation or not of thiscomponent. Corresponding validation of optional/mandatory/delayed-eval variables used by this component are also
applied dynamically, as well as mounting the necessary
nginxanddocker-composeextended configurations.Weaver: adjust user-context output directory hooks and permissions for
secure-data-proxy.When a process defined in Weaver (either a WPS provider or a local definition) is executed by a user that was granted
authorization to run a job, the corresponding user-context directory under
/wpsoutputs/users/{user-id}will be usedfor storing the execution outputs and will have the appropriate permissions set for that user to grant them access to
those outputs.
Magpie/Twitcher: update minimum version
magpie>=3.31.0to employtwitcher>=0.8.0inMapgieAdatepr.Resolve an issue where
response.requestreferences were not set in OWS proxy responses when handled by Twitcher.This caused
MapgieAdateprresponse hooks to fail, which in turn caused failing requests for any non-WPSservice that defined any proxy request hook, such as in the case of
weavercomponent.Adds the Twitcher
/ows/verify/{service_name}[/{extra_path}endpoint employed for validating authorized accessto Magpie service/resources, in the same fashion as the protected proxy endpoint, but without performing the proxied
request toward the target service. This is mandatory for using the new
secure-data-proxyoptional component, otherwise the proxy endpoint triggers data download twice, once for authorization and another
for actually accessing the data.
Breaking changes
New feature
secure-data-proxywith optional component should default to using the original methodology of public access if not enabled.Related Issue / Discussion
response.requestreference usingtwitcher>=0.8.0withadapter.send_requestmethod Ouranosinc/Magpie#571, which in turn depends on move ows proxysend_requestfunction under corresponding adapter method twitcher#118. Corresponding Dockersmagpie>=3.31.0andtwitcher>=0.8.0must be deployed and used.(planing ahead some related permissions to user-context / wps outputs under weaver)
Testing
To test the feature, simply enabled it in the
EXTRA_CONF_DIRS, and place some files under${DATA_PERSIST_ROOT}/wpsoutputs.There should be a predefined
/wpsoutputs/publiclocation where files would be accessible when not logged in. Other locations should return 403. Then, a specific directory/file can be defined in Magpie with the relevant user permission on that resource to validate access to that file when logged in.To Do
send_requestfunction under corresponding adapter method twitcher#118response.requestreference usingtwitcher>=0.8.0withadapter.send_requestmethod Ouranosinc/Magpie#571secure-data-proxybranch for PR test suite