Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bring back VRF native support #252

Open
kmisak opened this issue Jun 1, 2024 · 7 comments
Open

Bring back VRF native support #252

kmisak opened this issue Jun 1, 2024 · 7 comments
Assignees
@thatmattlove
Labels
feature New feature or request needs-info Issue lacks appropriate detail or requires additional information

Comments

@kmisak
Copy link

kmisak commented Jun 1, 2024

Feature Description

Previous versions of Hyperglass natively supports VRF, new one - not. It can be done with custom directives, but all pretty formatting of output is lost.

Is your feature request related to a problem? Please describe.

Lot of networks use VRFs, removing support for the reduces Hyperglass big flexibility.

Environment & Use Case

Any contemporary router supports VRFs

Additional Context
@kmisak kmisak added the feature New feature or request label Jun 1, 2024
@thatmattlove
Copy link
Owner

Try adding a custom directive and setting the following attribute:

your-directive:
    name: BGP Route (VRF x)
    table_output: __hyperglass_juniper_bgp_route_table__

And on the device:

devices:
    - name: your device
      platform: juniper
      directives:
          - your-directive
      structured_output: true

Supported values for table output are:

table_output Platform Type
__hyperglass_juniper_bgp_route_table__ Juniper BGP Route
__hyperglass_juniper_bgp_aspath_table__ Juniper BGP AS Path
__hyperglass_juniper_bgp_community_table__ Juniper BGP Community
__hyperglass_arista_eos_bgp_route_table__ Arista BGP Route
__hyperglass_arista_eos_bgp_aspath_table__ Arista BGP AS Path
__hyperglass_arista_eos_bgp_community_table__ Arista BGP Community

If this works, I will come up with a way to make this more user friendly and add docs.

@thatmattlove thatmattlove self-assigned this Jun 1, 2024
@thatmattlove thatmattlove added the needs-info Issue lacks appropriate detail or requires additional information label Jun 1, 2024
@kmisak
Copy link
Author

kmisak commented Jun 3, 2024

Thank you for suggestion, but that doesn't work. Table output uses | display xml format as input, I tried to use that too - no luck.

Here is device sample:

routers:

  • name: mx-moscow
    address: 10.x.x.x
    group: AS49800
    credential:
    username: xxxx
    password: xxxxx
    port: 22
    platform: juniper
    driver: netmiko
    attrs:
    source4: x.x.x.x
    source6: x::x
    directives:
    - builtins: false
    - bgp_route_vrf
    - ping_vrf
    - traceroute_vrf
    structured_output: true

And directive:

bgp_route_vrf:
name: BGP Route
rules:
- condition: 0.0.0.0/0
action: permit
command: "show route protocol bgp table AS49800.inet.0 {target} detail"
- condition: ::/0
action: permit
command: "show route protocol bgp table AS49800.inet6.0 {target} detail"
field:
description: "IP Address, Prefix, or Hostname"
table_output: hyperglass_juniper_bgp_route_table

@cameront-1
Copy link

cameront-1 commented Jun 7, 2024 •
edited
Loading

Confirming this issue exists on Arista as well. Same instructions. Custom directives result in loss of pretty-formatting, instructing table/structured output has no effect

    platform: arista_eos
    directives:
       - test3
    structured_output: true

test3:
  name: BGPtest
  table_output: __hyperglass_arista_eos_bgp_route_table__
  rules:
    - condition: '0.0.0.0/0'
      ge: 4
      le: 32
      command: 'show ip bgp {target} vrf INTERNET | json'

If I modify base files and force the base config to query against another VRF, we get the following errors being thrown:

hyperglass-1  | [CRITICAL] 20240607 13:31:38 |63 | parse_arista → 11 validation errors for AristaBGPTable
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.0.routeDetail.extCommunityListRaw.0
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=595497215591610, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.0.routeDetail.extCommunityListRaw.1
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=876972192302256, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.1.routeDetail.extCommunityListRaw.0
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=595497215590574, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.1.routeDetail.extCommunityListRaw.1
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=595497215591602, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.1.routeDetail.extCommunityListRaw.2
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=876972192302256, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.2.routeDetail.extCommunityListRaw.0
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=595497215591603, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.2.routeDetail.extCommunityListRaw.1
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=595497215596861, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.2.routeDetail.extCommunityListRaw.2
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=876972192302256, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.3.routeDetail.extCommunityListRaw.0
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=595497215591605, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.3.routeDetail.extCommunityListRaw.1
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=595497215593314, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.3.routeDetail.extCommunityListRaw.2
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=876972192302256, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type {'plugin': 'BGPRoutePluginArista'}
hyperglass-1  | [CRITICAL] 20240607 13:31:38 |48 | default_handler → Error {'method': 'POST', 'path': '/api/query', 'detail': 'expected str, got list'}
hyperglass-1  | ERROR - 2024-06-07 13:31:38,675 - litestar - config - Uncaught exception (connection_type=http, path=/api/query):
hyperglass-1  | Traceback (most recent call last):
hyperglass-1  |   File "/opt/hyperglass/hyperglass/plugins/_builtin/bgp_route_arista.py", line 42, in parse_arista
hyperglass-1  |     validated = AristaBGPTable(**routes)
hyperglass-1  |                 ^^^^^^^^^^^^^^^^^^^^^^^^
hyperglass-1  |   File "/usr/local/lib/python3.12/site-packages/pydantic/main.py", line 176, in __init__
hyperglass-1  |     self.__pydantic_validator__.validate_python(data, self_instance=self)
hyperglass-1  | pydantic_core._pydantic_core.ValidationError: 11 validation errors for AristaBGPTable
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.0.routeDetail.extCommunityListRaw.0
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=595497215591610, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.0.routeDetail.extCommunityListRaw.1
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=876972192302256, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.1.routeDetail.extCommunityListRaw.0
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=595497215590574, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.1.routeDetail.extCommunityListRaw.1
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=595497215591602, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.1.routeDetail.extCommunityListRaw.2
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=876972192302256, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.2.routeDetail.extCommunityListRaw.0
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=595497215591603, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.2.routeDetail.extCommunityListRaw.1
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=595497215596861, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.2.routeDetail.extCommunityListRaw.2
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=876972192302256, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.3.routeDetail.extCommunityListRaw.0
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=595497215591605, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.3.routeDetail.extCommunityListRaw.1
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=595497215593314, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  | bgpRouteEntries.`1.1.1.0/24`.bgpRoutePaths.3.routeDetail.extCommunityListRaw.2
hyperglass-1  |   Input should be a valid string [type=string_type, input_value=876972192302256, input_type=int]
hyperglass-1  |     For further information visit https://errors.pydantic.dev/2.7/v/string_type
hyperglass-1  |
hyperglass-1  | During handling of the above exception, another exception occurred:
hyperglass-1  |
hyperglass-1  | Traceback (most recent call last):
hyperglass-1  |   File "/usr/local/lib/python3.12/site-packages/litestar/middleware/_internal/exceptions/middleware.py", line 158, in __call__
hyperglass-1  |     await self.app(scope, receive, capture_response_started)
hyperglass-1  |   File "/usr/local/lib/python3.12/site-packages/litestar/routes/http.py", line 80, in handle
hyperglass-1  |     response = await self._get_response_for_request(
hyperglass-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
hyperglass-1  |   File "/usr/local/lib/python3.12/site-packages/litestar/routes/http.py", line 132, in _get_response_for_request
hyperglass-1  |     return await self._call_handler_function(
hyperglass-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
hyperglass-1  |   File "/usr/local/lib/python3.12/site-packages/litestar/routes/http.py", line 152, in _call_handler_function
hyperglass-1  |     response_data, cleanup_group = await self._get_response_data(
hyperglass-1  |                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
hyperglass-1  |   File "/usr/local/lib/python3.12/site-packages/litestar/routes/http.py", line 200, in _get_response_data
hyperglass-1  |     else await route_handler.fn(**parsed_kwargs)
hyperglass-1  |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
hyperglass-1  |   File "/opt/hyperglass/hyperglass/api/routes.py", line 110, in query
hyperglass-1  |     output = await execute(data)
hyperglass-1  |              ^^^^^^^^^^^^^^^^^^^
hyperglass-1  |   File "/opt/hyperglass/hyperglass/execution/main.py", line 69, in execute
hyperglass-1  |     output = await driver.response(response)
hyperglass-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
hyperglass-1  |   File "/opt/hyperglass/hyperglass/execution/drivers/_common.py", line 43, in response
hyperglass-1  |     response = self.plugin_manager.execute(output=output, query=self.query_data)
hyperglass-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
hyperglass-1  |   File "/opt/hyperglass/hyperglass/plugins/_manager.py", line 187, in execute
hyperglass-1  |     result = plugin.process(output=result, query=query)
hyperglass-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
hyperglass-1  |   File "/opt/hyperglass/hyperglass/plugins/_builtin/bgp_route_arista.py", line 91, in process
hyperglass-1  |     return parse_arista(output)
hyperglass-1  |            ^^^^^^^^^^^^^^^^^^^^
hyperglass-1  |   File "/opt/hyperglass/hyperglass/plugins/_builtin/bgp_route_arista.py", line 64, in parse_arista
hyperglass-1  |     raise ParsingError(err.errors()) from err
hyperglass-1  |           ^^^^^^^^^^^^^^^^^^^^^^^^^^
hyperglass-1  |   File "/opt/hyperglass/hyperglass/exceptions/_common.py", line 176, in __init__
hyperglass-1  |     self._message = self._safe_format(message, **kwargs)
hyperglass-1  |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
hyperglass-1  |   File "/opt/hyperglass/hyperglass/exceptions/_common.py", line 62, in _safe_format
hyperglass-1  |     keys = get_fmt_keys(template)
hyperglass-1  |            ^^^^^^^^^^^^^^^^^^^^^^
hyperglass-1  |   File "/opt/hyperglass/hyperglass/util/tools.py", line 117, in get_fmt_keys
hyperglass-1  |     for block in (b for b in string.Formatter.parse("", template) if isinstance(template, str)):
hyperglass-1  |                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
hyperglass-1  |   File "/usr/local/lib/python3.12/string.py", line 288, in parse
hyperglass-1  |     return _string.formatter_parser(format_string)
hyperglass-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
hyperglass-1  | TypeError: expected str, got list

Pre 2.0 vrf usage wasn't impacted. I would consider this a major regression.

@szferguson szferguson mentioned this issue Jun 9, 2024
Open
@szferguson
Copy link

szferguson commented Jun 9, 2024 •
edited
Loading

Following up on this, we tried adding the __hyperglass_arista_eos_bgp_route_table__ attribute and referencing this directive on each device.

directives.yaml

test3:
  name: BGPtest
  table_output: __hyperglass_arista_eos_bgp_route_table__
  rules:
    - condition: '0.0.0.0/0'
      ge: 4
      le: 32
      command: 'show ip bgp {target} vrf INTERNET | json'

devices.yaml

  - name: AMSCR6
    ...
    directives:
       - test3
    structured_output: true

However, the only way for the table to appear was adding test3 in the directives tuple inside bgp_route_arista.py:

directives: t.Sequence[str] = (
    "__hyperglass_arista_eos_bgp_route_table__",
    "__hyperglass_arista_eos_bgp_aspath_table__",
    "__hyperglass_arista_eos_bgp_community_table__",
    "test3"
)

@jcramiresco
Copy link

I think adding support for VRF to version 2.x should be considered again. This is really important when you have devices with several VRFs configured.

@erikgadioli
Copy link

erikgadioli commented Aug 24, 2024 •
edited
Loading

Hyperglass is really awesome but indeed I do miss vrf support. As I only have one set of juniper peering routers and one service full table vrf I hacked into the source code.

sed -i 's/inet.0/YOURVRFNAME.inet.0/g' hyperglass/defaults/directives/juniper.py

and for ping and traceroute i added "routing-instance YOURVRFNAME" in between, like:
command="traceroute inet routing-instance YOURVRFNAME {target} wait 1 source {source4}",

now hyperglass is fully working for me

@33Fraise33
Copy link

I proposed the following, giving better support for VRFs or any multiple variable command actually:
#293

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@thatmattlove thatmattlove

Labels
feature New feature or request needs-info Issue lacks appropriate detail or requires additional information
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@szferguson @33Fraise33 @kmisak @thatmattlove @cameront-1 @erikgadioli @jcramiresco