Skip to content

Commit

Permalink
Ensure the C++ impl of the triton stage only uses a single progress e…
Browse files Browse the repository at this point in the history
…ngine
  • Loading branch information
dagardner-nv committed Apr 18, 2024
1 parent e35748e commit ab3b144
Showing 1 changed file with 10 additions and 0 deletions.
10 changes: 10 additions & 0 deletions morpheus/stages/inference/triton_inference_stage.py
Original file line number Diff line number Diff line change
Expand Up @@ -781,3 +781,13 @@ def _get_cpp_inference_node(self, builder: mrc.Builder) -> mrc.SegmentObject:
self._needs_logits,
self._input_mapping,
self._output_mapping)

def _build_single(self, builder: mrc.Builder, input_node: mrc.SegmentObject) -> mrc.SegmentObject:
node = super()._build_single(builder, input_node)

# ensure that the C++ impl only uses a single progress engine
if (self._build_cpp_node()):
node.launch_options.pe_count = 1
node.launch_options.engines_per_pe = 1

return node

0 comments on commit ab3b144

Please sign in to comment.