Commits on May 21, 2024

1. Attach to cluster controller through Networking …

   This PR includes:
   - Attachment of PPM to cluster controller now use Networking
   - PPM observes partition processor status through a buffered watch mechanism
   - PPM can now send control messages (unused at this PR) to processor for future use.
   - PPM collects state information from running processors. This will be used in a follow PR to response to controller requests about partitions.
   
   
   Cluster controller grpc service is kept for external tooling integration (CLI, etc.)

   

   AhmedSoliman committed May 21, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for 03418f4
   • Browse repository at this point

   Copy the full SHA

   03418f4 View commit details

   Browse the repository at this point in the history

2. Partial progress on ClusterState …

   ClusterState is a new data structure maintained by ClusterController. ClusterState is established by a scatter-gather request (GetProcessorsState) to all workers. For every worker, partition processor manager is the entity responsible for responding to GetProcessorsState requests, it keeps taps on running partition processors via a set of tokio watches.
   
   This PR is a snapshot in time of progress done on this component, this is far from complete but it's best to merge the current progress until we circle back to it in a week or two.

   

   AhmedSoliman committed May 21, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for 9f28b58
   • Browse repository at this point

   Copy the full SHA

   9f28b58 View commit details

   Browse the repository at this point in the history