PyInstaller build spec + JSON RPC interface for client

[pranavgaikwad]

This PR adds a JSON RPC interface in front of the client CLI. It adds a new build spec to package the JSON RPC server into a binary. It also provides some example usage of the client via a JSON RPC client written in Python and Javascript.

Before merging this PR, I would like to finish:

• Adding new section to PlayPen README detailing process to build the binary and run sample clients
• Fixed issue with VCR.

I did NOT add PyInstaller as a dep to the project yet, I think we can do it later when we finalize the approach

[dymurray]

fixes #336

[jwmatthews]

@pranavgaikwad I tried running on MacOS arm64, below is what I saw.
Looked like I built the binary OK, didn't see any errors, but when I run I see an error.

$ cd dist
$ ls -larth
total 16160
drwxr-xr-x  11 jmatthews  staff   352B Aug 31 08:08 ../
-rwxr-xr-x   1 jmatthews  staff   7.9M Aug 31 08:08 cli*
drwxr-xr-x   3 jmatthews  staff    96B Aug 31 08:08 ./


$ ./cli 
Traceback (most recent call last):
  File "rpc.py", line 14, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 378, in exec_module
  File "cli.py", line 34, in <module>
ModuleNotFoundError: No module named 'kai'
[PYI-56524:ERROR] Failed to execute script 'rpc' due to unhandled exception!

 $ python rpc-client.py ../kai/config.toml coolstore ../example/analysis/coolstore/output.yaml ../example/coolstore/src/main/java/com/redhat/coolstore/service/ShoppingCartOrderProcessor.java
Traceback (most recent call last):
  File "/Users/jmatthews/git/jwmatthews/kai/playpen/rpc-client.py", line 10, in <module>
    from client import rpc as kaiRpcClient
  File "/Users/jmatthews/git/jwmatthews/kai/playpen/client/rpc.py", line 14, in <module>
    from cli import (
  File "/Users/jmatthews/git/jwmatthews/kai/playpen/client/cli.py", line 34, in <module>
    from kai.constants import PATH_KAI
ModuleNotFoundError: No module named 'kai'

More output here from what I tried: https://gist.github.com/jwmatthews/3bf9d5bce931d1cd695e84ec987881e3

[pranavgaikwad]

@jwmatthews did you have Kai installed using 'pip install -e .'

[jwmatthews]

I don't believe I had kai installed in the venv I was running in, so tried to correct that in below

$ pip install -e .
Obtaining file:///Users/jmatthews/git/jwmatthews/kai
  Preparing metadata (setup.py) ... done
Installing collected packages: kai
  DEPRECATION: Legacy editable install of kai==0.0.1 from file:///Users/jmatthews/git/jwmatthews/kai (setup.py develop) is deprecated. pip 25.0 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at https:/pypa/pip/issues/11457
  Running setup.py develop for kai
Successfully installed kai-0.0.1

$ python rpc-client.py ../kai/config.toml coolstore ../example/analysis/coolstore/output.yaml ../example/coolstore/src/main/java/com/redhat/coolstore/service/ShoppingCartOrderProcessor.java
running get_incident_solutions_for_file() with params {'config_path': '../kai/config.toml', 'app_name': 'coolstore', 'report_path': '../example/analysis/coolstore/output.yaml', 'input_file_path': '../example/coolstore/src/main/java/com/redhat/coolstore/service/ShoppingCartOrderProcessor.java'}
Traceback (most recent call last):
  File "rpc.py", line 14, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 378, in exec_module
  File "cli.py", line 34, in <module>
ModuleNotFoundError: No module named 'kai'
[PYI-67545:ERROR] Failed to execute script 'rpc' due to unhandled exception!

$ python 
Python 3.12.2 (main, Feb  6 2024, 20:19:44) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.path)
['', '/opt/homebrew/Cellar/[email protected]/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python312.zip', '/opt/homebrew/Cellar/[email protected]/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12', '/opt/homebrew/Cellar/[email protected]/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload', '/Users/jmatthews/git/jwmatthews/kai/env/lib/python3.12/site-packages', '/Users/jmatthews/git/jwmatthews/kai']
>>> exit()

Is the expectation that with the binary from pyinstaller at runtime it will need the python code accessible in PYTHON_PATH?

[pranavgaikwad]

@jwmatthews no all of the deps get bundled into the binary along with the python interpreter itself with all of the paths baked in. You as a user are not expected to do anything, the binary should just work.

[jwmatthews]

@pranavgaikwad thank you, got this working.

The issue was that prior to building the binary I did NOT have the python path configured with 'kai'.
So I had to add Kai to the to the python path: pip install -e . in the main project folder, then rebuild the binary

full output
https://gist.github.com/jwmatthews/e7813daa5f1d5e1f2dd63ccde60edb2d

$ ls -larth dist/cli 
-rwxr-xr-x  1 jmatthews  staff    79M Aug 31 10:11 dist/cli*


(env) [jmatthews] in ~/git/jwmatthews/kai/playpen (remotes/pranavgaikwad/kaiClient) $ python rpc-client.py ../kai/config.toml coolstore ../example/analysis/coolstore/output.yaml ../example/coolstore/src/main/java/com/redhat/coolstore/service/ShoppingCartOrderProcessor.java
running get_incident_solutions_for_file() with params {'config_path': '../kai/config.toml', 'app_name': 'coolstore', 'report_path': '../example/analysis/coolstore/output.yaml', 'input_file_path': '../example/coolstore/src/main/java/com/redhat/coolstore/service/ShoppingCartOrderProcessor.java'}
package com.redhat.coolstore.service;

import java.util.logging.Logger;
import jakarta.annotation.Resource;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;
import jakarta.jms.JMSContext;
import io.smallrye.mutiny.Multi;
import io.smallrye.reactive.messaging.Channel;
import io.smallrye.reactive.messaging.Emitter;
import com.redhat.coolstore.model.ShoppingCart;
import com.redhat.coolstore.utils.Transformers;

@ApplicationScoped
public class ShoppingCartOrderProcessor  {

    @Inject
    Logger log;

    @Inject
    private transient JMSContext context;

    @Resource(lookup = "java:/topic/orders")
    @Channel("orders")
    Emitter<String> ordersEmitter;

    public void  process(ShoppingCart cart) {
        log.info("Sending order from processor: ");
        Multi.createFrom().item(Transformers.shoppingCartToJson(cart))
            .subscribe().with(ordersEmitter::send);
    }

}


Received response successfully!

[jwmatthews]

@pranavgaikwad this is awesome!

Verified able to run on MacOS arm64