packages in packages.json not installed in UI server container

[melissachang]

Problem overview

Unfortunately package.json in docker containers doesn't work as expected.

What should happen:

• A package is added to ui/package.json
• docker-compose up --build -t 0 builds new ui image which contains new package (new directory in node_modules). Running ui container can see new package.

Sometimes the above works, but sometimes the UI server can't see the new package:

Failed to compile
./src/components/facets/FacetHistogram.js
Module not found: Can't resolve 'vega' in '/ui/src/components/facets'

Repro problem
Here's how I can repro the problem. Willy, let me know if this doesn't work for you.

• In data-explorer on master, run docker-compose up --build -t 0
  (The last time I reproed, I was at commit 2e5c3b5 on master)
  Ctrl-C.
• Switch to branch mc/mc/copy-of-bc-visualizations
• docker-compose up --build -t 0
• UI server output has:

./src/components/facets/FacetHistogram.js
Module not found: Can't resolve 'vega' in '/ui/src/components/facets'

Problem background

In the ui container, we volume mount the github repo ui directory into the container. This is how we get UI server source code into container.

Then in the ui container, we run npm install. This installs packages into /ui/node_modules in the container. So container /ui should be mounted from host, except for /ui/node_modules.

For "mount everything from host except node_modules", most people add a node_modules line to docker-compose.yml. However, that sometimes causes new node_modules packages to not be found, presumably due to this race condition.

Potential solutions

The issue describes alternatives:

• Add -V: docker-compose up --build -t 0 -V. (Only needed for the next invocation of docker-compose up.) This does fix the problem. However, I'd like a solution that doesn't require adding a flag as a workaround.
• List specific files/directories in /ui. Downside is, if a new file/directory is added, you have to remember to add it to docker-compose.yml
• This feels like the best solution: Move node_modules outside of /ui. See here and here. You can see my attempt here. There's a problem which requires digging into the internals of create-react-app/react-scripts and node package installation. See here.