[WIP] Adding CadQuery support

[jmwright]

Feel free to take this over and edit as much as needed. I can provide support on the CadQuery and cq-cli side of things. The cq-cli readme has several examples of how to use it, and example 6 shows how to use it with the stdin and stdout streams. I plan to add a Three.js plugin to cq-cli before the weekend, and the clone in the dockerfile should pick it up (at least with a clean build). I'll report back here when that is done.

Tagging @Irev-Dev and @franknoirot

Thanks!

[jmwright]

I forgot to post a link to CadQuery's SVG options.

https://cadquery.readthedocs.io/en/latest/importexport.html#exporting-svg

The one that's probably most interesting is the projectionDir vector to set the camera view.

[Irev-Dev]

Judging from this image

Is it safe to say that the "camera" is an orthogonal camera? (as opposed to perspective camera?)

Just check the CQ editor, and looks like it's orthogonal only too.

Cool, that actually simplifies things a little, but like what we discussed, we mightn't use the SVG output long term anyway.

[jmwright]

Is it safe to say that the "camera" is an orthogonal camera?

Correct, it should be orthogonal.

[jmwright]

cq-cli on the main branch now has the ability to export Three.js JSON text. I know this feature isn't necessary for the MVP, but it was a quick add, increases the usefulness of cq-cli, and gives you more options as you work through the integration. Seemed like an all-around win.

Example using the stdin and stdout streams:

echo "import cadquery as cq;result = cq.Workplane().box(10, 10, 10);show_object(result)" | ./cq-cli.py --codec threejs

Completes in 1.28 seconds and results in:

{
    "metadata" :
    {
        "formatVersion" : 3,
        "generatedBy"   : "ParametricParts",
        "vertices"      : 24,
        "faces"         : 12,
        "normals"       : 0,
        "colors"        : 0,
        "uvs"           : 0,
        "materials"     : 1,
        "morphTargets"  : 0
    },

    "scale" : 1.0,

    "materials": [    {
    "DbgColor" : 15658734,
    "DbgIndex" : 0,
    "DbgName" : "Material",
    "colorAmbient" : [0.0, 0.0, 0.0],
    "colorDiffuse" : [0.6400000190734865, 0.10179081114814892, 0.126246120426746],
    "colorSpecular" : [0.5, 0.5, 0.5],
    "shading" : "Lambert",
    "specularCoef" : 50,
    "transparency" : 1.0,
    "vertexColors" : false
    }],

    "vertices": [-5.0, -5.0, -5.0, -5.0, -5.0, 5.0, -5.0, 5.0, -5.0, -5.0, 5.0, 5.0, 5.0, -5.0, -5.0, 5.0, -5.0, 5.0, 5.0, 5.0, -5.0, 5.0, 5.0, 5.0, -5.0, -5.0, -5.0, 5.0, -5.0, -5.0, -5.0, -5.0, 5.0, 5.0, -5.0, 5.0, -5.0, 5.0, -5.0, 5.0, 5.0, -5.0, -5.0, 5.0, 5.0, 5.0, 5.0, 5.0, -5.0, -5.0, -5.0, -5.0, 5.0, -5.0, 5.0, -5.0, -5.0, 5.0, 5.0, -5.0, -5.0, -5.0, 5.0, -5.0, 5.0, 5.0, 5.0, -5.0, 5.0, 5.0, 5.0, 5.0],

    "morphTargets": [],

    "normals": [],

    "colors": [],

    "uvs": [[]],

    "faces": [0, 1, 2, 0, 0, 1, 3, 2, 0, 5, 4, 6, 0, 5, 6, 7, 0, 11, 8, 9, 0, 11, 10, 8, 0, 15, 13, 12, 0, 15, 12, 14, 0, 19, 16, 17, 0, 19, 18, 16, 0, 23, 21, 20, 0, 23, 20, 22]
}

Tessellation (and thus execution) time will ramp up with the complexity of the object. Tessellator improvements will hopefully be coming in the future that will improve speeds. I know that it's probably a non-starter because you're using AWS Lambdas, but if persistent storage was ever an option, there is a CadQuery cache decorator plugin that is being worked on currently. That would make the render times more consistent and fast, only being slower when there is a change in the script or its parameters.

In any case, we can hold off on the performance discussion until after the MVP is working and we start to see bottlenecks. Just being able to share basic CadQuery scripts quickly and easily through CadHub will be a huge win.

[Irev-Dev]

Awesome @jmwright 🙌 ,

It's super hacky but I've got an end to end example working atm.

Screen.Recording.2021-04-16.at.7.35.06.am.mov

With the three.js working I'll probably try and move to that straight away then.

The above does seems pretty slow even with it being local, and if it's going to get slower with tessellation I'm definitely open to other better optimised ways of deploying it. at the end of the day I'm more of a frontend dev so I'm drawn to the not having to worry about scaling part of lamdas (don't want to let my comfort get in the way of user experience), but yeah lets keep an open dialog about this as we iterate.

[jmwright]

@Irev-Dev I just did a quick comparison using this involute gear example from the CadQuery contrib repo to try to get more of a real world measurement. Below are the results for SVG vs Three.js.

SVG Time Required

$ time cat ~/Downloads/Involute_Gear.py | ./cq-cli.py --codec svg

real    0m3.158s
user    0m3.008s
sys     0m0.324s

Three.js Time Required

$ time cat ~/Downloads/Involute_Gear.py | ./cq-cli.py --codec threejs

real    0m2.721s
user    0m2.583s
sys     0m0.295s

So at least for single objects both methods are slow, but not horrifically so. About the fastest SVG or Three.js will render a simple box is 1.2 seconds, and most of that is the time it takes Python to spin up and import the OCP package, which is large and pulls the OCCT CAD kernel dependency with it.

I specified a static image for the designs I put in CadHub. Is the intention to show that most of the time, but then switch to the interactive 3D view only when the user wants to (or wants to edit the script/parameters), or do you always want the images and 3D views to be generated directly from source?

[adam-urbanczyk]

What part of this is export to file? Would it help if OCCT/CQ supported exporting to VTK format and this was used for rendering https://kitware.github.io/itk-vtk-viewer/docs/embeddedViewer.html ?

[Irev-Dev]

Thanks for that comparison @jmwright, this definitely makes me think that a 3d output is a better option if they are comparable speed, since having to wait after a code change is once thing, but then being able to drag and rotate a 3d output is a big usability win.

@adam-urbanczyk, the threejs output is perfect since we're already using three we're only using for the axis for the openscad integration atm, so we could switch, but I'm pretty confident the project will suit our needs.

[adam-urbanczyk]

The threejs json generation in CQ is extremely slow. In the future we'll have gltf and vtk directly generated from OCCT. I think both can be loaded into threejs,

[Irev-Dev]

Okay, sweet @adam-urbanczyk . GLTF sounds good. From the three.js docs:

Where possible, we recommend using glTF

[Irev-Dev]

Just as an FYI, I've gotten a little stuck with the three.js stuff, I'm getting the threejs output fine from the cli, but am struggling to pass it into three. I'm not entirely sure what three function I should be using for the task so I've asked on the forums.
https://discourse.threejs.org/t/issues-loading-json/25437

[jmwright]

Here's some Three.js JSON loader code from an Electron-based CadQuery GUI that I created years ago. https:/jmwright/cadquery-gui/blob/master/assets/js/viewer.js#L109

[Irev-Dev]

Here's some Three.js JSON loader code from an Electron-based CadQuery GUI that I created years ago. https:/jmwright/cadquery-gui/blob/master/assets/js/viewer.js#L109

Thanks for that, They've deprecated the JSONLoader.
https:/mrdoob/three.js/blob/354f955c835b880c65639b386c6f287c9bdb4a2c/src/Three.Legacy.js#L1882
When googling for this there are heaps of examples using it, so I think the ObjectLoader is what I'm supposed to be using now, but not sure.

[Irev-Dev]

As an update. I've been making decent progress, while the UI need attention, (Switching between cadquery and openscad while keeping the code there makes little sense, also the THREE scene looks terrible, I need to change the lighting or materials or something).

Screen.Recording.2021-04-18.at.5.49.34.pm.mov

But I'm happy with it so far, in terms of the two packages playing nice together.

I was ready to see if the deployment would work and alas I've hit an issue, I get "conda, permission denied" when deployed. It's only just happened so I haven't really had a chance to work through it yet. It's unlikely I'll get much done over the next two days as I'll be away from home with bad internet.

Work so far is here.

Oh and I ended up using stl as it seems format version 3 for JSON is deprecated, and at this point I don't care much about which format, so much as to getting it working.
https://discourse.threejs.org/t/issues-loading-json/25437/2

[jmwright]

@Irev-Dev If you want, I can create a Dockerfile that uses the binary distribution of cq-cli for Linux. It might have a longer startup time, and I can't remember what distro/ABI the lambdas run, so it may not be compatible. We could try it though.

[jmwright]

@Irev-Dev I pushed a change to the Dockerfile that uses the release package of cq-cli instead if you want to try that. It doesn't include Three.js export since you have to be logged into GitHub to get the development builds, but it sounds like you don't need that functionality right now anyway.

I think it's likely that using the release package could lead to slower spin-up (and thus processing) times for your lambdas because of how pyinstaller packages things, but I have no data to back that up. Maybe we'll get lucky and it won't be any worse for this use case. The Docker image does end up being about half the size when using the release package, which is really nice.

[Irev-Dev]

Great thanks @jmwright, that seems to have side stepped the permissions issue I was having.

I think spin up time is faster, at least locally it seems to be the case and I don't see why that would be different when deployed?

I got some clean up to do, and I want to figure out staging aws stuff, so that I get get a preview link working.

[Irev-Dev]

I have a preview link!

[jmwright]

That's great! Thanks for all the work in this. I'm only on mobile right now, but it worked fine.

[Irev-Dev]

Oh cool, I'm too scared to see what the website looks like on mobile.

[jmwright]

The layout certainly wasn't optimized for mobile, but I was able to tell what was going on, click the Render button, and the model appeared after about 5 seconds.

[jmwright]

I think what you've been able to accomplish in such a short time is fantastic. It looks great for an MVP.

Below are two things that I've noticed so far on Firefox 87.0 on Ubuntu 18.04. All I did was click on the link you posted and then click Render. I didn't switch to OpenSCAD mode even though the console mentions OpenSCAD.

1. The first time I clicked Render it timed out and I saw a network error. I assumed there was just something odd going on with the lambda, so I tried again. Clicking Render a second time worked and I saw the rendered shaft coupler.
   

2. I see this error in the web console as well.
   

[Irev-Dev]

Yup, 1 and 2 are one and the same issue, in that it's a timeout issue and when it doesn't respond it surfaces as a cors issue which I'm not sure why. The biggest problem here is the cold start of the docker-lamdas, I have the timeout set to 30s but I would have assumed that the timeout wouldn't include aws's own slow startup time but 🤷 , I think I might as well just crank that up to 45s.
Another option which I haven't properly considered is to use something like this to keep the lamdas warm
https:/juanjoDiaz/serverless-plugin-warmup
Rest assured I definitely want to make it robust.

"initialising OpenSCAD" is just a hardcoded message when the IDE starts, so easily fixed lol.

I've started trying random examples from the docs,

It's been fun, but a good way to find limits, and looks like I hit one with the parametric enclosure, it won't return anything and in the logs I get Request entity too large I think this is likely my fault related to the way I'm encoding the result. Things to work through for sure.

[jmwright]

Keeping the lambdas/containers warm does seem like a sensible way to try to solve the issue.

[Irev-Dev]

Closing this as is superseded by #281, (I don't think I'm able to push extra commits onto different fork??, either way I kept your initial commit @jmwright )

[jmwright]

I made it so that you could push to my fork's PR branch, but the end result is what matters, not whether or not this PR got merged. I'm happy that this PR helped get the desired result in #281

[Irev-Dev]

Ahh okay. I'll make sure to figure that out next time.
I don't have tonnes of experience with github. All the dev companies I've worked at (only 2) use Bitbucket (It's an Australian thing).

[Irev-Dev]

🤦