RFC: Make struct passing work properly

[Keno]

This is a WIP on the long standing issue of passing Julia structs to C. I have implemented the x86_64 ABI and am working on the others. This is passing the tests included in Jameson's original pull request, though as the ABI is rather complicated, I would appreciate more tests to include.

[Keno]

Now with 100% more ABI compliant cfunctions.
To keep track of issues (from comments) lost during the rebase:

• Need to add link to LDC
• Need to figure out what to do with fixed-size arrays.

[timholy]

Amazing!

[ViralBShah]

As always - awesome work @loladiro.

[Keno]

two down, one to go.

[timholy]

That's a ton of great work!

[timholy]

With regards to tests, do you mean "more examples of structs to pass to C libraries"? If it would help I can cull things from HDF5 (although do you want dependencies like that?).

[quinnj]

There are also some simple structs in ODBC that might be good for testing. I've been tracking this issue to implement these structs in my package anyway.

 typedef struct tagDATE_STRUCT {
    SQLSMALLINT year;
    SQLUSMALLINT month;
    SQLUSMALLINT day;
  } DATE_STRUCT;

  typedef struct tagTIME_STRUCT {
    SQLUSMALLINT hour;
    SQLUSMALLINT minute;
    SQLUSMALLINT second;
  } TIME_STRUCT;

  typedef struct tagTIMESTAMP_STRUCT {
    SQLSMALLINT year;
    SQLUSMALLINT month;
    SQLUSMALLINT day;
    SQLUSMALLINT hour;
    SQLUSMALLINT minute;
    SQLUSMALLINT second;
    SQLUINTEGER fraction;
  } TIMESTAMP_STRUCT;

[Keno]

More than the actual structs, I need examples of function signatures, where structs are being passed by value.

[quinnj]

Hmmm...yeah, ODBC wouldn't help much here.

[IainNZ]

SDL.jl?

[Keno]

I tried implementing win64, but LLVM seems to be somewhat broken in this case.

[Keno]

Done! :)

[IainNZ]

👍 Good stuff

[Keno]

Hmm interestingly this seems broken on Linux 64bit, but not on Mac 64bit (the parallel tests don't work). @JeffBezanson or @vtjnash are you aware of any ABI differences between Mac and Linux on 64bit?

[vtjnash]

I was discussing that with Jeff offline. Using the C convention internally gets everything (cfunction with structs, and redistributable sys.so image) faster, then we can think about optimizing it more later.

[Keno]

Fine with me.

[StefanKarpinski]

This completely fell through the cracks and should have been merged during the 0.3 cycle. See StackOverflow question about this: http://stackoverflow.com/questions/24870010/call-div-c-function-from-julia. I've encountered this occasionally myself – it would be really nice to be able to return structs.

[IainNZ]

0.3.1 milestone?

[StefanKarpinski]

Maybe. It's technically a breaking change though.

[IainNZ]

Ah in that case, 0.4. I was thinking of it as a purely-feature-adding thing.

[kmsquire]

I could also really use this. I've been working on wrapping libav/ffmpeg, which has a kajillion structs. My options seem to have been to pass pointers to immutables (which are then mutated by the C code), or use StrPack.

For now, I've gone with using immutables, as it has been the path of least resistance. But it feels rather sketchy to change immutable values (or even declare that a 100-member struct is immutable), so I'm looking forward to types having the right structure.

(I think I'll be able to publish a working version pretty soon.)

[vtjnash]

This is unrelated to immutables or to using pointers to types as structs

Once Keno finishes cleaning this up, or turns it over for someone else to finish, it could be merged.

[kmsquire]

This is unrelated to immutables or to using pointers to types as structs

Okay, I misinterpreted. This is for passing structs directly, then?

[vtjnash]

Correct, this is for correcting the calling-convention of passing structs by value.

[vtjnash]

Passing pointers to types works the same (and was implemented before) immutable types.

[kmsquire]

Passing pointers to types works the same (and was implemented before) immutable types.

Okay, thanks. I'm probably doing something wrong, then. Sorry for the noise, will post on julia-dev or a separate issue if I have further problems.

[zenna]

What is the latest on this?

[vtjnash]

Updated pr in #7906