types as C-structs [rebased] #10579

Keno · 2015-03-19T18:42:43Z

@JeffBezanson and I are in the process of looking at and hopefully merging #2818. This rebases that PR (with some unrelated changes already cherry-picked to master) hopefully to be merged soon.

this makes references to types allocated in Julia recursively forward compatible with C structs

rename jl_typetag_t => jl_taggedvalue_t rename jl_typetagof => jl_astaggedvalue

types as C-structs [rebased]

kmsquire · 2015-03-20T00:34:05Z

Does this deserve a NEWS entry?

kmsquire · 2015-03-20T01:26:27Z

Also need to remove the reference to #2818 from the "Calling C and Fortran" docs.

tkelman · 2015-03-20T13:41:11Z

This utterly broke the windows build. Can we please let CI do its job?

StefanKarpinski · 2015-03-20T13:45:46Z

True – there's not much point in paying AppVeyor for CI as a service if we're going to ignore it.

tkelman · 2015-03-20T16:14:25Z

@vtjnash or @Keno, how do we deal with this

$ usr/bin/julia-debug
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.4.0-dev+3939 (2015-03-20 12:38 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit b43f0b0* (0 days old master)
|__/                   |  x86_64-w64-mingw32


Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0xe08aab -- jl_setjmp at D:\cygwin64\home\Tony\julia\usr\bin\libjulia-debug.dll (unknown line)
jl_setjmp at D:\cygwin64\home\Tony\julia\usr\bin\libjulia-debug.dll (unknown line)
julia_init at D:\cygwin64\home\Tony\julia\usr\bin\libjulia-debug.dll (unknown line)
jl_f_yieldto at D:\cygwin64\home\Tony\julia\usr\bin\libjulia-debug.dll (unknown line)
wait at task.jl:305
stream_wait at stream.jl:271
uv_write at stream.jl:751
buffer_or_write at stream.jl:764
write at stream.jl:826
print at string.jl:68
jlcall_print_664 at  (unknown line)
jl_operator_precedence at D:\cygwin64\home\Tony\julia\usr\bin\libjulia-debug.dll (unknown line)
jl_trampoline at D:\cygwin64\home\Tony\julia\usr\bin\libjulia-debug.dll (unknown line)
jl_init_types at D:\cygwin64\home\Tony\julia\usr\bin\libjulia-debug.dll (unknown line)
jl_apply_generic at D:\cygwin64\home\Tony\julia\usr\bin\libjulia-debug.dll (unknown line)
banner at version.jl:232
_start at client.jl:415
jlcall__start_589 at  (unknown line)
jl_operator_precedence at D:\cygwin64\home\Tony\julia\usr\bin\libjulia-debug.dll (unknown line)
jl_trampoline at D:\cygwin64\home\Tony\julia\usr\bin\libjulia-debug.dll (unknown line)
jl_init_types at D:\cygwin64\home\Tony\julia\usr\bin\libjulia-debug.dll (unknown line)
jl_apply_generic at D:\cygwin64\home\Tony\julia\usr\bin\libjulia-debug.dll (unknown line)
unknown function (ip: 4199740)
unknown function (ip: 4203441)
unknown function (ip: 4204232)
unknown function (ip: 4199370)
unknown function (ip: 4199672)
BaseThreadInitThunk at C:\Windows\system32\kernel32.dll (unknown line)
RtlUserThreadStart at C:\Windows\SYSTEM32\ntdll.dll (unknown line)

Tony@TK-samsung ~/julia
$ gdb -q usr/bin/julia-debug
Reading symbols from usr/bin/julia-debug...done.
(gdb) r
Starting program: /home/Tony/julia/usr/bin/julia-debug
[New Thread 9584.0x1e30]
[New Thread 9584.0x11bc]
[New Thread 9584.0x3180]
[New Thread 9584.0x5d34]
[New Thread 9584.0x47d4]
[New Thread 9584.0x2440]
[New Thread 9584.0x4c28]
[New Thread 9584.0x4444]
[New Thread 9584.0x56d4]
[New Thread 9584.0x5b84]
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.4.0-dev+3939 (2015-03-20 12:38 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit b43f0b0* (0 days old master)
|__/                   |  x86_64-w64-mingw32

[New Thread 9584.0x2b34]

Program received signal SIGSEGV, Segmentation fault.
jl_setjmp () at _setjmp.win64.S:23
23          movaps XMMWORD PTR [rcx+96],xmm6
(gdb) bt
#0  jl_setjmp () at _setjmp.win64.S:23
#1  0x0000000000dc6ce0 in ctx_switch (t=0x8170f0c8, where=0x8170f118) at task.c:296
#2  0x0000000000dc6ebb in switchto (t=0x8170f0c8) at task.c:371
#3  0x0000000000dc7dd4 in jl_f_yieldto (F=0x80020728, args=0xd5ece0, nargs=2) at task.c:887
#4  0x0000000010aa799e in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

vtjnash · 2015-03-20T17:27:31Z

Ah, I was considering fixing that, but couldn't come up with a test case originally.

On my phone now, but if someone wants to take a shot at this -- every pool we allocate in the gc should be shifted by (sizeof(jl_typetag_t)%16 bytes) before slicing it up

tkelman · 2015-03-20T17:29:33Z

AppVeyor's also idle right now so if someone can translate that into a patch it shouldn't take long to check.

JeffBezanson · 2015-03-20T17:32:15Z

@StefanKarpinski is it possible to change some permissions to let more people cancel appveyor builds? I don't think I can.

tkelman · 2015-03-20T17:33:21Z

@JeffBezanson you have to log out then log back in as StefanKarpinski. Everyone with commit access should be allowed to do that.

tkelman · 2015-03-20T17:38:06Z

It's a really odd UI (cc @FeodorFitsner this should probably be made simpler / more obvious) - you log in with your own Github credentials, but then you should see a screen that looks like this:

You have to select the StefanKarpinski entry there to be able to cancel / restart Julia builds.

Keno · 2015-03-20T17:52:52Z

I don't have that option, I presume because my GitHub account was renamed and AppVeyor got confused.

FeodorFitsner · 2015-03-20T18:45:37Z

Sure, this will be changed though it's non-trivial change (API is affected) that requires careful planning. Perhaps, will be a good chance to re-write UI on Angular 2.0 and TypeScript :)

tkelman · 2015-03-21T08:22:23Z

So, aside from the MinGW failure which I was hoping someone could translate from Jameson English into commit-worthy code (I can't...), there's also this: https://msdn.microsoft.com/en-us/library/0scy7z2d.aspx

vtjnash · 2015-03-21T18:38:59Z

there's also this: https://msdn.microsoft.com/en-us/library/0scy7z2d.aspx

i assume you mean that issue comes up when you try to do a msvc compile? you could probably just make this macro define empty:

julia/src/julia.h

Line 60 in 0d8cec3

struct _jl_value_t *fieldptr0[0];

it seems that may be a C99 feature

tkelman · 2015-03-22T05:18:36Z

Will try that. Thanks for fixing the alignment issue.

tkelman · 2015-03-22T06:51:46Z

That helps a bit. How about this line

julia/src/julia.h

Line 86 in efb878e

jl_value_t value[0];

giving https://msdn.microsoft.com/en-us/library/81zca88z.aspx ?

I've also been trying with the VS 2015 rc's that are available on AppVeyor, and it doesn't help fix these.

tkelman · 2015-03-22T16:11:16Z

I don't think the size-zero array is a C99 thing, it looks from https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html like it's a GNU extension and the MSVC error is strictly speaking true according to the C standard.

Someone should check whether ICC allows this construct.

nalimilan · 2015-03-22T17:14:47Z

If the intent is to have a variable-length array, then you should be able to replace value[0] with value[], right? That's how it's done here:

julia/src/gc.c

Line 49 in e1f0310

char data[];

tkelman · 2015-03-22T19:15:48Z

No, that doesn't work https://ci.appveyor.com/project/tkelman/julia/build/1.0.644

The other struct types with C99 flexible array members aren't causing problems because there don't appear to be any arrays of them anywhere. value is an array of jl_value_t, which violates "A structure containing a flexible array member, or a union containing such a structure (possibly recursively), may not be a member of a structure or an element of an array." - since jl_value_t itself has a flexible array member in it for fieldptr.

nalimilan · 2015-03-22T20:27:47Z

Ah, got it, that's the recursive nature of these that's the problem.

Ref #227

Ref JuliaIO#227

JeffBezanson mentioned this pull request Mar 19, 2015

types as C-structs #2818

Closed

5 tasks

JeffBezanson force-pushed the jbkf/hidetypetag branch from 5239666 to 6792f06 Compare March 19, 2015 19:54

vtjnash and others added 2 commits March 19, 2015 20:16

hide the julia type pointer from the struct jl_value_t definition

65cfde9

this makes references to types allocated in Julia recursively forward compatible with C structs

some renaming and fix overloads changed by merge

cbf9da5

rename jl_typetag_t => jl_taggedvalue_t rename jl_typetagof => jl_astaggedvalue

JeffBezanson force-pushed the jbkf/hidetypetag branch from 6792f06 to cbf9da5 Compare March 20, 2015 00:16

JeffBezanson added a commit that referenced this pull request Mar 20, 2015

Merge pull request #10579 from JuliaLang/jbkf/hidetypetag

175d732

types as C-structs [rebased]

JeffBezanson merged commit 175d732 into master Mar 20, 2015

JeffBezanson added a commit that referenced this pull request Mar 20, 2015

fix a mistake in merging changes from #10528 and #10579

1de119e

tkelman mentioned this pull request Mar 20, 2015

Julia Nightly build segfault at startup on windows #10591

Closed

tkelman deleted the jbkf/hidetypetag branch March 22, 2015 05:18

tkelman mentioned this pull request Mar 23, 2015

WIP: more MSVC compatibility #10609

Merged

simonster added a commit to JuliaIO/HDF5.jl that referenced this pull request Apr 2, 2015

Fix reading non-pointerfree immutables since JuliaLang/julia#10579

5b60a15

Ref #227

tkelman mentioned this pull request Apr 19, 2015

remove extra word from object header (better approach to alignment) #10898

Closed

simonster added a commit to JuliaIO/HDF5.jl that referenced this pull request Apr 24, 2015

Fix reading non-pointerfree immutables since JuliaLang/julia#10579

4466f27

Ref #227

tkelman mentioned this pull request Jul 10, 2015

Add tests for base/abstractarray.jl #12086

Merged

kmsquire mentioned this pull request Aug 29, 2015

RFC: Update calling-c-and-fortran docs #10799

Closed

DSLituiev pushed a commit to DSLituiev/JLD.jl that referenced this pull request Oct 19, 2015

Fix reading non-pointerfree immutables since JuliaLang/julia#10579

72d3716

Ref JuliaIO#227

staticfloat mentioned this pull request May 1, 2016

Remove mention of issue #2818 from 0.4 manual #16156

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

types as C-structs [rebased] #10579

types as C-structs [rebased] #10579

Keno commented Mar 19, 2015

kmsquire commented Mar 20, 2015

kmsquire commented Mar 20, 2015

tkelman commented Mar 20, 2015

StefanKarpinski commented Mar 20, 2015

tkelman commented Mar 20, 2015

vtjnash commented Mar 20, 2015

tkelman commented Mar 20, 2015

JeffBezanson commented Mar 20, 2015

tkelman commented Mar 20, 2015

tkelman commented Mar 20, 2015

Keno commented Mar 20, 2015

FeodorFitsner commented Mar 20, 2015

tkelman commented Mar 21, 2015

vtjnash commented Mar 21, 2015

tkelman commented Mar 22, 2015

tkelman commented Mar 22, 2015

tkelman commented Mar 22, 2015

nalimilan commented Mar 22, 2015

tkelman commented Mar 22, 2015

nalimilan commented Mar 22, 2015

types as C-structs [rebased] #10579

types as C-structs [rebased] #10579

Conversation

Keno commented Mar 19, 2015

kmsquire commented Mar 20, 2015

kmsquire commented Mar 20, 2015

tkelman commented Mar 20, 2015

StefanKarpinski commented Mar 20, 2015

tkelman commented Mar 20, 2015

vtjnash commented Mar 20, 2015

tkelman commented Mar 20, 2015

JeffBezanson commented Mar 20, 2015

tkelman commented Mar 20, 2015

tkelman commented Mar 20, 2015

Keno commented Mar 20, 2015

FeodorFitsner commented Mar 20, 2015

tkelman commented Mar 21, 2015

vtjnash commented Mar 21, 2015

tkelman commented Mar 22, 2015

tkelman commented Mar 22, 2015

tkelman commented Mar 22, 2015

nalimilan commented Mar 22, 2015

tkelman commented Mar 22, 2015

nalimilan commented Mar 22, 2015