UndefVarError when using with Distributed

[jonas-schulze]

How do I document/test examples that involve Distributed? Is it somehow possible to deactivate the sandboxing, such that @everywhere works as expected?

Example

Consider the following src/index.md:

```jldoctest demo1 ; output=false
using Distributed: @everywhere

@everywhere begin
    struct Foo end
end

Foo

# output

Foo
```

Building the docs yields this:

julia> makedocs(sitename="boom")
[...]
[ Info: Doctest: running doctests.
┌ Error: doctest failure in src/index.md:1-13
│
│ ```jldoctest demo1 ; output=false
│ using Distributed: @everywhere
│
│ @everywhere begin
│     struct Foo end
│ end
│
│ Foo
│
│ # output
│
│ Foo
│ ```
│
│ Subexpression:
│
│ using Distributed: @everywhere
│
│ @everywhere begin
│     struct Foo end
│ end
│
│ Foo
│
│ Evaluated output:
│
│ ERROR: UndefVarError: Foo not defined
│
│ Expected output:
│
│ Foo
│
│   diff = FooERROR: UndefVarError: Foo not defined
└ @ Documenter.DocTests src/index.md:1
[ Info: ExpandTemplates: expanding markdown templates.
[...]

[fredrikekre]

I believe @everywhere evaluates in Main, but doctests are run in their own module. Just to test this "theory", does it work if you reference Foo as Main.Foo after @everywhere?

[jonas-schulze]

Yes, Main.Foo does work. That's what I meant by "sandboxing":
https:/JuliaDocs/Documenter.jl/blob/c3c019a/src/DocTests.jl#L270

In the bigger picture, I would like to have a demo (that is included in the docs) tested by doctests. I didn't use @example blocks as I feared they may not reliably catch a broken demo. How would you approach this?

[fredrikekre]

Ah, you had already figured that out, didn't read the issue fully, sorry.

There is no way to turn it off, maybe you can wrap the block in

@eval Main begin #hide
    # ...
end #hide

but I have not tried this.

may not reliably catch a broken demo

The build fails if you have errors in @example blocks though?

[jonas-schulze]

There is no way to turn it off, maybe you can wrap the block in ...

This works partially. The code does not fail, but the output still is Main.Foo instead of Foo.
Also, the # hide trick does unfortunately not work for jldoctest.

The build fails if you have errors in @example blocks though?

Yes, the build fails if an error occurs, but what if I screwed up and the example succeeds but does no longer return what I explain in the surrounding documentation? Putting every output in a separate variable (which I could not # hide) and then replicating the checks of jldoctest (and hiding them) doesn't feel like a proper solution. I would therefore still prefer to use jldoctest.

---

I just encountered the next problem: when using an @example block, the build hangs for

```@example 1
using Distributed

addprocs(1)
```

but if I hit ^C it continues and reaches a reasonable output: the terminal logs

[ Info: ExpandTemplates: expanding markdown templates.
^C
signal (2): Interrupt: 2
in expression starting at /Users/jonas/Desktop/documenter-distributed/make.jl:3
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
kevent at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
Allocations: 24257033 (Pool: 24247533; Big: 9500); GC: 31
IOError: stream is closed or unusable
Stacktrace:
 [ Info: CrossReferences: building cross-references.
 [1][ Info: CheckDocument: running document checks.
 [ Info: Populate: populating indices.
check_open[ Info: RenderDocument: rendering document.

[ Info: HTMLWriter: rendering HTML pages.
    @ ./stream.jl%

and the docs then contain

1-element Vector{Int64}:
 2

as the result of that code example.

[ArnoStrouwen]

At SciML, we also have issues with the @everywhere macro.
We want to set up the following tutorial with an @example macro:
https://docs.sciml.ai/DiffEqDocs/dev/features/ensemble/#Example-1:-Solving-an-ODE-With-Different-Initial-Conditions,
SciML/DiffEqDocs.jl#627 (comment).

[fredrikekre]

One idea would be to shadow the @everywhere macro in a setup block which calls remotecall_eval with the right module instead of Main.

[fredrikekre]

Or for this specific example you could just shadow it with a no-op macro.

[jonas-schulze]

For my use case, performance is not critical and over-subscription is fine. I would much rather have the code properly executed and thus tested.