Julia - Transferring methods between workers -
i've been using sendto() method send variables between workers
i under impression in julia, functions 'first class citizens', cannot transfer functions between workers using sendto() method
what recommended way of transferring predefined methods between workers?
update
i seem able use sendto() method send anonymous functions
a = function(x) x*2 end sendto(2,a=a) remotecall_fetch(2,a,5) 4
as opposed to
function g(x) x*2 end sendto(2,g=g) remotecall_fetch(2,g,10) error: on worker 2: function g not defined on process 2
this suffices in cases, need able send defined functions over, of them complex , defined in scripts included when starting julia
there no way send subset of methods in package machine. methods refer other types , functions in same module, system have @ least send dependencies well. work, bigger problem deciding responsibility distribute code, , when. example, library might decide send (or parts of itself) other nodes, user might later want parallel map of library functions, such whole library needed on every node. gets complex, far simpler load needed code on nodes possible.
it factor code smaller modules. code defines reusable functions should wrapped in module separate "real work", loading not take significant time.
if code loaded on client node needs different what's loaded on workers, can put definitions need sent in own module, helpers
. main code contain using helpers
. after code loaded on client node, can start workers , run @everywhere include("helpers.jl")
suggested chris. references functions inside helpers
should work across machines.
Comments
Post a Comment