r - How to log when using foreach (print or futile.logger) -

-

i want use foreach package in conjunction logging. use futile.logger package. when work given workers logging information lost (which strange need indicate foreach logging package)

i've seen this post not use foreach

  library(foreach)                                                                                                                                                                                                                                                                                                          library(futile.logger)                                                                                                                                                                                                                                                                                                    library(doparallel)                                                                                                                                                                                                                                                                                                       flog.threshold(debug)                                                                                                                                                                                                                                                                                                     cluster <- makecluster(8)   registerdoparallel(cluster)   dostuff <- function(input){                                                                                                                                                                                                                                                                                                 flog.debug('doing stuff %s', input)                                                                                                                                                                                                                                                                           return(input)                                                                                                                                                                                                                                                                                                         }                                                                                                                                                                                                                                                                                                                         res <- lapply(fun=dostuff, x=seq(1,8,1))   # >> prints                                                                                                                                                                                                                                                                            res2 <- foreach(input = seq(1,8,1)) %do% dostuff(input)                                                                                                                                                                                                                                                                   # >> prints   res3 <- foreach(input = seq(1,8,1), .packages='futile.logger') %dopar% dostuff(input)           # >> not                                                                                                                                                                                                                             identical(res,res2) && identical(res,res3)

i not care parallele backend, can anything, how can symply logging working

following solution how can print when using %dopar%: idea use snow set cluster, , set outfile="" redirect worker output master.

library(foreach) library(futile.logger) library(doparallel)  library(dosnow) cluster <- makecluster(3, outfile="") # have 4 cores, 8 registerdosnow(cluster) flog.threshold(debug)  dostuff <- function(input){   flog.info('doing stuff %s', input) # change flog.info   return(input)    }  res <- lapply(fun=dostuff, x=seq(1,8,1)) # >> prints                                                               res2 <- foreach(input = seq(1,8,1)) %do% dostuff(input)  # >> prints res3 <- foreach(input = seq(1,8,1), .packages='futile.logger') %dopar% dostuff(input)   # >> prints

output:

> res3 <- foreach(input = seq(1,8,1), .packages='futile.logger') %dopar% dostuff(input)   type: exec  type: exec  type: exec  type: exec  type: exec  type: exec  info [2016-08-08 08:22:39] doing stuff 3 type: exec  info [2016-08-08 08:22:39] doing stuff 1 info [2016-08-08 08:22:39] doing stuff 2 type: exec  type: exec  info [2016-08-08 08:22:39] doing stuff 5 info [2016-08-08 08:22:39] doing stuff 4 type: exec  type: exec  info [2016-08-08 08:22:39] doing stuff 6 info [2016-08-08 08:22:39] doing stuff 7 info [2016-08-08 08:22:39] doing stuff 8

output log file. here's alternative outputs log file, following how log using futile logger within parallel method in r?. has advantage of having cleaner output, still requires flog.info:

library(dosnow) library(foreach) library(futile.logger) nworkers <- 3 cluster <- makecluster(nworkers) registerdosnow(cluster) loginit <- function(logfile) flog.appender(appender.file(logfile)) foreach(input=rep('~/desktop/out.log', nworkers),    .packages='futile.logger') %dopar% loginit(input) dostuff <- function(input){   flog.info('doing stuff %s', input)   return(input)    }  foreach(input = seq(1,8,1), .packages='futile.logger') %dopar% dostuff(input)  stopcluster(cluster) readlines("~/desktop/out.log")

output:

> readlines("~/desktop/out.log") [1] "info [2016-08-08 10:07:30] doing stuff 2" [2] "info [2016-08-08 10:07:30] doing stuff 1" [3] "info [2016-08-08 10:07:30] doing stuff 3" [4] "info [2016-08-08 10:07:30] doing stuff 4" [5] "info [2016-08-08 10:07:30] doing stuff 5" [6] "info [2016-08-08 10:07:30] doing stuff 6" [7] "info [2016-08-08 10:07:30] doing stuff 7" [8] "info [2016-08-08 10:07:30] doing stuff 8"

Comments

Post a Comment

Popular posts from this blog

Spring Boot + JPA + Hibernate: Unable to locate persister -

-
i trying work entity inside request method of spring boot application. entity in project , pom.xml has dependencies it. why error message?: org.hibernate.unknownentitytypeexception: unable locate persister: com.ric.bill.model.bs.par package com.ric.web; import java.util.concurrent.atomic.atomiclong; import javax.persistence.entitymanager; import javax.persistence.persistencecontext; import net.sf.ehcache.cachemanager; import net.sf.ehcache.management.cache; import net.sf.ehcache.management.cachestatistics; import org.hibernate.stat.statistics; import org.springframework.beans.factory.annotation.autowired; import org.springframework.boot.autoconfigure.domain.entityscan; import org.springframework.cache.annotation.enablecaching; import org.springframework.transaction.annotation.enabletransactionmanagement; import org.springframework.transaction.annotation.propagation; import org.springframework.transaction.annotation.transactional; import org.springframework.web.bind.annotati
Read more

go - Golang: panic: runtime error: invalid memory address or nil pointer dereference using bufio.Scanner -

-
i implementing go program uses bufio.scanner , bufio.writer have packaged code follows package main import ( "fmt" "player/command" "strings" ) func main() { //enter code here. read input stdin. print output stdout commands.scanner.scan() { //scan new line , send comand variable check command exist or not input := strings.split(strings.trim(commands.scanner.text(), " "), " ") command := input[0] if command == "" { fmt.printf("$ %s:", commands.pwd) continue } if !commands.commands[command] { commands.throwerror("cannot recognize input.") } else { commands.execute(command, input[1:], nil) } fmt.printf("$ %s:", commands.pwd) } } i using init.go file in main package follows package main import ( "flag" "player/sourc
Read more

c - double free or corruption (fasttop) -

-
this code reads null terminated string file (file containing null terminated strings & should provide index wanted string). error: *** error in `./main': double free or corruption (fasttop): 0x090a4a80 *** aborted code: char *tmp_realloc=null; char *sstring=null; int i=1; /*set file pointer point string*/ fseek(fh,index,seek_set); sstring=(char*)malloc(sizeof(char)); if(sstring == null) return null; tmp_realloc=sstring; while( (*(sstring+i-1)=(char)fgetc((file*)fh)) != 0x0 ) { /*reallocate more memory*/ tmp_realloc=(char*)realloc((char*)sstring,++i); /*if not same address copy old new & set old point new*/ if(tmp_realloc != sstring){ strcpy((char*)tmp_realloc,(char*)sstring); free(sstring); sstring=tmp_realloc; } } could tell me pointer causing issue (& reason) ? remove part /*if not same address copy old new & set old point new*/ if(tmp_realloc != sstring){ strc
Read more