Google drive python api: export never completes. -

-

summary:

i have issue google-drive-sdk python not detect end of document being exported. seems think google document of infinite size.

background, source code , tutorials followed:

i working on own python based google-drive backup script (one nice cli interface browsing around). git link source code

its still in making , finds new files , downloads them (with 'pull' command).

to important google-drive commands, followed official google drive api tutorials downloading media. here

what works:

when document or file non-google-docs document, file downloaded properly. however, when try "export" file. see need use different mimetype. have dictionary this.

for example: map application/vnd.google-apps.document application/vnd.openxmlformats-officedocument.wordprocessingml.document when exporting document.

when downloading google documents documents google drive, seems work fine. mean: while loop code status, done = downloader.next_chunk() eventual set done true , download completes.

what not work:

however, on files, done flag never gets true , script download forever. amounts several gb. perhaps looking wrong flag says file complete when doing export. surprised google-drive never throws error. know cause this?

current status

for have exporting of google documents disabled in code.

when use scripts "drive rakyll" (at least version have) puts link online copy. proper export offline system can maintain complete backup of on drive.

p.s. it's fine put "you should use service instead of api" sake of others finding page. know there other services out there this, i'm looking explore drive-api functions integration own other systems.

ok. found pseudo solution here.

the problem google api never returns content-length , response done in chunks. however, either chunk returned wrong, or python api not able process correctly.

what did was, grab code mediaiobasedownload from here

i left same, changed part:

if 'content-range' in resp:     content_range = resp['content-range']     length = content_range.rsplit('/', 1)[1]     self._total_size = int(length) elif 'content-length' in resp:     self._total_size = int(resp['content-length']) else:     # pseudo bug fix: no content-length, no chunk info, cut response here.     self._total_size = self._progress

the else @ end i've added. i've changed default chunk size setting default_chunk_size = 2*1024*1024. have copy few imports file, including 1 from googleapiclient.http import _retry_request, _should_retry_response

of course not solution, says "if don't understand response, stop here". make exports not work, @ least doesn't kill server. until can find solution.

update:

bug reported here: https://github.com/google/google-api-python-client/issues/15

and of january 2017, workaround not use mediaiobasedownload , instead (not suitable large files):

req = service.files().export(fileid=file_id, mimetype=mimetype) resp = req.execute(http=http)

Comments

Post a Comment

Popular posts from this blog

Spring Boot + JPA + Hibernate: Unable to locate persister -

-
i trying work entity inside request method of spring boot application. entity in project , pom.xml has dependencies it. why error message?: org.hibernate.unknownentitytypeexception: unable locate persister: com.ric.bill.model.bs.par package com.ric.web; import java.util.concurrent.atomic.atomiclong; import javax.persistence.entitymanager; import javax.persistence.persistencecontext; import net.sf.ehcache.cachemanager; import net.sf.ehcache.management.cache; import net.sf.ehcache.management.cachestatistics; import org.hibernate.stat.statistics; import org.springframework.beans.factory.annotation.autowired; import org.springframework.boot.autoconfigure.domain.entityscan; import org.springframework.cache.annotation.enablecaching; import org.springframework.transaction.annotation.enabletransactionmanagement; import org.springframework.transaction.annotation.propagation; import org.springframework.transaction.annotation.transactional; import org.springframework.web.bind.annotati
Read more

go - Golang: panic: runtime error: invalid memory address or nil pointer dereference using bufio.Scanner -

-
i implementing go program uses bufio.scanner , bufio.writer have packaged code follows package main import ( "fmt" "player/command" "strings" ) func main() { //enter code here. read input stdin. print output stdout commands.scanner.scan() { //scan new line , send comand variable check command exist or not input := strings.split(strings.trim(commands.scanner.text(), " "), " ") command := input[0] if command == "" { fmt.printf("$ %s:", commands.pwd) continue } if !commands.commands[command] { commands.throwerror("cannot recognize input.") } else { commands.execute(command, input[1:], nil) } fmt.printf("$ %s:", commands.pwd) } } i using init.go file in main package follows package main import ( "flag" "player/sourc
Read more

c - double free or corruption (fasttop) -

-
this code reads null terminated string file (file containing null terminated strings & should provide index wanted string). error: *** error in `./main': double free or corruption (fasttop): 0x090a4a80 *** aborted code: char *tmp_realloc=null; char *sstring=null; int i=1; /*set file pointer point string*/ fseek(fh,index,seek_set); sstring=(char*)malloc(sizeof(char)); if(sstring == null) return null; tmp_realloc=sstring; while( (*(sstring+i-1)=(char)fgetc((file*)fh)) != 0x0 ) { /*reallocate more memory*/ tmp_realloc=(char*)realloc((char*)sstring,++i); /*if not same address copy old new & set old point new*/ if(tmp_realloc != sstring){ strcpy((char*)tmp_realloc,(char*)sstring); free(sstring); sstring=tmp_realloc; } } could tell me pointer causing issue (& reason) ? remove part /*if not same address copy old new & set old point new*/ if(tmp_realloc != sstring){ strc
Read more