google app engine - appengine toy program hitting quota limits for reads and writes -
let's wanted create appengine application stores 50,000 word dictionary , equivalent dictionaries in 10 other languages of similar size.
i got working locally on dev server, when went load first dictionary real appserver went on writes per day quota. had no thought how many dictionary entries made datastore. so, 24 hours later, went , tried mass download dictionary , see how many entries had, doing nail reads per day quota , got nil trouble. tried enabling billing @ setting daily maximum of $1.00 , nail quota through bulkloader , didn't info problem or $1.00.
anyway, looked @ datastore viewer , shows each of dictionary words required 8 writes datastore.
so, mean sort of application inappropriate appengine? should not trying store dictionary in there? there smarter way this? instance, store dictionary in file form in blob store somehow , things programatically there?
thank suggestions
it's you'll reading much less writing problem getting info in, not much reading it.
so need utilize current configuration slow downwards write rate. then, presumably you'll getting each word it's id (the word hope!) , reads fast , small, want.
you this: chop source info 1 file per letter . upload files application , create task read each file in turn write info datastore. 1 time task completes, it's lastly action phone call next task.
it might take week finish 1 time it's there it'll alot more handy opaque blob have blob store, reading n words every single 1 interested in, , unpack , process every word.
you can utilize mass downloader upload info well!
google-app-engine blobstore google-datastore gae-quotas
No comments:
Post a Comment