sql - Designing the filesystem and database for JSON data files -

sql - Designing the filesystem and database for JSON data files -

i have api accepts json files(which json serialised objects contains user transaction data) , stores same server. every such json file has unique global id , unique user associated. user should able query through json files associated him , produce bunch of aggregated results calculated on top of files.

**edits:

a typical json file needs stored looks like:

[{"sequencenumber":125435,"currencycode":"inr","vatregistrationnumber":"10868758650","receiptnumber":{"value":"1e466gdx5x2c"},"retailtransaction":[{"otherattributes":{},"lineitem":[{"sequencenumber":1000,"otherattributes":{},"sale":{"otherattributes":{},"description":"samsung galaxy s3","unitcostprice":{"quantity":1,"value":35000},"discountamount":{"value":2500,"currency":"inr"},"itemsubtype":"smartphone"}},{"sequencenumber":1000,"otherattributes":{},"customerorderforpickup":{"otherattributes":{},"description":"iphone5","unitcostprice":{"quantity":1,"value":55000},"discountamount":{"value":5000,"currency":"inr"},"itemsubtype":"smartphone"}}],"total":[{"value":35000,"type":"transactiongrossamount","otherattributes":{}}],"grandtotal":90000.0,"reason":"delivery"},null]}]

the above json serialised version of complex object containing single or array of objects of other classes attributes. 'receiptnumber' universal id of json file.

to reply sammaye's question, need query stuff quantity , value of customerorderforpickup or grandtotal of transaction, , in aggegate of various such transaction jsons **

i have suggestion how go about: 1) storing these json files on server, file scheme ie 2) kind of database should utilize query through these json files such complex structure

my research has resulted in couple of possibilities: 1) utilize mongodb database store json representatives of object , query through database. how json files stored? best way store transaction jsons in mongodb database? 2) couple sql database containing unique global id, user id , address of json file on server, aggregating code on files. uncertainty if can scaled

would glad if has insights on problem. thanks.

i can see 2 options:

store in mongodb, mentioned, need create collection, , add together each json file straight document collection. may need alter layout of json bit improve queryability. store in hdfs, , layer hive on it. there json serde (serializer deserializer) in hive. scale well.

sql json mongodb filesystems webserver