java - Fastest way to decode a sequence of integers -
assume next file format:
integer1 integer2 integer3 integer4 ...
these binary integers written using datainputstream. there 22 billions of them written file size 90g.
the available ram > 2 * size of file.
so, can fit in memory easily. fastest approach read , decode such integers. currently,
mappedbytebuffer reads 500m numbers in time: 212.986 seconds datainputstream wrapped around bufferedinputstream reads 500m numbers in time 306.755 seconds.
the reply depends on exact combination of jvm, os, hardware , access patterns. martin thompson has written programme can utilize test sequential i/o performance of various ways read , write: http://mechanical-sympathy.blogspot.com/2011/12/java-sequential-io-performance.html
conclusion of test on platform: utilize randomaccessfile.
java io memory-mapped-files
No comments:
Post a Comment