python: how to create persistent in-memory structure for debugging -
[python 3.1]
my program takes long time run because of pickle.load
method on huge data structure. makes debugging annoying , time-consuming: every time make small change, need wait few minutes see if regression tests passed.
i replace pickle
in-memory data structure.
i thought of starting python program in 1 process, , connecting another; afraid inter-process communication overhead huge.
perhaps run python function interpreter load structure in memory. modify rest of program, can run many times (without exiting interpreter in between). seems work, i'm not sure if suffer overhead or other problems.
you can use mmap
open view on same file in multiple processes, access @ speed of memory once file loaded.
Comments
Post a Comment