Saturday, 15 May 2010

Multi-threaded psycopg2 and python not returning results -



Multi-threaded psycopg2 and python not returning results -

i have programme parent process has db connection, , each kid process has own db connection (created in constructor), using python 2.6 , psycopg2.

every 5 seconds, parent process queries db progress study on kid process. each kid process doing x things , storing step on in db.

i've set simplified version of code below

def getstatus( conn ): query = "select job_name, status job_status_table" cursor = conn.getcursor() cursor.execute( query ) homecoming cursor.fetchall() def simplestatus(): conn = dbinit() # output right here print getstatus( conn ) queue = getjoblist( conn ) q in queue: p = multiprocessing.process( target=run, args=q ) p.start() while: # status terminate, not germane time.sleep( 5 ) # output wrong here print getstatus( conn ) ...

inside kid process, calls following:

def updatestatus( self, status_i ): update = "update job_status_table set status='%s' job_name='%s'"%( status_i, self.name ) cursor = self.conn.getcursor() cursor.execute( update ) self.conn.commit()

external querying of database (psql) shows query returning right results @ time beingness run in program. however, in programme not. if alter programme re-initialize db after time.sleep call, gives right output. why?

the parent process in own transaction , not see changes until terminates (by commit() or rollback()). have 2 choices:

put parent process connection in autocommit mode (conn.autocommit = true); or just issue commit()/rollback() on connection before executing query create sure execute in new, date, transaction.

python psycopg2 multiprocess

No comments:

Post a Comment