* FAQ    * Search  * Register * Login 
Active topics
Unanswered topics

All times are UTC-06:00



Post new topic  Reply to topic  [ 4 posts ] 
Author Message
 Post subject: save to one file vs many files
PostPosted: Tue Jan 11, 2011 9:43 am 
Offline
DBB Artist
DBB Artist
User avatar

Joined: Mon Aug 01, 2005 8:47 am
Posts: 7124
Location: Ơ̸̦͇̲̬̭̱̰͎̞͈̣͎͚̳ͬ͋̃̀̇͊͂͋͐ͦ̽ͣ̂ͥ͊̅̀̚͠ B̶͖̯͉̜̰̲̓̔͋̈́ͅ È̯ Y̪̤̼͉̠̙͝
I have a few lists, each with a key column (like user IDs) and reference columns. I have a few of these that are pulled together using a master list that has one key column and reference column, which the app can quickly pull from.

Is it better to have this all in one file or is it faster for python have separate files, which are read and written during each user's session? Sometimes these sessions run parallel to each other.


Top
   
 Post subject:
PostPosted: Tue Jan 11, 2011 9:08 pm 
Offline
DBB Master
DBB Master
User avatar

Joined: Sun Sep 05, 1999 2:01 am
Posts: 6377
Location: ☃☃☃
I don't think I understand the motivating problem, but if you're only reading or changing small bits of the file at a time, then it's generally better to have it spread over multiple files so as to avoid rewriting the rest when you only change a bit.

Having the data spread over multiple files also reduces the chance for file system race conditions, but the problem is still present. Since most Web servers can handle multiple requests--even from the same person--simultaneously, you need to take extra precautions to be robust to these. Suppose I click on 'increment.py' simultaneously in two windows. This could be the order of events:

Process A reads i=10 in counter.txt.
Process B reads i=10 in counter.txt.
Process A writes i=11 to counter.txt.
Process B writes i=11 to counter.txt.

I clicked increment twice, but it only incremented once! In general, you can get around this using file locks, but at this point, instead of reinventing a transactional database, why not use one like Postgres or MySQL. (Postgres seems to be more popular in the Python world in my experience.)


Top
   
 Post subject:
PostPosted: Sat Jan 15, 2011 3:52 pm 
Offline
DBB Artist
DBB Artist
User avatar

Joined: Mon Aug 01, 2005 8:47 am
Posts: 7124
Location: Ơ̸̦͇̲̬̭̱̰͎̞͈̣͎͚̳ͬ͋̃̀̇͊͂͋͐ͦ̽ͣ̂ͥ͊̅̀̚͠ B̶͖̯͉̜̰̲̓̔͋̈́ͅ È̯ Y̪̤̼͉̠̙͝
Cool ok. I'll start reading about the two.


Top
   
 Post subject:
PostPosted: Sat Jan 15, 2011 4:22 pm 
Offline
DBB Admiral
DBB Admiral
User avatar

Joined: Sun Jun 06, 2004 2:44 am
Posts: 1497
ICQ: 17713745
Website: http://www.radnimax.com
WLM: xamindar@gmail.com
Yahoo Messenger: xamindar
AOL: xamindar
Location: California
Also, if it is a small need check out sqlite as well. You might not need to have a full fledged database server installed just for what you are doing.

_________________
Why doesn't it work?


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 4 posts ] 

All times are UTC-06:00


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  



Descent'rs have piloted these pages
 
The layout and contents contained within this site are © DescentBB.net 1997-2006.
Descent, Descent II are © Parallax Software Corporation.
Descent III is Outrage Entertainment.
Descent is a Trademark of Interplay Productions.

Miner Wars™ is trademark of Keen Software House s. r. o.
.


Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group