3write Writing a list to a file with Python
What are you going to do with the file? Does this file exist for humans, or other programs with clear interoperability requirements, or are you just trying to serialize a list to disk for later use by the same python app. If the second case is it, you should be pickleing the list.
import pickle with open('outfile', 'wb') as fp: pickle.dump(itemlist, fp)
To read it back:
with open ('outfile', 'rb') as fp: itemlist = pickle.load(fp)
Is this the cleanest way to write a list to a file, since
writelines() doesn't insert newline characters?
file.writelines(["%s\n" % item for item in list])
It seems like there would be a standard way...
>>> import simplejson >>> f = open('output.txt', 'w') >>> simplejson.dump([1,2,3,4], f) >>> f.close()
If you examine output.txt:
[1, 2, 3, 4]
This is useful because the syntax is pythonic, it's human readable, and it can be read by other programs in other languages.
I thought it would be interesting to explore the benefits of using a genexp, so here's my take.
The example in the question uses square brackets to create a temporary list, and so is equivalent to:
file.writelines( list( "%s\n" % item for item in list ) )
Which needlessly constructs a temporary list of all the lines that will be written out, this may consume significant amounts of memory depending on the size of your list and how verbose the output of
Drop the square brackets (equivalent to removing the wrapping
list() call above) will instead pass a temporary generator to
file.writelines( "%s\n" % item for item in list )
This generator will create newline-terminated representation of your
item objects on-demand (i.e. as they are written out). This is nice for a couple of reasons:
- Memory overheads are small, even for very large lists
str(item)is slow there's visible progress in the file as each item is processed
This avoids memory issues, such as:
In : import os In : f = file(os.devnull, "w") In : %timeit f.writelines( "%s\n" % item for item in xrange(2**20) ) 1 loops, best of 3: 385 ms per loop In : %timeit f.writelines( ["%s\n" % item for item in xrange(2**20)] ) ERROR: Internal Python error in the inspect module. Below is the traceback from this internal error. Traceback (most recent call last): ... MemoryError
(I triggered this error by limiting Python's max. virtual memory to ~100MB with
ulimit -v 102400).
Putting memory usage to one side, this method isn't actually any faster than the original:
In : %timeit f.writelines( "%s\n" % item for item in xrange(2**20) ) 1 loops, best of 3: 370 ms per loop In : %timeit f.writelines( ["%s\n" % item for item in xrange(2**20)] ) 1 loops, best of 3: 360 ms per loop
(Python 2.6.2 on Linux)
Following is the syntax for writelines() method
fileObject.writelines( sequence )
#!/usr/bin/python # Open a file fo = open("foo.txt", "rw+") seq = ["This is 6th line\n", "This is 7th line"] # Write sequence of lines at the end of the file. line = fo.writelines( seq ) # Close opend file fo.close()
You can also use the print function if you're on python3 as follows.
f = open("myfile.txt","wb") print(mylist, file=f)
This logic will first convert the items in list to
string(str). Sometimes the list contains a tuple like
alist = [(i12,tiger), (113,lion)]
This logic will write to file each tuple in a new line. We can later use
eval while loading each tuple when reading the file:
outfile = open('outfile.txt', 'w') # open a file in write mode for item in list_to_persistence: # iterate over the list items outfile.write(str(item) + '\n') # write to the file outfile.close() # close the file
Let avg be the list, then:
In : a = n.array((avg)) In : a.tofile('avgpoints.dat',sep='\n',dtype = '%f')
You can use
%s depending on your requirement.