[Python] Output data from all columns in a dataframe in pandas



Answers

Use:

pandas.set_option('display.max_columns', 7)

This will force Pandas to display the 7 columns you have. Or more generally:

pandas.set_option('display.max_columns', None)

which will force it to display any number of columns.

Explanation: the default for max_columns is 0, which tells Pandas to display the table only if all the columns can be squeezed into the width of your console.

Question

I have a csv file with the name params.csv. I opened up ipython qtconsole and created a pandas dataframe using:

import pandas
paramdata = pandas.read_csv('params.csv', names=paramnames)

where, paramnames is a python list of string objects. Example of paramnames (the length of actual list is 22):

paramnames = ["id",
"fc",
"mc",
"markup",
"asplevel",
"aspreview",
"reviewpd"]

At the ipython prompt if I type paramdata and press enter then I do not get the dataframe with columns and values as shown in examples on Pandas website. Instead, I get information about the dataframe. I get:

In[35]: paramdata
Out[35]: 
<class 'pandas.core.frame.DataFrame'>
Int64Index: 59 entries, 0 to 58
Data columns:
id                    59  non-null values
fc                    59  non-null values
mc                    59  non-null values
markup                59  non-null values
asplevel              59  non-null values
aspreview             59  non-null values
reviewpd              59  non-null values

If I type paramdata['mc'] then I do get the values as expected for the mc column. I have two questions:

(1) In the examples on the pandas website (see, for example, the output of df here: http://pandas.sourceforge.net/indexing.html#additional-column-access) typing the name of the dataframe gives the actual data. Why am I getting information about the dataframe as shown above instead of the actual data? Do I need to set some output options somewhere?

(2) How do I output all columns in the dataframe to the screen without having to type their names, i.e., without having to type something like paramdata[['id','fc','mc']].

I am using pandas version 0.8.

Thank you.




In ipython, I use this to print a part of the dataframe that works quite well (prints the first 100 rows):

print paramdata.head(100).to_string()



you can use sequence slicing syntax i.e

paramdata[:5] # first five records
paramdata[-5:] # last five records
paramdata[:] # all records

sometimes the dataframe might not fit in the screen buffer in which case you are probably better off either printing a small subset or exporting it to something else, plot or (csv again)




Links