from __future__ import print_function
One of the main things that we want to do in scientific computing is get data into and out of our programs. In addition to plain text files, there are modules that can read lots of different data formats we might encounter.
Print#
We’ve already been using print quite a bit, but now we’ll look at how to control how information is printed. Note that there is an older and newer way to format print statements – we’ll focus only on the newer way (it’s nicer).
This is compatible with both python 2 and 3
x = 1
y = 0.0000354
z = 3.0
s = "my string"
print(x)
1
We write a string with {}
embedded to indicate where variables are to be inserted. Note that {}
can take arguments. We use the format()
method on the string to match the variables to the {}
.
print("x = {}, y = {}, z = {}, s = {}".format(x, y, z, s))
x = 1, y = 3.54e-05, z = 3.0, s = my string
Before a semi-colon, we can give an optional index/position/descriptor of the value we want to print.
After the semi-colon we give a format specifier. It has a number field and a type, like f
and g
to describe how floating point numbers appear and how much precision to show. Other bits are possible as well (like justification).
print("x = {0}, y = {1:10.5g}, z = {2:.3f}, s = {3}".format(x, y, z, s))
x = 1, y = 3.54e-05, z = 3.000, s = my string
there are other formatting things, like justification, etc. See the tutorial
print("{:^80}".format("centered string"))
centered string
File I/O#
as expected, a file is an object. Here we’ll use the try
, except
block to capture exceptions (like if the file cannot be opened).
try: f = open("./sample.txt", "w") # open for writing -- any file of the same name will be overwritten
except:
print("cannot open the file")
print(f)
<_io.TextIOWrapper name='./sample.txt' mode='w' encoding='UTF-8'>
f.write("this is my first write\n")
f.close()
we can easily loop over the lines in a file
try:
f = open("./test.txt", "r")
except:
print("error: cannot open the file")
for line in f:
print(line.split())
f.close()
['Lorem', 'ipsum', 'dolor', 'sit', 'amet,', 'consectetur', 'adipisicing', 'elit,', 'sed', 'do']
['eiusmod', 'tempor', 'incididunt', 'ut', 'labore', 'et', 'dolore', 'magna', 'aliqua.', 'Ut', 'enim', 'ad']
['minim', 'veniam,', 'quis', 'nostrud', 'exercitation', 'ullamco', 'laboris', 'nisi', 'ut']
['aliquip', 'ex', 'ea', 'commodo', 'consequat.', 'Duis', 'aute', 'irure', 'dolor', 'in']
['reprehenderit', 'in', 'voluptate', 'velit', 'esse', 'cillum', 'dolore', 'eu', 'fugiat', 'nulla']
['pariatur.', 'Excepteur', 'sint', 'occaecat', 'cupidatat', 'non', 'proident,', 'sunt', 'in']
['culpa', 'qui', 'officia', 'deserunt', 'mollit', 'anim', 'id', 'est', 'laborum.']
[]
as mentioned earlier, there are lots of string functions. Above we used strip()
to remove the trailing whitespace and returns
CSV Files#
comma-separated values are an easy way to exchange data – you can generate these from a spreadsheet program. In the example below, we are assuming that the first line of the spreadsheet/csv file gives the headings that identify the columns.
Note that there is an amazing amount of variation in terms of what can be in a CSV file and what the format is – the csv module does a good job sorting this all out for you.
class Item(object):
def __init__(self):
self.name = ""
self.quantity = 0
self.unitprice = 0.0
self.total = 0.0
import csv
reader = csv.reader(open("shopping.csv", "r"))
headings = None
shopping_list = []
for row in reader:
if headings == None:
# first row
headings = row
else:
my_item = Item()
my_item.name = row[headings.index("item")]
my_item.quantity = row[headings.index("quantity")]
my_item.unitprice = row[headings.index("unit price")]
my_item.total = row[headings.index("total")]
shopping_list.append(my_item)
for i in shopping_list:
print ("item: {}, quantity: {}, unit price: {}, total: {}".format(i.name, i.quantity, i.unitprice, i.total))
item: apples, quantity: 2, unit price: 0.33, total: 0.66
item: bananas, quantity: 5, unit price: 0.1, total: 0.5
item: milk, quantity: 1, unit price: 2.5, total: 2.5
item: soda, quantity: 3, unit price: 1, total: 3
item: rolls, quantity: 12, unit price: 0.33, total: 3.96
item: eggs, quantity: 1, unit price: 2.5, total: 2.5
INI Files#
INI or Config files are a common way of specifying options to a program. They can take the form (from the ConfigParser
page):
[My Section]
foodir: %(dir)s/whatever
dir=frob
long: this value continues
in the next line
Here we look at how to read in options and store them in a dictionary of the
form dict["sec.option"] = value
We’ll use a sample .ini file from a regression test suite (VARDEN-tests.ini
)
(Note: the name of the module is ConfigParser
in python 2 but configparser
in python 3 – the latter confirms to the python style guidelines.)
import configparser
options = {}
cp = configparser.ConfigParser()
cp.optionxform = str # this makes options case-sensitive
cp.read("VARDEN-tests.ini")
for sec in cp.sections():
for opt in cp.options(sec):
key = str(sec) + "." + str(opt)
value = cp.get(sec,opt)
options[key] = value
for k, v in options.items():
print("{:32s}: {}".format(k, v))
main.boxLibDir : /home/regtester/RegTesting/BoxLib/
main.sourceDir : /home/regtester/RegTesting/VARDEN/
main.testTopDir : /home/regtester/RegTesting/rt-VARDEN/
main.webTopDir : /home/regtester/RegTesting/rt-VARDEN/web
main.compareToolDir : /home/regtester/RegTesting/AmrPostprocessing/F_Src
main.MAKE : make
main.sourceTree : F_Src
main.numMakeJobs : 8
main.COMP : g++
main.FCOMP : gfortran
main.suiteName : VARDEN
main.reportActiveTestsOnly : 1
main.goUpLink : 1
main.MPIcommand : /usr/local/bin/mpiexec -n @nprocs@ @command@
main.MPIhost :
bubble-2d.buildDir : varden/test
bubble-2d.inputFile : inputs_2d-regt
bubble-2d.dim : 2
bubble-2d.restartTest : 0
bubble-2d.useMPI : 1
bubble-2d.numprocs : 2
bubble-2d.useOMP : 0
bubble-2d.numthreads : 2
bubble-2d.compileTest : 0
bubble-2d.doVis : 0
bubble-3d.buildDir : varden/test
bubble-3d.inputFile : inputs_3d-regt
bubble-3d.dim : 3
bubble-3d.restartTest : 0
bubble-3d.useMPI : 1
bubble-3d.numprocs : 3
bubble-3d.useOMP : 1
bubble-3d.numthreads : 2
bubble-3d.compileTest : 0
bubble-3d.doVis : 0
bubble-restart.buildDir : varden/test
bubble-restart.inputFile : inputs-restart-regt
bubble-restart.dim : 3
bubble-restart.restartTest : 1
bubble-restart.restartFileNum : 4
bubble-restart.useMPI : 1
bubble-restart.numprocs : 3
bubble-restart.useOMP : 1
bubble-restart.numthreads : 2
bubble-restart.compileTest : 0
bubble-restart.doVis : 0