from __future__ import print_function

One of the main things that we want to do in scientific computing is get data into and out of our programs. In addition to plain text files, there are modules that can read lots of different data formats we might encounter.

Print#

We’ve already been using print quite a bit, but now we’ll look at how to control how information is printed. Note that there is an older and newer way to format print statements – we’ll focus only on the newer way (it’s nicer).

This is compatible with both python 2 and 3

x = 1
y = 0.0000354
z = 3.0
s = "my string"
print(x)
1

We write a string with {} embedded to indicate where variables are to be inserted. Note that {} can take arguments. We use the format() method on the string to match the variables to the {}.

print("x = {}, y = {}, z = {}, s = {}".format(x, y, z, s))
x = 1, y = 3.54e-05, z = 3.0, s = my string

Before a semi-colon, we can give an optional index/position/descriptor of the value we want to print.

After the semi-colon we give a format specifier. It has a number field and a type, like f and g to describe how floating point numbers appear and how much precision to show. Other bits are possible as well (like justification).

print("x = {0}, y = {1:10.5g}, z = {2:.3f}, s = {3}".format(x, y, z, s))
x = 1, y =   3.54e-05, z = 3.000, s = my string

there are other formatting things, like justification, etc. See the tutorial

print("{:^80}".format("centered string"))
                                centered string                                 

File I/O#

as expected, a file is an object. Here we’ll use the try, except block to capture exceptions (like if the file cannot be opened).

try: f = open("./sample.txt", "w")   # open for writing -- any file of the same name will be overwritten
except: 
    print("cannot open the file")

print(f)
<_io.TextIOWrapper name='./sample.txt' mode='w' encoding='UTF-8'>
f.write("this is my first write\n")
f.close()

we can easily loop over the lines in a file

try: 
    f = open("./test.txt", "r")
except:
    print("error: cannot open the file")
    
for line in f:
    print(line.split())
    
f.close()
['Lorem', 'ipsum', 'dolor', 'sit', 'amet,', 'consectetur', 'adipisicing', 'elit,', 'sed', 'do']
['eiusmod', 'tempor', 'incididunt', 'ut', 'labore', 'et', 'dolore', 'magna', 'aliqua.', 'Ut', 'enim', 'ad']
['minim', 'veniam,', 'quis', 'nostrud', 'exercitation', 'ullamco', 'laboris', 'nisi', 'ut']
['aliquip', 'ex', 'ea', 'commodo', 'consequat.', 'Duis', 'aute', 'irure', 'dolor', 'in']
['reprehenderit', 'in', 'voluptate', 'velit', 'esse', 'cillum', 'dolore', 'eu', 'fugiat', 'nulla']
['pariatur.', 'Excepteur', 'sint', 'occaecat', 'cupidatat', 'non', 'proident,', 'sunt', 'in']
['culpa', 'qui', 'officia', 'deserunt', 'mollit', 'anim', 'id', 'est', 'laborum.']
[]

as mentioned earlier, there are lots of string functions. Above we used strip() to remove the trailing whitespace and returns

CSV Files#

comma-separated values are an easy way to exchange data – you can generate these from a spreadsheet program. In the example below, we are assuming that the first line of the spreadsheet/csv file gives the headings that identify the columns.

Note that there is an amazing amount of variation in terms of what can be in a CSV file and what the format is – the csv module does a good job sorting this all out for you.

class Item(object):
    def __init__(self):
        self.name = ""
        self.quantity = 0
        self.unitprice = 0.0
        self.total = 0.0
    
import csv

reader = csv.reader(open("shopping.csv", "r"))

headings = None

shopping_list = []

for row in reader:
    if headings == None:
        # first row
        headings = row
    else:
        my_item = Item()
        my_item.name = row[headings.index("item")]
        my_item.quantity = row[headings.index("quantity")]
        my_item.unitprice = row[headings.index("unit price")]
        my_item.total = row[headings.index("total")]
        shopping_list.append(my_item)
    

for i in shopping_list:
    print ("item: {}, quantity: {}, unit price: {}, total: {}".format(i.name, i.quantity, i.unitprice, i.total))
item: apples, quantity: 2, unit price: 0.33, total: 0.66
item: bananas, quantity: 5, unit price: 0.1, total: 0.5
item: milk, quantity: 1, unit price: 2.5, total: 2.5
item: soda, quantity: 3, unit price: 1, total: 3
item: rolls, quantity: 12, unit price: 0.33, total: 3.96
item: eggs, quantity: 1, unit price: 2.5, total: 2.5

INI Files#

INI or Config files are a common way of specifying options to a program. They can take the form (from the ConfigParser page):

[My Section]
foodir: %(dir)s/whatever
dir=frob
long: this value continues
   in the next line

Here we look at how to read in options and store them in a dictionary of the form dict["sec.option"] = value

We’ll use a sample .ini file from a regression test suite (VARDEN-tests.ini)

(Note: the name of the module is ConfigParser in python 2 but configparser in python 3 – the latter confirms to the python style guidelines.)

import configparser

options = {}


cp = configparser.ConfigParser()
cp.optionxform = str    # this makes options case-sensitive
cp.read("VARDEN-tests.ini")

for sec in cp.sections():
    for opt in cp.options(sec):
        key = str(sec) + "." + str(opt)
        value = cp.get(sec,opt)
        options[key] = value
    
for k, v in options.items():
    print("{:32s}: {}".format(k, v))
main.boxLibDir                  : /home/regtester/RegTesting/BoxLib/
main.sourceDir                  : /home/regtester/RegTesting/VARDEN/
main.testTopDir                 : /home/regtester/RegTesting/rt-VARDEN/
main.webTopDir                  : /home/regtester/RegTesting/rt-VARDEN/web
main.compareToolDir             : /home/regtester/RegTesting/AmrPostprocessing/F_Src
main.MAKE                       : make
main.sourceTree                 : F_Src
main.numMakeJobs                : 8
main.COMP                       : g++
main.FCOMP                      : gfortran
main.suiteName                  : VARDEN
main.reportActiveTestsOnly      : 1
main.goUpLink                   : 1
main.MPIcommand                 : /usr/local/bin/mpiexec -n @nprocs@ @command@
main.MPIhost                    : 
bubble-2d.buildDir              : varden/test
bubble-2d.inputFile             : inputs_2d-regt
bubble-2d.dim                   : 2
bubble-2d.restartTest           : 0
bubble-2d.useMPI                : 1
bubble-2d.numprocs              : 2
bubble-2d.useOMP                : 0
bubble-2d.numthreads            : 2
bubble-2d.compileTest           : 0
bubble-2d.doVis                 : 0
bubble-3d.buildDir              : varden/test
bubble-3d.inputFile             : inputs_3d-regt
bubble-3d.dim                   : 3
bubble-3d.restartTest           : 0
bubble-3d.useMPI                : 1
bubble-3d.numprocs              : 3
bubble-3d.useOMP                : 1
bubble-3d.numthreads            : 2
bubble-3d.compileTest           : 0
bubble-3d.doVis                 : 0
bubble-restart.buildDir         : varden/test
bubble-restart.inputFile        : inputs-restart-regt
bubble-restart.dim              : 3
bubble-restart.restartTest      : 1
bubble-restart.restartFileNum   : 4
bubble-restart.useMPI           : 1
bubble-restart.numprocs         : 3
bubble-restart.useOMP           : 1
bubble-restart.numthreads       : 2
bubble-restart.compileTest      : 0
bubble-restart.doVis            : 0