Python has one of the most straightforward file reading and writing implementations. It may be from creating simple text documents to analyzing a server’s internal logs. All of those tasks require essential file reading and writing commands.
In this article, we will be looking at all the ways possible in Python to read a file line by line. Python contains predefined functions for file handling. Therefore, we won’t have any external dependencies.
You try the following implementations on your browser here.
Using Python Read Lines Function
Python readlines()
method is a predefined function. Upon calling, it returns us a list type consisting of each line from the document as an element.
Syntax – filename.readlines()
Parameters – hint
. This is an optional parameter that mentions the maximum number of bytes to be returned. The default value is -1, which returns all bytes.
Returns – A list type consisting of the file contents as list elements.
Let’s look at the following example.
The following text document will be taken as an example.
myFile = open("PythonPool.txt", "r")
print(myFile.readlines())
Output
['Welcome to PythonPool! Source for Free Python Material.']
Using For Loop to Read Lines
We can use the for loop to read files line by line in Python. The open() function returns an iterable object while opening the document. Using the object, we can iterate over a for loop to read the lines. Let’s take a look at an example.
This is the text file we will be using
myFile = open("PythonPool.txt", "r")
for eachLine in myFile:
print(eachLine)
Output
Welcome to PythonPool! Source for Free Python Material. This is line 2 This is line 3
Reading File Lines Using Context Manager
Let’s look at another method in Python to read a file line by line.
In Python, a context manager allows the user to manage files using the “with” keyword alongside File I/O methods. It monitors the file being opened and closes immediately after the file operations finish executing.
In this demonstration, we will be using a context manager along with the For Loop
This is the sample document we will be using:
with open("/content/PythonPool.txt") as myFile:
myFileContents = myFile.read()
print(myFileContents)
Output
Welcome to PythonPool! Source for Free Python Material. This is line 2 This is line 3
Reading CSV File Line by Line into Array
Using the genfromtxt()
feature, we can read data from the file. The function requires 2 parameters:
- File Name
delimiter
– Set of characters is used to separate each line.
from numpy import genfromtxt
myCSV = genfromtxt('/content/sample_data/california_housing_test.csv', delimiter=',')
print(myCSV)
Output
Reading a File Line by Line Backwards
To read a given file backward, we will be making use of the reversed()
function. Refer to the following demonstration.
The text document we will be using:
for line in reversed(list(open("/content/PythonPool.txt"))):
print(line.rstrip())
Output
This is line 3 This is line 2 Welcome to PythonPool! Source for Free Python Material.
However, there is an issue with this implementation. If the entire file cannot be contained within memory, the above program fails.
The alternative solution is to use the file_read_backwards
module. This allows us to read large files in memory in a much more operative manner.
from file_read_backwards import FileReadBackwards
with FileReadBackwards("/content/PythonPool.txt", encoding="utf-8") as myFile:
for eachLine in myFile:
print(eachLine)
Output
This is line 3 This is line 2 Welcome to PythonPool! Source for Free Python Material.
Reading the File Contents Into a Dictionary
In this scenario, we will be reading the file contents from the following text document into a dictionary.
By creating an empty dictionary, we can enter keys and values from the text file using the .split()
function. In order to repeat the process for multiple lines, for loop is used.
prices = {}
with open("/content/SampleText.txt") as f:
for eachLine in f:
# splitting between each space character
(key, value) = eachLine.split()
prices[str(key)] = int(value)
print(prices)
Output
{'iPhone': 150000, 'Samsung': 50000, 'OnePlus': 80000, 'Oppo': 35000}
Skipping New Line Spaces While Reading files in Python
Using the context manager, we are able to pass filters on each line and strip off new-line space characters. Refer to the following implementation.
Here is the document in question:
with open("/content/PythonPool.txt") as myFile:
lines = filter(None, (eachLine.rstrip() for eachLine in myFile))
for i in lines:
print(i)
Output
Welcome to PythonPool! Source for Free Python Material. This is line 2 This is line 3
Reading a CSV File Line by Line using Pandas Module
In very large files, memory may be an issue. Therefore, we may need external dependencies. Let’s see how we can read lines from a file using the Pandas module.
Using the chuncksize
parameter of .read_csv()
, we are able to read only the aforementioned amount.
Each line is placed into the predefined list.
import pandas as pd
lines=[]
for eachLine in pd.read_csv('/content/sample_data/california_housing_test.csv', encoding='utf-8', header=None, chunksize=1):
lines.append(eachLine.iloc[0,0])
print(lines)
Output
['longitude', -122.05, -118.3, -117.81, -118.36, -119.67, -119.56, -121.43 ... ... ... ... 117.12, -119.63]
Recommended Reading | 6 Ways to Read a CSV file with Numpy in Python
Reading XML File Lines in Python
With the help of the Python XML package, we are able to read XML files line by line.
import xml.etree.ElementTree as ET
tree = ET.parse('/content/myXML.xml')
bookCatalog = tree.findall('book')
print('Number of Books:', len(bookCatalog))
for eachBook in bookCatalog:
print('Author: ', eachBook.find('author').text)
print('Title: ', eachBook.find('title').text)
print('Price: ', eachBook.find('price').text)
Output
Number of Books: 10 Author: Gambardella, Matthew Title: XML Developer's Guide Price: 44.95 ... ... Author: O'Brien, Tim Title: Microsoft .NET: The Programming Bible Price: 36.95
Reading JSON Files Line by Line in Python
JSON or JavaScript Object Notation is a format in which data is interchanged. Most APIs send and receive requests in JSON format. Therefore, it is important that we learn how to manage such files within Python.
The following demonstration shows how we can read JSON files line by line.
With the help of the JSON standard library in Python, we are able to work with JSON files.
Here’s the JSON document we will be working with:
import json
data = [ ]
with open("/content/myJSON.txt") as myFile:
for eachLine in myFIle:
data.append(json.loads(eachLine))
print(data)
Output
Mango, Large, Yellow
FAQs
Within the context manager, you can provide the index value on the readlines()
function like so:with open(fileName) as fileVar:
variable = fileVar.readlines()[1]
print(variable)
With the help of the PyPDF2 module, we are able to handle PDF files within Python.import PyPDF2
pdfFileObj = open('samplepdf.pdf','rb')
pdfReader = PyPDF2.PdfFileReader(pdfFileObj)
Using readlines()
you can read each line of the report file.myReport = open("/content/report.rpt", "r")
repLines = myReport.readlines()
Conclusion
We have looked at how we can read files line by line in Python regardless of what format it may be. Various examples for text files, JSON files, CSV files, and XML files have been reviewed.