Python Log Reader V1 (Python)

Colour coded log lines
~ Geek_Dude

Log files are a key aid to any technician. They help identify errors and why they might be happening and can be a gold mine of information. Sadly they are sometimes so full of information that it is like looking for a needle in a haystack.

If a log file is in plain text (.txt) format it can be hard to see the key lines that you are looking for, so today I am looking at developing a Python program to do just that.

For log reader I am going to use the VNC log from my Raspberry Pi 3.

Raspbian's /var/log contains lots of log files
Raspbian’s /var/log contains lots of log files

This log can be found under /var/log. I have wrote the Python program using modules that should work on Raspbian (Linux), Mac OS and Windows.

geektechstuff_log_reader_1
A log file in plain text format

The user.log.1 log shows lots of information but as the above screen grab shows, spotting the “error” lines is little hard.

Colour coded log lines
Colour coded log lines

The Log Reader reads each line of the log and then changes the colour of the text depending on the wording of the line; red for error, yellow for warning and blue for regular.

NOTE: I am using Python’s Colorama module, which works great in the Raspbian Terminal but not so great in the Python IDE.

The output looks clearer in Visual Studio Code’s terminal, so I may need to alter the color choices.

Colour coded log lines
Colour coded log lines

—–

#!/usr/bin/python3
# geektechstuff
# Log File Reader
# uses tkinter to draw file dialog screen
from tkinter import *
from tkinter import filedialog
# Colorama allows different colours in terminal
from colorama import init, Fore, Back, Style
init()
# regular expressions for word search
import re
# creates Tk window, hides it and asks the user which file they want to open
def file_browse():
root = Tk()
root.withdraw()
root.update()
try:
file_to_open = filedialog.askopenfilename()
except:
print(“Unable to display File Dialog Window”)
root.destroy()
return(file_to_open)
# regular expression to search for word in line (case insensitive)
def find_word(word, text):
search_w = r'(^|[^\w]){}([^\w]|$)’.format(word)
search_w = re.compile(search_w, re.IGNORECASE)
search_result = re.search(search_w, text)
returnbool(search_result)
def log_reader():
# lists to hold lines depending on content
error_lines = []
warning_lines = []
general_lines = []
# opens and reads file
withopen(file_browse(), ‘r’) as filehandle:
filecontents = filehandle.readlines()
# ERROR Lines
for line in filecontents:
if find_word(“error”,line) ==True:
print(Fore.RED+line)
print(Style.RESET_ALL)
elif find_word(“failed”,line) ==True:
print(Fore.YELLOW+line)
print(Style.RESET_ALL)
else:
print(Fore.BLUE+line)
print(Style.RESET_ALL)
return()
log_reader()

—-

I have used Tk to create a GUI driven interface for choosing the log file to read as I think that works better for the user than navigating to the log file using the termainal / command line. I have used regular expressions to search for the words (e.g. error, failure) so that case sensitivity is not an issue.
The code for this project is available on my GitHub page –https://github.com/geektechdude/Python_Log_Reader
Future considerations
  • Counting how many lines contain an error or a failure
  • Counting how many lines are in a log file
  • Out putting the colour coded log file to HTML format

—

EDIT (15th July 2019):

Added a function to read the log files in reverse order.

log_reader_backwards function
log_reader_backwards function
def log_reader_backwards():
# opens and reads file
withopen(file_browse(), ‘r’) as filehandle:
filecontents = filehandle.readlines()
# ERROR Lines
for line inreversed(list(filecontents)):
if find_word(“error”,line) ==True:
print(Fore.RED+line)
print(Style.RESET_ALL)
elif find_word(“failed”,line) ==True:
print(Fore.YELLOW+line)
print(Style.RESET_ALL)
else:
print(Fore.BLUE+line)
print(Style.RESET_ALL)
return()
—

Published by Geek_Dude

I'm a tech enthusiast that enjoys science, science fiction, comics and video games - pretty much anything geeky.

5 thoughts on “Python Log Reader V1 (Python)”

    1. Thank you for the comments, I should be able to add this by changing the print lines to include back.WHITE e.g.

      print(Fore.RED+line)

      would become

      print(Fore.RED+Back.WHITE+line)

      Liked by 1 person

  3. Pingback: Python Log File Reader Built To Use With a Raspberry Pi 3 @Raspberry_Pi #PiDay #RaspberryPi @GeekTechStuff « Adafruit Industries – Makers, hackers, artists, designers and engineers!

Comments are closed.