pythome.com

pythome.com was a site I set up in April 2016 to collect, in one place, the error messages you run into when working with Python. This repository contains the source code of its first version. I wrote that version with Django, using Jinja2 for templates; records were stored in MySQL and frequently visited pages were cached with memcached.

Purpose

When a Python library raises an error, the message on the screen is often short and lacks context. Searching for the message rarely tells you which module and which file produced it. I wanted to approach this from the other side: collect error messages straight from the source code of the libraries and give each of them its own page. That way, someone searching for an error message could see the module, file and line where it is raised, and easily reach other errors of the same type.

How it works

The data comes from the Python 2.7 libraries installed on the server. A view called hatabul uses grep to search the library folder for raise lines containing Error(". If a message is split across several lines, it reads the next one or two lines with sed and joins them. From each match it extracts the error type (for example TypeError or AssertionError), the module name, the file path, the line number and the message itself, and writes them to the Hatalar table. A slug is generated from the message as well, so page addresses take the form /module/error-type/message/.

The home page lists all records sorted by module, error type and line number. Each record sits in a collapsible box that shows the error type, the module and the full message. The search box at the top works with jQuery Easy Autocomplete and suggests matching error messages as soon as you start typing. You can also search by message or by error type through the address bar. The detail page shows the file path, the line number and an explanation field, followed by links to other errors of the same type.

Highlights

• The error messages were not entered by hand; they were extracted automatically from the source code of the installed libraries.
• The module, file path and line number of every error are stored with the record.
• Errors of the same type are linked to each other on the detail page.
• The home page, search results and detail pages are cached with memcached.
• A robots.txt and a sitemap.xml with 540 addresses help search engines find the pages.

Technical background

The project dates from the Django 1.9 era and runs on Python 2.7. I used Jinja2 instead of Django's own template engine and added the static and url helpers to the Jinja2 environment myself. I wrote the database queries directly with MySQLdb rather than the Django ORM. The interface uses Bootstrap and jQuery. During development I kept Django Debug Toolbar enabled to watch queries and cache usage.

Looking back

In this version, queries were built by concatenating strings. A few days later, when I rewrote the site with Flask, I switched to parameterised queries; that version lives in the pythome.com_flask repository.