Will you ever read me?

Introduction

Paroxython is a set of command line tools which tag and filter by algorithmic features your collection of Python programming exercises.

Audience

You are a teacher, in charge of an introductory programming course in an educational institution. Over the years, you have accumulated many—far too many—programs and code snippets that may be of interest to your students.

Or, as a seasoned developer, you would like to share your knowledge by helping a loved one learn how to code. A cursory search for pedagogical material yields an overwhelming amount of websites and repositories stuffed with Python programs of various levels (e.g., 1, 2, 3, 4, 5, 6, and a lot more from Awesome Python in Education).

In any case, the Python source codes you have gathered are typically numerous (hundreds or even thousands), reasonably sized (anything below 100 lines of code), and educational in nature (e.g., snippets, examples, quizzes, exercise solutions, classic algorithms). The programming concepts you plan to teach remain relatively low level (e.g. assignments, nested loops, accumulation patterns, tail recursive functions, etc.).

If all that sounds familiar, keep reading me.

Main goals

Paroxython aims to help you select, from your collection, the one program that best suits your needs. For instance, it will gladly answer the following questions:

• How can this concept be illustrated?
• What problems use the same algorithmic and data structures as this one?
• What homework assignment should I give my students so they can practice the content of the last lesson?

Moreover, since Paroxython knows what your class knows, it can recommend the right program at the right time:

• What would make a good review exercise?
• Which exercises can I give on this exam?
• What is the current learning cost of this example?

In the long run, Paroxython may guide you and somehow make you rethink your course outline:

• What are the prerequisites for the concept of assignment?
• Do I have enough material to introduce subroutines before I even talk about conditionals and loops?
• Among the loops, which must come first: the most powerful (while), or the most useful (for)?
• How to logically structure this bunch of usual iterative patterns?
• What are the basics, exactly?

All issues on which the author changed his mind since he started to work on this project!

In an ideal world, Paroxython could even put an end to the deadliest religious wars, with rational, data-driven arguments:

• Father, is it a sin to exit early?
• Should a real byte use a mask?

How it works

Paroxython starts from a given folder of programs. Its contents is parsed, and all features that meet the provided specifications are labelled and associated with their spans (e.g., "assignment_lhs_identifier:a": 4, 6, 18 or "loop_with_late_exit:while": 3-7, 20-29).

These labels constitute only scattered knowledge. The next step is to map them onto a taxonomy designed with basic hierarchical constraints in mind (e.g., the fact that the introduction of the concept of early exit must come after that of loop, which itself requires that of control flow, is expressed by the taxon "flow/loop/exit/early").

Extract of the taxonomy generated from The Algorithms - Python.
Click to jump to its full dynamic version in the user manual.

Everything is then persisted in a tag database, which can later be filtered through a pipeline of commands, for instance:

• include only the programs which feature a recursive function;
• exclude this or that program you want to set aside for the exam;
• “impart” all programs studied so far, i.e, consider that all the notions they implement are acquired.

The result is a list of program recommendations ordered by increasing learning cost.

Example

Suppose that the programs directory contains these simple programs.

First, build this tag database:

> paroxython collect programs
Labelling 21 programs.
Mapping taxonomy on 21 programs.
Writing programs_db.json.

Then, filter it through this pipeline:

> paroxython recommend programs
Processing 5 commands on 21 programs.
  19 programs remaining after operation 1 (impart).
  18 programs remaining after operation 2 (exclude).
  12 programs remaining after operation 3 (exclude).
  10 programs remaining after operation 4 (include).
  10 programs remaining after operation 5 (hide).
Dumped: programs_recommendations.md.

Et voilà, your recommendation report!

Installation and test-drive

Command line

Much to no one's surprise:

python -m pip install paroxython

The following command should print a help message and exit:

paroxython --help

IPython magic command

If you use Jupyter notebook/lab, you've also just installed a so-called magic command. Load it like this:

%load_ext paroxython

This should print "paroxython 0.7.0 loaded.". Run it on a cell of Python code:

%%paroxython                          # Lines
def fibonacci(n):                     # 2
    result = []                       # 3
    (a, b) = (0, 1)                   # 4
    while a < n:                      # 5
        result.append(a)              # 6
        (a, b) = (b, a + b)           # 7
    return result                     # 8

As you can see, in this program, Paroxython identifies among others:

• the use of the procedural paradigm (style/procedural);
• an impure function (def/subroutine/function/impure);
• a while loop (flow/loop/while) with a late exit (flow/loop/exit/late);
• a little bit of voodoo on lists (type/sequence/list/literal/empty and call/subroutine/method/sequence/list/append);
• a simple tuple assignment (var/assignment/explicit/parallel). Note that we distinguish between explicit (with =) and implicit (parameters and iteration variables) assignments;
• a “sliding” tuple assignment (var/assignment/explicit/parallel/slide). If the denomination is unique to us, the pattern itself occurs in a number of programs: implementations of C-finite sequences with C greater than 1, Greatest Common Divisor, Quicksort, etc.
• four local variables (var/scope/local);
• an estimation of the variety of concepts involved (meta/count/variety/***), depending on the number of lines, features and distinct features.

The magic command %%paroxython (corresponding to the subcommand tag) only scratches the surface of the system. As shown before, to estimate the learning cost of the features and get actionable recommendations, you will need first to construct the tag database with collect, and then call recommend on a pipeline of yours.

About

What's in a name?

Far from being an arbitrary label, “Paroxython” has been scientifically formulated by an independent laboratory to possess the following properties:

1. It features both “Python” and “taxon”.
2. It provides a future-proof template for adaptations to other languages (Carox, Jaroxava, JaroxavaScript, Garoxo, Laroxua, Ocaroxaml, Haroxaskell, etc.) or fields of knowledge (Haroxarmony, Garoxeography, Maroxagnetohydrodynamics, etc.).
3. “Paroxytone” denotes a word accented or stressed on the penult¹ and translates as “paroxyton” in my mother tongue.

Citing this work

@manual{,
  title = {Paroxython: Tag and Recommend Python Exercises Based on Algorithmic Features},
  author = {{Aristide Grange}},
  organization = {Université de Lorraine},
  address = {Metz, France},
  year = {2020},
  url = {https://laowantong.github.io/paroxython/},
}

Collaborating

Paroxython is on GitHub and welcomes your issues or pull requests. You may:

• contribute to the code;
• add new features in spec.md;
• submit your personal changes to taxonomy.tsv for integration in the default taxonomy;
• translate some of this documentation into English.

If you use Paroxython for your programming course, I would love to get your feedback. If you're a researcher and interested in a scientific collaboration, drop an email to my dot-separated name (cf. previous section) at univ-lorraine.fr.

Sub-modules

paroxython.assess_costs

Assess the learning cost associated with the introduction of the given programs.

paroxython.cli

Command line interface for the main functionalities of Paroxython.

paroxython.cli_collect

Walk a directory, tag its Python files and make a database of the results.

paroxython.cli_recommend

Read and execute a pipeline of commands and report the learning costs.

paroxython.cli_tag

Tag one Python file and output the table of its taxa or labels.

paroxython.compare_spans

A dictionary of predicates listing all possible relations between two intervals $(x_1, x_2)$ and $(y_1, y_2)$.

paroxython.derived_labels_db

An in-memory database collecting the labels of a given program.

paroxython.filter_programs

Evolve a set of selected programs and a set of taxa representing the imparted knowledge.

paroxython.flatten_ast

Return a non-recursive, multiline dump of the AST (with some tweaks).

paroxython.goodies

A grab bag of various generic functions and factories.

paroxython.label_programs

List all the programs of a given directory, with their field labels populated.

paroxython.list_programs

Scan a directory for Python programs and yield the corresponding Program objects.

paroxython.make_db

Collect all infos pertaining to the programs, labels and taxa into a so-called tag database.

paroxython.map_taxonomy

Translate labels into taxa.

paroxython.normalize_predicate

Process an input string to interpret it as a known predicate.

paroxython.parse_program

Search a program for the features specified in spec.md.

paroxython.preprocess_source

Bring together everything relative to the pre-processing of a given source code.

paroxython.recommend_programs

A wrapper around ProgramFilter, feeding it with commands and assessing the learning costs of its final state.