What is This?
Pluggable Output Processor is an engine to wrap any executabe and capture its output through a pluggable module to colorize it and/or (re)format.
Pluggable Output Processor is my attempt to get rid of a bunch of various colorizers
from my system (like
and take everything under control ;-).
Some of them are written on Perl (and I’m not a fun of it :-) and after few hacks I’ve made to improve
colorgcc, I realized that I don’t want to
waste my time learning Perl.
Yes, I know there is a lot of stuff like this, but I have few problems w/ it:
- I’m not a Perl programmer (and don’t want to be), but I feel a permanent desire to improve that programs in different ways.
- Some of that stuff is actually abandoned – i.e. even if I can fix or improve smth, there is nobody I can send my patch to…
- Some of that stuff is
too muchend-user oriented so inflexible– they can colorize almost everything via regular expressions and configuration files. The only problem I have w/ them: some things I’d like to colorize (or fix formatting) is impossible or damn hard to express via regexes … particularly because line-by-line processing implemented in that tools have no state… Yep, I understand that, it will be hard to code, and even harder to use such a tool for end users (i.e. ppl w/o programming skills) – it is why authors took the easy way: allow to user to write regexes in configs.
- pluggable architecture – easy (to Python programmers ;-) to extend
- 256 and 16M color terminal support ;-) Configuration files in addition to standard named colors
may contain color definitions as
- colorizers for
diffout of the box (more plugins to come ;-)
- some modules are not just a stupid colorizers ;-) For example
gcccan reformat text for better readability (really helps to understand template errors). Also
cmakemodule can reduce amount of lines printed during test by collapsing test intro message and result into a single one.
$ pip install outproc
$ tar -xzf outproc-X.Y.tar.gz $ cd outproc-X.Y $ sudo easy_install .
For Gentoo users there is a live ebuild
in my repository.
Also (for Gentoo users again ;-)
eselect module from
contrib/ will be installed by the ebuild.
Users of other distros have to make symlinks to required modules manually:
# ln -s /usr/bin/outproc /usr/lib/outproc/bin/<module-name>
and then make sure
/usr/lib/outproc/bin placed before
/usr/bin (and anything else) in your
PATH environment. Personally I use the following piece of code in
/etc/env.d files will not give desired place inside a list of paths:
if [ -d /usr/lib/outproc/bin ]; then export PATH="/usr/lib/outproc/bin:$PATH" fi
Available modules (plugins) can be found at
For example, to install the
gcc module do the following:
$ ln -s /usr/bin/outproc /usr/lib/outproc/bin/gcc
Then you may edit
/etc/outproc/gcc.conf to adjust color settings.
CMake module can recognize (and colorize) various detection
SPAM info during test phase.
Also it will reduce a little lines count to be printed by one trick: if a previous line (remembered
internally after print) is a subset of the current one, just move one line above and print over it…
This module can colorize error and service messages from
make (like entering/leaving directory).
Also it can recognize some “information” messages from
gcc command line (when
used to build) – i.e. it works for cmake-based projects (my favorite build utility nowadays).
This module is capable to colorize errors, warnings, C++ code snippets (best viewed w/ 16M color terminals ;-) and reformat/simplify some loooong error messages…
Mount plugin can format columns into a table. Also it can colorize filesystems by types:
- kernel – i.e. service filesystems like
- real – i.e. a real devices was mounted
- rebind – i.e. filesystems w/
Here is mine
/etc/outproc/gcc.conf for example:
# Settings for GNU gcc output processor # Message colors by severity level error = rgb(5,1,1) warning = rgb(4,2,0) notice = normal # Source file and position location = gray(8)+itallic # Code snippet colors code = white code-keyword = rgb(4,5,0)+bold code-builtin-type = rgb(5,1,0) code-modifier = rgb(4,3,0) code-std-namespace = rgb(2,5,0) code-boost-namespace = rgb(0,2,1) code-data-member = normal code-preprocessor = green code-numeric-literal = rgb(0,2,4) code-string-literal = magenta+itallic code-comment = rgb(2,2,1)+itallic code-cursor = rgb(2,0,0) # Add a new line after source code snippet # NOTE The `gcc` module will remove lines w/ error position indicator ('^') # (error position will be indicated in a source line w/ a background color # specified by `code-cursor` color). As a result error messages become # slightly condensed ;-) -- So this setting allow you to control whether # a new line will be added instead of line w/ error position indicator. new-line-after-code = false
How to Extend
To add a new module you have to name it after a tool you want to wrap (process output from) +
.py extension and put that module into
Then make a symlink from
eselect in Gentoo. Being executed
outproc will realize (from the symlink name) what binary
to execute and capture output from. All captured lines will be piped through
A minimal plugin code will looks like this:
#!/usr/bin/env python # -*- coding: utf-8 -*- import outproc class Processor(outproc.Processor): def __init__(self, config, binary): super().__init__(config, binary) # If you want some parameters from `/etc/outproc/<your-module>.conf # use `self.config` to get them... def handle_line(self, line): ''' Handle one line from a wrapped executable ''' # TODO transfrom the line and return it back... return line
Actually a plugin may control few more things like:
- what configuration file to use
- is it want to handle an output for particular command (analyzing command line options)
or the engine should just
exec()a wrapped command
- it also may want to handle a whole block of output text instead of line-by-line
continue to improve C++ tokenizernow it is capable to handle almost everything I wanted to implement. Just one thing remains: reformat too long template names into a smth shorter than terminal width… unit tests for tokenizer
- test files w/ to cause various error messages from gcc (+ unit test for colorizer somehow)
- continue to improve
cmakesupport (+ unit tests)
mountoutput into a human readable look-n-feel
dfdepending on free space threshold
diff(easy! :-) – Done for
eselectmodule to manage tools under control
ask module is it want to handle a current command or we can do
STDINreader (pipe mode)
KeyboardInterruptand hide Python crap
ctestmodule to colorize test results
gcc -Q --help=target|optimizers|warningsDone!