My irssi setup is fairly standard: irssi running in screen on a remote server. There’s a few ways people have come up to run around this:

• Run the SSH session with X forwarding and use notify-send to pop up new messages
• Run a second SSH session to access a fifo of new messages and pop them up as they come
• Move your windows around so that you can always see the status bar at the bottom of irssi

The first two are, if you’re not careful, prone to attack — the most basic bash script to send messages to notify-send can easily run commands to remove files, etc., if you don’t escape quotes. The third one is impossible if you use a tiling window manager like awesome.

Since I use awesome, it’s really stupidly easy to create a widget in the status bar at the top. Add something along the lines of this to your rc.lua:

myirssistatus = widget({ type = "textbox" })

Add it to the wibox:

-- Create the wibox
mywibox[s] = awful.wibox({ position = "top", screen = s })
-- Add widgets to the wibox - order matters
mywibox[s].widgets = {
    {
        mylauncher,
        mytaglist[s],
        mypromptbox[s],
        layout = awful.widget.layout.horizontal.leftright
    },
    mylayoutbox[s],
    mytextclock,
    myirssistatus,
    s == 1 and mysystray or nil,
    mytasklist[s],
    layout = awful.widget.layout.horizontal.rightleft
}

And it’s there. Of course, it’s empty. That’s where the combination of an irssi script and a bash script come into play.

The irssi script activity_file.pl simply spits out a CSV file listing the IRC channels your in, in the following format:

num,act,channel,network

num is the channel number, act is an integer representing channel activity (0 for none, 1 for crap, 2 for messages, and 3 for hilights), channel is, well, the channel (e.g., #fedora-devel) and network is the name of the network that channel is related to (see /help network). Everything you need.

The script spits that data out to ~/.irssi/activity_file, and I have it symlinked into somewhere in ~/public_html (not the most secure, but it’s not like there’s any personal information in my channel list), and this bash script picks it up and runs with it:

#!/bin/bash

trap cleanup 2

cleanup()
{
	echo "myirssistatus.text = ''" | awesome-client
	exit 0
}

while true; do
	DATA=""
	for line in $(curl -s URL_GOES_HERE); do
		win=$(echo $line | cut -d ',' -f 1)
		sta=$(echo $line | cut -d ',' -f 2)
		if [ "$sta" = "1" ]; then
			DATA="$DATA $win"
		elif [ "$sta" = "2" ]; then
			DATA="$DATA $win"
		elif [ "$sta" = "3" ]; then
			DATA="$DATA $win"
		fi
	done
	if [ "$DATA" ]; then
		DATA="$DATA  "
	fi
	echo "myirssistatus.text = \"$DATA\"" | awesome-client
	sleep 5
done

The key there is when it pipes data into awesome-client. That’s really frickin’ neato.

So anyway. Is it a hack? Yes. Is it pretty? No. Does it work for me? Hell yeah. :)

Ian and I will rush to the RDU airport tomorrow and hope to $DEITY that Delta assigns us seats on our flight to San Jose, where we will meet up with Karsten and Mel to eat In-N-Out for three days. This will be the first time since October that all of us are in the same state, and it will give us a chance to see where we stand on the 14 yellow cards given at the World Cup Final. This will ensure we’ve got our priorities aligned and plan a solid future for the team.

We’ll also be crashing the Community Leadership Summit and OSCON. A few members of the community are coming to assist with the impending invasion of blue-shirted ensigns. We also have a couple of presenters at OSCON, which will help us crash the par-tay even more.

Everyone on the team will be communicating to the Fedora Orbital Laser using semaphore about what we’re up to this entire week, and the FOL will beam it directly to all of your minds. I’ve also assigned Ian to write an opensource.com article, which I expect to solve world hunger.

Oh, I almost forgot!

HUP HOLLAND

(The above is a parody. Max, don’t fire me.)

We just have a few issues with it — namely, it doesn’t support a lot of glyphs needed to even spell some of our contributors’ names, let alone speak in other languages. Here’s a few examples of where MgOpen Modata fails:

The top two are names of Fedora contributors. The accents versions of letters above are not available in the font, so in Inkscape, they are replaced by the system default font (which I believe is DejaVu). Those are highlighted in red. The third line is a line in Russian, nabbed from the Russian translation of the wiki’s home page. The only character in that line rendered as Modata is the question mark at the end of the line.

Mo kicked off the discussion on finding a new font a little over a week ago. As the discussion continued, it was decided that the best candidate for the font was Comfortaa, a font licensed under a CC BY-ND license, which is not permissible for inclusion in Fedora. Mo sent an email to the font author asking for the license to be changed, and we received wonderful news back.

That’s not to say the search is over — we still need your feedback and suggestions. We don’t plan on using Comfortaa as a body text font, only as a headline font; the front runner is Droid Sans (Droid Serif for print). If you can suggest a better typeface for either the headline or body text fonts, please let us know! Either comment on this blog post or send an email to the design-team list.

Edit: Here’s some samples of Comfortaa, as suggested by Nicu.

Second, datanommer. (om nom nom) A couple of weeks ago, I posted to the datanommer list a totally new way forward for the datanommer project, based on feedback from the target audience to datanommer as well as members of the Infrastructure team.

The new new master plan is to provide a simple executable, aptly named “datanommer”. It reads a configuration file which specifies queries to be run on different applications. Here’s an example query file, currently checked in to the git repo:

[memcached]
servers = 127.0.0.1:11211

[module:wikipedia]
path = mediawiki.MediaWiki
api_url = http://en.wikipedia.org/w/api.php

[data:wikipedia.recently_active_editors]
grapher = matplot.bar_chart
grapher.ylabel = Edit count
num = 10

[module:fedorawiki]
path = mediawiki.MediaWiki
api_url = https://fedoraproject.org/w/api.php

[data:fedorawiki.recently_active_editors]
grapher = matplot.bar_chart
grapher.ylabel = Edit count
num = 10

Now, what on earth does this do? It fetches up to the latest 1000 edits on a MediaWiki, specifically the Fedora Project’s wiki and Wikipedia, determines the top ten editors, and places them on a bar graph created with matplotlib. The Fedora Project graph looks something like this. Neato!

Next up — the part I’m planning to work on this week — is the part that creates an HTML presentation for the data. Of course, totally configurable. After that, it’s the endless process of writing code to pull data from miscellaneous applications. Which you, of course, can help with. (Documentation coming soon!)

If you’ve been to the main page of the wiki in the last two hours or so, you’ll have noticed the shiny new “In other languages” box at the top of the page.

All of the translated versions of the main page that I could find that seemed remotely up to date were added to that template. There’s other translations of this page on the wiki, but they haven’t been touched since the transfer from MoinMoin.

So: instead of doing anything related to my goals at Red Hat, I decided that today would be a great day to fix the fact that we don’t have a standard way to do translations on the website. We have current instructions that contains a lot of old cruft from MoinMoin and uses a lot of MediaWiki features in a way that can be bettered.

I borrowed some template code from meta.wikimedia.org — big ups and shouts out to all the folks over at Meta-Wiki who produced those templates.

Quick summary of how it works:

• Add {{autolang|base=yes}} to the top of the English version of a page, save
• Click the set up link for the lang box, don’t touch the pre-filled code, and hit save
• Click [edit] and add the language codes for the page’s translations, save
• Click the red links to create the pages, adding {{autolang}} to the top of each one

The full details, which you should read if you’re interested in doing this, are over at [[FedoraProject:Translating]] (or if you like neat shortcuts, you can remember [[FP:LANG]]).

An email has been sent to trans@ to get suggestions from translators within the project and see if there’s anything we need to fix/improve. You can also provide your suggestions as comments to this blog.

As a bonus for reading this whole blog post, here’s some comments my blog’s spam collector caught that I thought were funny: one two three four

Obviously not the sort of database engine we want running datanommer, which will have that many rows in the MediaWiki table alone.

So I set up a mongoDB locally. Rebuilt the RPM that’s available in this review request, packaged up pymongo, and rewrote my Python script to use mongoDB.

Not only can the process of inserting my JSON dump of wiki edits be written in 10 lines of code, but it took less than 6 seconds to do.

Wow.

Hats off to the entire SELF team, especially David Nalley, the go-to guy for Fedora contributors by default, and Jeremy Sands, the speaker coordinator. Everybody did an amazing job at making the conference run very smoothly, all things considered. :)

Max and I drove down Friday morning to make it in time for a Docs/wiki hackfest. When I got there, we were talking about the main [[Docs Project]] wiki page; when we were done, we had reworked the join process for Docs. (Or, at least that’s what I can remember doing.)

After the speaker dinner that evening, Michael DeHaan and I skipped on the loud music party and went to go take pictures of Spartanburg. I got some interesting ones:

Saturday I went to go see a couple talks and gave one of my own on the datanommer project. About half the people were Red Hat/Fedora people, which is fine, but the other half were people who I had not seen before. That’s good — somebody outside of Fedora is interested. That’s pretty much the whole point on presenting it outside of a FUDCon. ;)

On Sunday, we had FAD @ SELF, and I think we got a few new people interested in contributing back to Fedora. Awesome!

Overall, I think it was a good weekend, and I’ll be waiting for SELF next year. :)

The Fedora Project, from the beginning of its collection of statistics surrounding itself, has been open and transparent about the numbers we get and how we get them.

There’s just one problem with that: a lot of the actual raw data isn’t publicly available.

Of course, we don’t want to go about publishing raw httpd access logs to public locations. We don’t want everybody to be able to see the IP addresses that visit fedoraproject.org. But we do want people to be able to come up with a number for themselves that answers questions like “how many distinct IP addresses visited fedoraproject.org between January 4 at 4:32 a.m. and February 2 and 6:28 p.m.?” without giving access to our log servers to everybody.

Or, even if the data is publicly available, it’s difficult to get that data because the application doesn’t provide an API of sorts (Mailman, for example). Writing a screen scraper for Mailman is non-trivial.

What if there was a central API that held raw data about the everyday activity of the Fedora community?

I plan to write that. And it shall be called “datanommer.” It’ll use the TG2 stack, at the request of Infrastructure, and, although it will be designed around Fedora’s existing infrastructure, will be agnostic so that other free software projects can use it right out of the box.

Here’s a quick summary of how it’ll work.

• Applications that already make log files will have those transferred to our log servers by normal means. Applications that don’t already make log files will either use an extension, module or the like to write a log file, or an external script will create a log file, which will then be transferred to the log servers.
• A cron job will populate a database used for datanommer based on those log entries.
• The TG2 front end of datanommer will provide a RESTful API to access the data in the database. Applications that provide data and what data they provide to datanommer will be automatically documented for maximum usability.

At first glance, this may seem like a lot of hoops just to get some data. But here’s some reasons we’re doing it this way, specifically:

• Less load on the app servers. If we programmed datanommer to collect data from each application about once per hour, the app servers and databases would be under somewhat heavy load while that data is generated.
• If datanommer is down for some reason, it doesn’t matter, because data entry is done directly to the database.
• If the database is down for some reason, it doesn’t matter. The cron job will just wait another hour to populate the databases.
• If the log servers are down for some reason, it doesn’t matter. Logs are generated locally on each app server, much like httpd. The log servers will go through and pick up the logs when they get around to it.
• If the applications are down for some reason, they won’t be generating any data anyway, so it doesn’t matter. :)

For the end-user, accessing the data will be extremely easy. Since a REST API is just based on query parameters, you don’t have to be an expert to download data. It’ll be encoded in JSON so it’s easy to use in any language (especially Python, the lingua franca of Fedora Infrastructure.)

Of course, your thoughts about this process are definitely wanted. You can comment on this blog post to leave your suggestions.

Edit: I forgot to include a bit about privacy — information that shouldn’t be publicly available, such as IP addresses or email addresses, will be stored in the database as UUIDs. Another table in the database will relate UUIDs to their original values for the purposes of allowing statistics to determine pageviews from distinct IP addresses, for example. Privacy is of top priority in this project and if we feel like we’re infringing on the privacy of our users and contributors too much, we will not report that information through this system.