dear lazyweb…
August 26th, 2010
I’m tired of getting .pingalls and other highlights from zodbot in IRC. Is there a way I can configure Irssi to consider all messages from zodbot as regular messages — not highlights — even if they contain my name?
while I wasn’t doing my job…
July 27th, 2010
I’ve been wanting a (good) way to get notifications from irssi in the foreground for a while now. Let’s say, oh, two years.
My irssi setup is fairly standard: irssi running in screen on a remote server. There’s a few ways people have come up to run around this:
- Run the SSH session with X forwarding and use notify-send to pop up new messages
- Run a second SSH session to access a fifo of new messages and pop them up as they come
- Move your windows around so that you can always see the status bar at the bottom of irssi
The first two are, if you’re not careful, prone to attack — the most basic bash script to send messages to notify-send can easily run commands to remove files, etc., if you don’t escape quotes. The third one is impossible if you use a tiling window manager like awesome.
Since I use awesome, it’s really stupidly easy to create a widget in the status bar at the top. Add something along the lines of this to your rc.lua:
myirssistatus = widget({ type = "textbox" })
Add it to the wibox:
-- Create the wibox
mywibox[s] = awful.wibox({ position = "top", screen = s })
-- Add widgets to the wibox - order matters
mywibox[s].widgets = {
{
mylauncher,
mytaglist[s],
mypromptbox[s],
layout = awful.widget.layout.horizontal.leftright
},
mylayoutbox[s],
mytextclock,
myirssistatus,
s == 1 and mysystray or nil,
mytasklist[s],
layout = awful.widget.layout.horizontal.rightleft
}
And it’s there. Of course, it’s empty. That’s where the combination of an irssi script and a bash script come into play.
The irssi script activity_file.pl simply spits out a CSV file listing the IRC channels your in, in the following format:
num,act,channel,network
num is the channel number, act is an integer representing channel activity (0 for none, 1 for crap, 2 for messages, and 3 for hilights), channel is, well, the channel (e.g., #fedora-devel) and network is the name of the network that channel is related to (see /help network). Everything you need.
The script spits that data out to ~/.irssi/activity_file, and I have it symlinked into somewhere in ~/public_html (not the most secure, but it’s not like there’s any personal information in my channel list), and this bash script picks it up and runs with it:
#!/bin/bash
trap cleanup 2
cleanup()
{
echo "myirssistatus.text = ''" | awesome-client
exit 0
}
while true; do
DATA=""
for line in $(curl -s URL_GOES_HERE); do
win=$(echo $line | cut -d ',' -f 1)
sta=$(echo $line | cut -d ',' -f 2)
if [ "$sta" = "1" ]; then
DATA="$DATA $win"
elif [ "$sta" = "2" ]; then
DATA="$DATA $win"
elif [ "$sta" = "3" ]; then
DATA="$DATA $win"
fi
done
if [ "$DATA" ]; then
DATA="$DATA "
fi
echo "myirssistatus.text = \"$DATA\"" | awesome-client
sleep 5
done
The key there is when it pipes data into awesome-client. That’s really frickin’ neato.
So anyway. Is it a hack? Yes. Is it pretty? No. Does it work for me? Hell yeah. :)
hitting the road, v2
July 13th, 2010
Max wrote a post on our trip to Santa Cruz and Portland, but he glossed over a few things. I rewrote his post correctly below.
Ian and I will rush to the RDU airport tomorrow and hope to $DEITY that Delta assigns us seats on our flight to San Jose, where we will meet up with Karsten and Mel to eat In-N-Out for three days. This will be the first time since October that all of us are in the same state, and it will give us a chance to see where we stand on the 14 yellow cards given at the World Cup Final. This will ensure we’ve got our priorities aligned and plan a solid future for the team.
We’ll also be crashing the Community Leadership Summit and OSCON. A few members of the community are coming to assist with the impending invasion of blue-shirted ensigns. We also have a couple of presenters at OSCON, which will help us crash the par-tay even more.
Everyone on the team will be communicating to the Fedora Orbital Laser using semaphore about what we’re up to this entire week, and the FOL will beam it directly to all of your minds. I’ve also assigned Ian to write an opensource.com article, which I expect to solve world hunger.
Oh, I almost forgot!
HUP HOLLAND
(The above is a parody. Max, don’t fire me.)
another double post: Fedora’s fonts and more datanommer
July 12th, 2010
First off, fonts. The Fedora Project has used MgOpen Modata for a while now as its official font. It’s a free font and loosely resembles the font in our logo.
We just have a few issues with it — namely, it doesn’t support a lot of glyphs needed to even spell some of our contributors’ names, let alone speak in other languages. Here’s a few examples of where MgOpen Modata fails:
The top two are names of Fedora contributors. The accents versions of letters above are not available in the font, so in Inkscape, they are replaced by the system default font (which I believe is DejaVu). Those are highlighted in red. The third line is a line in Russian, nabbed from the Russian translation of the wiki’s home page. The only character in that line rendered as Modata is the question mark at the end of the line.
Mo kicked off the discussion on finding a new font a little over a week ago. As the discussion continued, it was decided that the best candidate for the font was Comfortaa, a font licensed under a CC BY-ND license, which is not permissible for inclusion in Fedora. Mo sent an email to the font author asking for the license to be changed, and we received wonderful news back.
That’s not to say the search is over — we still need your feedback and suggestions. We don’t plan on using Comfortaa as a body text font, only as a headline font; the front runner is Droid Sans (Droid Serif for print). If you can suggest a better typeface for either the headline or body text fonts, please let us know! Either comment on this blog post or send an email to the design-team list.
Edit: Here’s some samples of Comfortaa, as suggested by Nicu.
Second, datanommer. (om nom nom) A couple of weeks ago, I posted to the datanommer list a totally new way forward for the datanommer project, based on feedback from the target audience to datanommer as well as members of the Infrastructure team.
The new new master plan is to provide a simple executable, aptly named “datanommer”. It reads a configuration file which specifies queries to be run on different applications. Here’s an example query file, currently checked in to the git repo:
[memcached] servers = 127.0.0.1:11211 [module:wikipedia] path = mediawiki.MediaWiki api_url = http://en.wikipedia.org/w/api.php [data:wikipedia.recently_active_editors] grapher = matplot.bar_chart grapher.ylabel = Edit count num = 10 [module:fedorawiki] path = mediawiki.MediaWiki api_url = https://fedoraproject.org/w/api.php [data:fedorawiki.recently_active_editors] grapher = matplot.bar_chart grapher.ylabel = Edit count num = 10
Now, what on earth does this do? It fetches up to the latest 1000 edits on a MediaWiki, specifically the Fedora Project’s wiki and Wikipedia, determines the top ten editors, and places them on a bar graph created with matplotlib. The Fedora Project graph looks something like this. Neato!
Next up — the part I’m planning to work on this week — is the part that creates an HTML presentation for the data. Of course, totally configurable. After that, it’s the endless process of writing code to pull data from miscellaneous applications. Which you, of course, can help with. (Documentation coming soon!)
double post: wiki translations and funny spam comments
June 23rd, 2010
OK, here’s the important half: Translations on the Fedora Project Wiki are about to become more friendly — to viewers and translators both.
If you’ve been to the main page of the wiki in the last two hours or so, you’ll have noticed the shiny new “In other languages” box at the top of the page.
All of the translated versions of the main page that I could find that seemed remotely up to date were added to that template. There’s other translations of this page on the wiki, but they haven’t been touched since the transfer from MoinMoin.
So: instead of doing anything related to my goals at Red Hat, I decided that today would be a great day to fix the fact that we don’t have a standard way to do translations on the website. We have current instructions that contains a lot of old cruft from MoinMoin and uses a lot of MediaWiki features in a way that can be bettered.
I borrowed some template code from meta.wikimedia.org — big ups and shouts out to all the folks over at Meta-Wiki who produced those templates.
Quick summary of how it works:
- Add
{{autolang|base=yes}}to the top of the English version of a page, save - Click the set up link for the lang box, don’t touch the pre-filled code, and hit save
- Click [edit] and add the language codes for the page’s translations, save
- Click the red links to create the pages, adding
{{autolang}}to the top of each one
The full details, which you should read if you’re interested in doing this, are over at [[FedoraProject:Translating]] (or if you like neat shortcuts, you can remember [[FP:LANG]]).
An email has been sent to trans@ to get suggestions from translators within the project and see if there’s anything we need to fix/improve. You can also provide your suggestions as comments to this blog.
As a bonus for reading this whole blog post, here’s some comments my blog’s spam collector caught that I thought were funny: one two three four
mongoDB: whoa.
June 17th, 2010
Yesterday, I timed inserting metadata about 172,201 wiki edits from the Fedora Project wiki into a local SQLite database with SQLAlchemy and Python. The process took about 8 minutes, and used up over 600 MB of RAM and did not stop using the disk the entire time.
Obviously not the sort of database engine we want running datanommer, which will have that many rows in the MediaWiki table alone.
So I set up a mongoDB locally. Rebuilt the RPM that’s available in this review request, packaged up pymongo, and rewrote my Python script to use mongoDB.
Not only can the process of inserting my JSON dump of wiki edits be written in 10 lines of code, but it took less than 6 seconds to do.
Wow.
SouthEast LinuxFest: Growth is a good thing
June 15th, 2010
The first thing I noticed Saturday morning was that, in its second year, SouthEast LinuxFest has significant growth. Last year, there were enough booths for a single public space in a student union; this year, booth space was expanded to nearly every hallway used by SELF. I was amazed.
Hats off to the entire SELF team, especially David Nalley, the go-to guy for Fedora contributors by default, and Jeremy Sands, the speaker coordinator. Everybody did an amazing job at making the conference run very smoothly, all things considered. :)
Max and I drove down Friday morning to make it in time for a Docs/wiki hackfest. When I got there, we were talking about the main [[Docs Project]] wiki page; when we were done, we had reworked the join process for Docs. (Or, at least that’s what I can remember doing.)
After the speaker dinner that evening, Michael DeHaan and I skipped on the loud music party and went to go take pictures of Spartanburg. I got some interesting ones:
Saturday I went to go see a couple talks and gave one of my own on the datanommer project. About half the people were Red Hat/Fedora people, which is fine, but the other half were people who I had not seen before. That’s good — somebody outside of Fedora is interested. That’s pretty much the whole point on presenting it outside of a FUDCon. ;)
On Sunday, we had FAD @ SELF, and I think we got a few new people interested in contributing back to Fedora. Awesome!
Overall, I think it was a good weekend, and I’ll be waiting for SELF next year. :)
datanommer: Making Fedora metrics more transparent
June 10th, 2010
I kind of surprised myself when I realized I hadn’t blogged about this yet. I talked about it with Max, I talked about it with folks in #fedora-infrastructure, and I’m giving a talk at SELF that circles around this very project.
The Fedora Project, from the beginning of its collection of statistics surrounding itself, has been open and transparent about the numbers we get and how we get them.
There’s just one problem with that: a lot of the actual raw data isn’t publicly available.
Of course, we don’t want to go about publishing raw httpd access logs to public locations. We don’t want everybody to be able to see the IP addresses that visit fedoraproject.org. But we do want people to be able to come up with a number for themselves that answers questions like “how many distinct IP addresses visited fedoraproject.org between January 4 at 4:32 a.m. and February 2 and 6:28 p.m.?” without giving access to our log servers to everybody.
Or, even if the data is publicly available, it’s difficult to get that data because the application doesn’t provide an API of sorts (Mailman, for example). Writing a screen scraper for Mailman is non-trivial.
What if there was a central API that held raw data about the everyday activity of the Fedora community?
I plan to write that. And it shall be called “datanommer.” It’ll use the TG2 stack, at the request of Infrastructure, and, although it will be designed around Fedora’s existing infrastructure, will be agnostic so that other free software projects can use it right out of the box.
Here’s a quick summary of how it’ll work.
- Applications that already make log files will have those transferred to our log servers by normal means. Applications that don’t already make log files will either use an extension, module or the like to write a log file, or an external script will create a log file, which will then be transferred to the log servers.
- A cron job will populate a database used for datanommer based on those log entries.
- The TG2 front end of datanommer will provide a RESTful API to access the data in the database. Applications that provide data and what data they provide to datanommer will be automatically documented for maximum usability.
At first glance, this may seem like a lot of hoops just to get some data. But here’s some reasons we’re doing it this way, specifically:
- Less load on the app servers. If we programmed datanommer to collect data from each application about once per hour, the app servers and databases would be under somewhat heavy load while that data is generated.
- If datanommer is down for some reason, it doesn’t matter, because data entry is done directly to the database.
- If the database is down for some reason, it doesn’t matter. The cron job will just wait another hour to populate the databases.
- If the log servers are down for some reason, it doesn’t matter. Logs are generated locally on each app server, much like httpd. The log servers will go through and pick up the logs when they get around to it.
- If the applications are down for some reason, they won’t be generating any data anyway, so it doesn’t matter. :)
For the end-user, accessing the data will be extremely easy. Since a REST API is just based on query parameters, you don’t have to be an expert to download data. It’ll be encoded in JSON so it’s easy to use in any language (especially Python, the lingua franca of Fedora Infrastructure.)
Of course, your thoughts about this process are definitely wanted. You can comment on this blog post to leave your suggestions.
Edit: I forgot to include a bit about privacy — information that shouldn’t be publicly available, such as IP addresses or email addresses, will be stored in the database as UUIDs. Another table in the database will relate UUIDs to their original values for the purposes of allowing statistics to determine pageviews from distinct IP addresses, for example. Privacy is of top priority in this project and if we feel like we’re infringing on the privacy of our users and contributors too much, we will not report that information through this system.
Proudly powered by WordPress. Ian Weller's blog by Ian Weller is licensed under a Creative Commons Attribution 3.0 Unported License.
![[FSF Associate Member]](/pub/88x31/fsf-member-7305.png)
