July 27th, 2010
I’ve been wanting a (good) way to get notifications from irssi in the foreground for a while now. Let’s say, oh, two years.
My irssi setup is fairly standard: irssi running in screen on a remote server. There’s a few ways people have come up to run around this:
- Run the SSH session with X forwarding and use notify-send to pop up new messages
- Run a second SSH session to access a fifo of new messages and pop them up as they come
- Move your windows around so that you can always see the status bar at the bottom of irssi
The first two are, if you’re not careful, prone to attack — the most basic bash script to send messages to notify-send can easily run commands to remove files, etc., if you don’t escape quotes. The third one is impossible if you use a tiling window manager like awesome.
Since I use awesome, it’s really stupidly easy to create a widget in the status bar at the top. Add something along the lines of this to your rc.lua:
myirssistatus = widget({ type = "textbox" })
Add it to the wibox:
-- Create the wibox
mywibox[s] = awful.wibox({ position = "top", screen = s })
-- Add widgets to the wibox - order matters
mywibox[s].widgets = {
{
mylauncher,
mytaglist[s],
mypromptbox[s],
layout = awful.widget.layout.horizontal.leftright
},
mylayoutbox[s],
mytextclock,
myirssistatus,
s == 1 and mysystray or nil,
mytasklist[s],
layout = awful.widget.layout.horizontal.rightleft
}
And it’s there. Of course, it’s empty. That’s where the combination of an irssi script and a bash script come into play.
The irssi script activity_file.pl simply spits out a CSV file listing the IRC channels your in, in the following format:
num,act,channel,network
num is the channel number, act is an integer representing channel activity (0 for none, 1 for crap, 2 for messages, and 3 for hilights), channel is, well, the channel (e.g., #fedora-devel) and network is the name of the network that channel is related to (see /help network). Everything you need.
The script spits that data out to ~/.irssi/activity_file, and I have it symlinked into somewhere in ~/public_html (not the most secure, but it’s not like there’s any personal information in my channel list), and this bash script picks it up and runs with it:
#!/bin/bash
trap cleanup 2
cleanup()
{
echo "myirssistatus.text = ''" | awesome-client
exit 0
}
while true; do
DATA=""
for line in $(curl -s URL_GOES_HERE); do
win=$(echo $line | cut -d ',' -f 1)
sta=$(echo $line | cut -d ',' -f 2)
if [ "$sta" = "1" ]; then
DATA="$DATA $win"
elif [ "$sta" = "2" ]; then
DATA="$DATA $win"
elif [ "$sta" = "3" ]; then
DATA="$DATA $win"
fi
done
if [ "$DATA" ]; then
DATA="$DATA "
fi
echo "myirssistatus.text = \"$DATA\"" | awesome-client
sleep 5
done
The key there is when it pipes data into awesome-client. That’s really frickin’ neato.
So anyway. Is it a hack? Yes. Is it pretty? No. Does it work for me? Hell yeah. :)
Tags: awesome, bash, fedora, irc, irssi | Comments Off
July 12th, 2010
First off, fonts. The Fedora Project has used MgOpen Modata for a while now as its official font. It’s a free font and loosely resembles the font in our logo.
We just have a few issues with it — namely, it doesn’t support a lot of glyphs needed to even spell some of our contributors’ names, let alone speak in other languages. Here’s a few examples of where MgOpen Modata fails:
The top two are names of Fedora contributors. The accents versions of letters above are not available in the font, so in Inkscape, they are replaced by the system default font (which I believe is DejaVu). Those are highlighted in red. The third line is a line in Russian, nabbed from the Russian translation of the wiki’s home page. The only character in that line rendered as Modata is the question mark at the end of the line.
Mo kicked off the discussion on finding a new font a little over a week ago. As the discussion continued, it was decided that the best candidate for the font was Comfortaa, a font licensed under a CC BY-ND license, which is not permissible for inclusion in Fedora. Mo sent an email to the font author asking for the license to be changed, and we received wonderful news back.
That’s not to say the search is over — we still need your feedback and suggestions. We don’t plan on using Comfortaa as a body text font, only as a headline font; the front runner is Droid Sans (Droid Serif for print). If you can suggest a better typeface for either the headline or body text fonts, please let us know! Either comment on this blog post or send an email to the design-team list.
Edit: Here’s some samples of Comfortaa, as suggested by Nicu.
Second, datanommer. (om nom nom) A couple of weeks ago, I posted to the datanommer list a totally new way forward for the datanommer project, based on feedback from the target audience to datanommer as well as members of the Infrastructure team.
The new new master plan is to provide a simple executable, aptly named “datanommer”. It reads a configuration file which specifies queries to be run on different applications. Here’s an example query file, currently checked in to the git repo:
[memcached]
servers = 127.0.0.1:11211
[module:wikipedia]
path = mediawiki.MediaWiki
api_url = http://en.wikipedia.org/w/api.php
[data:wikipedia.recently_active_editors]
grapher = matplot.bar_chart
grapher.ylabel = Edit count
num = 10
[module:fedorawiki]
path = mediawiki.MediaWiki
api_url = https://fedoraproject.org/w/api.php
[data:fedorawiki.recently_active_editors]
grapher = matplot.bar_chart
grapher.ylabel = Edit count
num = 10
Now, what on earth does this do? It fetches up to the latest 1000 edits on a MediaWiki, specifically the Fedora Project’s wiki and Wikipedia, determines the top ten editors, and places them on a bar graph created with matplotlib. The Fedora Project graph looks something like this. Neato!
Next up — the part I’m planning to work on this week — is the part that creates an HTML presentation for the data. Of course, totally configurable. After that, it’s the endless process of writing code to pull data from miscellaneous applications. Which you, of course, can help with. (Documentation coming soon!)
Tags: comfortaa, datanommer, fedora, fonts, mgopen modata, wiki | 3 Comments »
June 10th, 2010
I kind of surprised myself when I realized I hadn’t blogged about this yet. I talked about it with Max, I talked about it with folks in #fedora-infrastructure, and I’m giving a talk at SELF that circles around this very project.
The Fedora Project, from the beginning of its collection of statistics surrounding itself, has been open and transparent about the numbers we get and how we get them.
There’s just one problem with that: a lot of the actual raw data isn’t publicly available.
Of course, we don’t want to go about publishing raw httpd access logs to public locations. We don’t want everybody to be able to see the IP addresses that visit fedoraproject.org. But we do want people to be able to come up with a number for themselves that answers questions like “how many distinct IP addresses visited fedoraproject.org between January 4 at 4:32 a.m. and February 2 and 6:28 p.m.?” without giving access to our log servers to everybody.
Or, even if the data is publicly available, it’s difficult to get that data because the application doesn’t provide an API of sorts (Mailman, for example). Writing a screen scraper for Mailman is non-trivial.
What if there was a central API that held raw data about the everyday activity of the Fedora community?
I plan to write that. And it shall be called “datanommer.” It’ll use the TG2 stack, at the request of Infrastructure, and, although it will be designed around Fedora’s existing infrastructure, will be agnostic so that other free software projects can use it right out of the box.
Here’s a quick summary of how it’ll work.
- Applications that already make log files will have those transferred to our log servers by normal means. Applications that don’t already make log files will either use an extension, module or the like to write a log file, or an external script will create a log file, which will then be transferred to the log servers.
- A cron job will populate a database used for datanommer based on those log entries.
- The TG2 front end of datanommer will provide a RESTful API to access the data in the database. Applications that provide data and what data they provide to datanommer will be automatically documented for maximum usability.
At first glance, this may seem like a lot of hoops just to get some data. But here’s some reasons we’re doing it this way, specifically:
- Less load on the app servers. If we programmed datanommer to collect data from each application about once per hour, the app servers and databases would be under somewhat heavy load while that data is generated.
- If datanommer is down for some reason, it doesn’t matter, because data entry is done directly to the database.
- If the database is down for some reason, it doesn’t matter. The cron job will just wait another hour to populate the databases.
- If the log servers are down for some reason, it doesn’t matter. Logs are generated locally on each app server, much like httpd. The log servers will go through and pick up the logs when they get around to it.
- If the applications are down for some reason, they won’t be generating any data anyway, so it doesn’t matter. :)
For the end-user, accessing the data will be extremely easy. Since a REST API is just based on query parameters, you don’t have to be an expert to download data. It’ll be encoded in JSON so it’s easy to use in any language (especially Python, the lingua franca of Fedora Infrastructure.)
Of course, your thoughts about this process are definitely wanted. You can comment on this blog post to leave your suggestions.
Edit: I forgot to include a bit about privacy — information that shouldn’t be publicly available, such as IP addresses or email addresses, will be stored in the database as UUIDs. Another table in the database will relate UUIDs to their original values for the purposes of allowing statistics to determine pageviews from distinct IP addresses, for example. Privacy is of top priority in this project and if we feel like we’re infringing on the privacy of our users and contributors too much, we will not report that information through this system.
Tags: datanommer, fedora, infrastructure, metrics, statistics, turbogears | 3 Comments »
May 26th, 2010
One of the great advantages of having membership within the Fedora Project (including all the little subgroups like ambassadors) centralized in FAS is that you can write a simple script to get some meaningful numbers.
We were discussing ambassador mentoring at FAD NA 2010 and one of the many proposals tossed back and forth was to require that ambassadors are within the project for a period of time before they apply to be an ambassador. David Nalley asked the group: “How long do people wait now before they join the ambassadors group in FAS?” Three seconds later he turned to me and asked me to write a script to do that. Here it is.
This script downloads a bunch of group data from FAS (which takes a little while because it needs to grab cla_done), finds users who have signed the CLA (approved in cla_done), and who have applied to be an ambassador but have not yet been approved. It then determines the amount of time the user spent between signing the CLA and applying to be an ambassador in FAS (what we’ll call the “delta”). It prints two lines: the first is a sorted Python list of the delta, converted to seconds; the second is the number of users the list describes (a count of the elements in the list).
(It should be noted that there is a cutoff for the usability of time-based data in FAS. For some reason or another—whether it was beacuse FAS1 didn’t track times, or because the upgrade to FAS2 overwrote the times—timestamps for group joins and approvals are all horribly wrong before March 12, 2008 at 02:06 UTC. See line 11 in the script.)
As of the FAD, here’s the data it produced (with line breaks added):
[65, 71, 90, 100, 117, 157, 177, 359, 367, 390, 432, 455, 518, 1032, 4174, 4327,
10162, 18168, 21257, 66571, 120267, 122254, 230746, 451587, 904754, 1293886,
1378508, 2001388, 2619665, 3862083, 6272559, 10794330, 15915004, 19977760,
36867582, 39432762]
36
Some conclusions we can make based on this data:
- The average delta was 1544 seconds, which is about 26 minutes.
- 20 of the 36 users (55.6%) had a delta of less than a day (86400 seconds). 7 of the 36 users (19.4%) had a delta of less than 5 minutes (300 seconds).
- The maximum delta was 456.4 days (about 15 months).
If you look to the comment on line 19 of the script, it’s a simple one-line change to get data for those who already have become ambassadors as well. Here’s the data for that, as of now(ish), with line breaks added:
[-7046682, -2244969, -2169415, -2105694, -1210664, -946193, -171773, -132781,
-105235, -88070, -11491, -2193, -380, -70, -31, -30, -13, 18, 19, 22, 26, 26,
26, 33, 33, 33, 36, 39, 40, 41, 43, 46, 47, 47, 47, 57, 59, 60, 61, 62, 62, 66,
66, 66, 67, 68, 69, 71, 71, 75, 76, 76, 77, 80, 85, 90, 90, 90, 92, 93, 95, 96,
98, 104, 105, 106, 109, 109, 109, 110, 111, 118, 119, 119, 120, 120, 127, 128,
131, 134, 135, 137, 139, 143, 145, 145, 146, 150, 150, 152, 155, 156, 158, 159,
168, 169, 176, 183, 185, 189, 191, 194, 194, 196, 198, 199, 205, 210, 211, 214,
217, 222, 222, 225, 237, 240, 243, 245, 252, 256, 258, 262, 264, 270, 272, 272,
278, 283, 294, 294, 295, 296, 297, 304, 306, 319, 321, 321, 323, 328, 335, 343,
346, 353, 361, 374, 378, 400, 400, 402, 412, 421, 441, 450, 452, 452, 455, 478,
484, 491, 520, 531, 575, 589, 607, 607, 621, 648, 658, 663, 705, 720, 722, 724,
732, 733, 738, 749, 753, 814, 827, 832, 874, 880, 929, 950, 956, 1012, 1014,
1041, 1046, 1131, 1286, 1381, 1408, 1430, 1559, 1577, 1821, 1845, 1887, 1906,
1971, 2028, 2165, 2195, 2424, 2479, 2640, 2901, 2934, 3094, 3339, 3354, 3364,
3413, 3414, 3711, 4874, 5386, 5426, 5577, 6329, 7416, 8916, 11001, 18324, 18575,
19330, 19936, 21462, 24887, 27708, 28870, 31331, 37117, 37872, 43673, 45269,
45565, 48128, 49488, 63696, 66359, 68765, 69655, 69813, 70958, 73441, 75468,
76693, 78022, 80469, 81074, 83926, 84313, 85884, 94732, 97918, 109199, 132682,
153970, 159001, 159096, 166200, 167190, 172526, 203033, 209366, 232599, 254839,
298215, 335812, 338047, 346164, 347030, 350391, 373753, 390049, 402758, 419056,
419722, 426483, 473510, 516436, 573911, 602051, 677595, 692417, 760878, 763579,
765369, 856220, 857455, 988386, 988834, 1000077, 1100141, 1208640, 1209160,
1296560, 1298298, 1391236, 1399265, 1409442, 1462069, 1468372, 1475776, 1549503,
1551292, 1556641, 1570053, 1644704, 1724047, 1727078, 1736449, 1819393, 1852417,
1883617, 1908922, 1969031, 1989497, 2075824, 2122750, 2139385, 2145740, 2186876,
2267192, 2292659, 2410660, 2430179, 2503012, 2594221, 2644249, 2699353, 2711578,
2826634, 2905727, 2917899, 2926825, 2928264, 3087834, 3130616, 3133132, 3772561,
4058559, 4446452, 4477283, 4590461, 4666894, 4771861, 4809502, 4868847, 5005004,
5058314, 5092264, 5183777, 5196236, 5411273, 5593249, 5628497, 5873109, 5947922,
6105292, 6240295, 6368175, 6488855, 7137656, 7348233, 7412019, 7524910, 7695694,
7712467, 7743736, 7950337, 8184019, 8226472, 8898541, 9143874, 9157720, 9354098,
9481789, 9552013, 9850428, 10295579, 10468848, 11302343, 11365382, 11483738,
11680912, 12374970, 12556286, 12776962, 12916884, 14004298, 14098912, 14506093,
14567374, 14836520, 15074649, 15868294, 16877210, 16920294, 17261366, 17462813,
17654050, 18496770, 18578171, 19207671, 19240507, 20335751, 20650780, 21510299,
21576474, 22797578, 25967324, 26705809, 26819684, 27315401, 27475767, 27628951,
28697835, 29272369, 29484943, 30322585, 30675304, 31282206, 31359463, 35558509,
36867582, 37016239, 37389204, 40520264, 43289246, 45256091, 45268939, 49846083,
56418326]
438
The first thing I noticed was that there were negative numbers. (lolwut?) These were probably before FAS had the ability to require that you were in cla_done before you joined ambassadors.
The main reason I’m posting about this is because I want to show that it’s really easy to pull group information from FAS and start messing with numbers. Take a look at pydoc fedora.client.fas2 and some other modules inside python-fedora. Looking at numbers can help you figure out what you can do within Fedora to help the project move along. (As for the requiring a certain amount of time as a contributor before becoming an ambassador proposal, I’m not sure where that ended up. I think we determined it was unneeded, but I can’t quite remember.)
Tags: fad na 2010, famna, fas, fedora, python, statistics | Comments Off
![[FSF Associate Member]](/pub/88x31/fsf-member-7305.png)
