invention

After thinking about de-nuctifying the world I've started to investigate creating a Pidgin (formerly Gaim) plugin to control Skype. This means you're still using a closed network, but from an open program, which makes migrating easier (and allows people to try multiple networks from the same interface).

The Skype API docs are pretty good and it looks like the API can support at least controlling status, sending and receiving text messages, and making and receiving phone calls. So in principle a Skype plugin for Pidgin is a possibility.

There are a few docs on writing libpurple plugins for Pidgin but less comprehensively - the author of the Sametime plugin or the guy working on a MySpaceIM plugin probably know exactly how it works (the MySpaceIM plugin is a Google Summer of Code project that fits in with my idea very nicely). There is a C Plugin HOWTO in the source code which should get things started. Some ideas in the Adium Skype plugin bug as well.

On the way I read Silver Needle in the Skype which is an article on reverse-engineering Skype and using it - very interesting tech reading... but a shame they haven't made the code available

Well since it's 8 days till we leave Cape Town and head to Japan I'm not planning to actually do anything about this, but thought I'd write it up so I don't lose the links...

PS This blog doesn't currently have a comment mechanism so email comments are welcome to davidf at sjsoft dot com.

Well seeing as Google has release a Code Search on open source code, I thought I would put it to good use: searching for pootle in filenames ending with .po helps you to quickly find some projects that are using Pootle to edit their PO files, and you can even see which version

Another nice way of showing how Pootle is gaining traction...

There's been lots of discussion recently about Pootle on the debian lists, the direction it should take, whether to use a database to store translations, etc... It may be a nice idea to add a database as one of the backend options but keeping with handling the complexity of translation files has been important. Seems like a good consensus to this effect is emerging; it's nice to have more interest in the project.

I've tried to keep coding rather than get too drawn into the discussion, and this is what I've been up to the last week:

Pootle: Mostly ongoing architecture work, feels like we're heading towards a good structure...

• Wrote up discussion on putting metadata into a relational database
• Base class migration work on the Pootle-locking-branch: made pootleunit and pootlefile encapsulate a pounit and pofile respectively rather than inheriting from them. This will make it possible to use other storage classes (although there is still more cleaning up needed where we use PO-specific methods).
• Participated (minimally) in the debian discussions on direction of Pootle and using databases
• Fixing some minor bugs in the webserver etc

Translate Toolkit: Focused on escaping being correct and using the base classes

• Sorted out escaping in dtds - it doesn't exist. So we will no longer support \' or \n having a special meaning in dtd files, being translated to \n in a PO file etc
• Fixed various failing tests for 0.9 (including fixing some modifications we were doing to the minidom XML library)
• Added some tests for dtd quoting to HEAD - currently we allow opening a quote again after it is shut (<!ENTITY name "first part""second part">) which is invalid according to the spec - just for correctness, haven't hit any errors here
• Making properties to PO conversion use base class API more, sorting out escaping there
• Discussion with Axel from Mozilla - for 0.9 the Mozilla .properties files will be output in proper UTF-8, not with \uNNNN escaping (although that will still be understood as input. See bug 114
• Made dtd classes inherit from base classes - could be made cleaner, but at least the API is implemented now

Trying to proxy network traffic through my wireless network on my laptop which is unfortunately running Windows... (oh, the irony)... these are handy hinters:

First, use ssh -D to create a SOCKS proxy. That's fairly easy. Use tsocks to make any application use the proxy. (Or, try and battle with why GAIM isn't listening to the Gnome network preferences, then get fed up and run it through tsocksify).

Next, its helpful to have a caching DNS server. I finally found a good one for Windows that actually works natively (via cygwin) and doesn't crash: Posadis. Simply start it up and it functions as a DNS cache. They also include some basic DNS querying tools that are missing from the Windows command line.

I also tried using dproxy which is much lighter but although it compiles in cygwin fine it produces various errors (misformed packets sent in response to queries etc).

Finally you need to let the Windows machine know to use the wireless network as its default route rather than the LAN. To do this follow this article's recommendations to adjust the Automatic metric under Advanced TCP/IP settings for each network connection (lower is higher priority) otherwise Windows XP will always select the LAN as the default network. (You can check which is default by saying 'route print' and seeing what the default gateway is).

This latest Sourceforge CVS outage has been most unfortunate. We tried to do a bug day aroud the one the week before, which was also unfortunate... Now for almost a week we haven't had any CVS access, and we had planned a release in that time of Pootle and the translate toolkit.

Good thing I had a script getting the CVS tarballs as we could set that up as an alternate (read-only) root. I then set up a different directory I used off that root and maintained a queue of patches... which I applied this morning as follows:

for f in `ls offline-patches/*.patch -t -r` do patch -p0 < $f echo $f for g in `lsdiff $f` do grep `basename "$g"` `dirname "$g"`/CVS/Entries || cvs add $g done cvs-commit `lsdiff $f` done

I've now basically completed converting Pootle to work using Kid templates which makes the code a lot cleaner (was using an old object-based HTML construction set).

A few things to sort out (spell-checking system), but it mostly works beautifully. The UI is exactly the same as before, but we should now be able to improve it without messing around with the Python code.

This is all on the Pootle-kid-conversion branch, and you'll need at least version 0.7.6 of jToolkit to run it. Other requirements have increased - Kid requires ElementTree too. This is now documented in the pootleadmin page on the wiki.

I've been wanting to get this going for ages, so it's nice to finally have it done.

Something I have been promising for many months has finally happened... We've been going backwards and forwards with so-called "release candidates" for 0.8 that were really betas or new versions, I managed eight!

The combined effect of this and doing the last release candidate when Pavel was too busy with OOo 2.0.2 is that nobody's complained about the last one.

But we did have a bug day, add lots of tests and fix lots of bugs. There's a new branch for the 0.8 series if any bug fixes are needed, so please report if you find any problems.

New work on the HEAD branch will focus on the wordforge project.

This is just a quick announcement for those interested as we've got another power failure - the Western Cape is having fun with those at the moment

We had a great Bug Day for the Translate Toolkit.

I wrote little scripts to track bug and test status on different machines, giving these results:

Bug resolution

10:20  NEW 23  ASSIGNED 2  RESOLVED 21  VERIFIED 3  TOTAL 50
10:40  NEW 22  ASSIGNED 2  RESOLVED 22  VERIFIED 3  TOTAL 50
11:00  NEW 21  ASSIGNED 2  RESOLVED 23  VERIFIED 3  TOTAL 50
11:20  NEW 19  ASSIGNED 1  RESOLVED 26  VERIFIED 3  TOTAL 50
12:20  NEW 17  ASSIGNED 1  RESOLVED 28  VERIFIED 3  TOTAL 50
14:20  NEW 17  ASSIGNED 1  RESOLVED 27  VERIFIED 3  TOTAL 50
14:40  NEW 15  ASSIGNED 1  RESOLVED 29  VERIFIED 3  TOTAL 50

So we basically resolved 14 bugs, nice progress.

Test status

09:20 83 passed, 2 failed
10:00 87 passed, 2 failed
10:20 90 passed, 1 failed
10:40 93 passed
11:00 92 passed, 1 failed
11:20 93 passed
11:40 95 passed
17:20 95 passed
17:32 99 passed, 1 failed

So we added 15 new tests, all except one of which pass (Dwayne added it later), and hopefully we can fix that soon.

A lot of these problems have been hanging around for ages, and having a bug day was a nice way to get everyone together and finish them off. Thanks to all who participated!

The bug day was called Long Walk to Freedom but I concluded we made a Great Leap Forward :-). The main point behind all of this is to release version 0.8 (which we've branched for) as a stable version that everyone's happy with, and then move on with other plans... In the past we've been doing too much normal development work in between "release candidates", with the result that things have been broken, which means this has dribbled on for months. Unit tests are a nice aid to making sure we don't break things.

Once the final test is fixed, I'll do a release candidate, make sure everyone's happy, then release 0.8 (I hope I don't find myself reading this in a few months time without this done!)

The new things we're moving on to are encapsulated in the WordForge project, and I'm quite excited about them. It should make a big difference to open source localisation projects in the future.

Some interesting articles to read when I came back from holiday. It seems Swahili free software stuff is moving forward in lots of fronts. Tomorrow I'm going up to Pretoria to train some of the KiLinux team on using ooo-build for building localized OpenOffice.org.

Kamusi Project Internet Living Swahili Dictionary is a good read, this is a great project.

Tanzanian government uses OSS for localisation shows how the government is realising the importance of free software.

Also found this article on the Guardian on Owning ideas a good read - not neccessarily technically right in everything, but has some good insight and is accessible to people unfamiliar with the debates... e.g. Patenting ideas rewards failure and makes success more difficult. ... This is madness. Ideas aren't things. They're much more valuable than that

Looks like Firefox 1.5 is going to be released shortly, thus making this a fun week Most of the issues in the translate toolkit for Firefox 1.5 have been resolved, there are still some that I'm working on but people seem able to produce their translations OK using it.

Well I clearly didn't manage to release 0.8, there has been a fair bit of patching and adding tests (particularly ensuring round-trip maintenance of source files in Mozilla, David Farning has been filing many bugs). And quite a few bugs in our bugzilla have been fixed.

I'm off on a week-long break so I've released 0.8rc5 (with a Windows build that the snapshots don't have) for anyone wanting to use the latest code.

I did my first real patch for OpenOffice.org: Support for multiscreen displays in Impress is now beginning to work (but more work to do as you'll see in the bug).

I have finally gotten round to fixing up the Translate Toolkit so that moz2po / po2moz can be properly used with language pack xpis from the Firefox 1.5 series.

  moz2po -i en-US.xpi -o firefox-pot/ -P
  # convert to .po files, edit / merge with other .po files
  po2moz -t en-US.xpi -i firefox-po-${lang}/ -o ${lang}.xpi -l ${lang}

And that's literally all that's needed! The next snapshot should work, and I'll aim for a release by Monday.

After being busy with lots of behind the scenes things and other work, I feel like I'm finally making progress on the following:

Building OpenOffice.org on Windows - I've been working with Michael Meeks to get ooo-build working nicely on Windows with the 2.0 series of OpenOffice.org builds, and my builds are running quite happily now. This will make it easier to manage patches, updating between versions etc (compared to using the standard OpenOffice.org build system). I plan to update the Windows building page on the go-oo Wiki as neccessary, some of my notes from build problems are in openofficebuild on the translate wiki.

Pootle - have fixed one or two small bugs and hope to keep on doing so, as well as planning to add better searching support so people can file bugs or suggestions for translations more easily.

Translate Toolkit - actually integrated some patches (from Matthias Klose) - there are quite a few more on bugs.wordforge.org sitting around for me to process, but its nice to be off the ground.

South African OpenOffice.org - which is what the build work above is for: I am now building OOo 2.0rc2 successfully, need to move to rc3, incorporate the latest changes, and get our branding, spellcheckers etc in.

Have managed to fix up the Pootle registration process, and set up a Pootle mailing list for any discussion.

Also found that Pootle was being Spidered by search engines, so added a robots.txt file as that was placing a lot of load on the server.

Now the TODO list is getting bigger, so hopefully some other developers will come on board (hint, hint). My next priorities are implementing PO uploading and an interface for assigning translations

In the mean time, some other related web sites that we can take ideas from:

• Gnome CVS has a nice folder viewing style we could borrow ideas from.
• KDE Translators Centre has a nice index of languages that you can sort in different ways
• RedHat's i18n-status page has a nice system for showsing statistics, assigning translations / letting people "Take" them
• Mandrake's l10n statistics are also interesting

In Tanzania for the training camp. Has been pretty interesting discovering how the Swahili Localization Project works.

They have a pretty thorough team incorporating top experts on Swahili from Universities and research institutions and tehnical people. They have worked hard on generating a glossary first before translating software which is a good approach. They are just beginning to translate OpenOffice.org and are finding many of the same frustrations and challenges we found when starting to get into this.

Have had good talks about the localization process in a third-world context with Alberto and Louise and have persuaded them to learn Python, so hopefully they will be able to contribute code... (this is actually a hint in case they read my blog)

Bandwidth is better than I expected (we are at the university computing centre) but is a real limitation when you're trying to localize projects that have high bandwidth expectations, and power goes off from time to time and you need to start again ...

Did a session yesterday morning on how to use our tools, fairly basic, explained the procedure for OpenOffice.org and Mozilla using oo2po/po2oo and moz2po/po2moz, did a demo.


me doing my talk, complete with diagram

Over the last week I've started development on Pootle, a simple web portal for localization... named after a Flumps character thanks to Danielle's suggestion.

Its been really fun and the progress has been quick. Once its workable we'll try get a public implementation somewhere. Just needs a few additions (user registration, etc).

I just use PO files on the disk and aggregate stats on them to make it nice and fast. Hopefully will use it in Tanzania for our upcoming training course

I didn't have a laptop at OOoCon 2004 and didn't manage to write anything in detail then, and have been busy since, so this is my executive summary...

Was fantastic meeting loads of different people, lots of whom I've been in contact with before but not met in real life... and attending some great sessions. For me the discussions afterwards were often the best part. Was able to make some contributions about how difficult the localization process currently is, and what we could do to improve it. Also a lot of discussion on how to make it easier for developers / others to contribute to OpenOffice.org.

Javier Sola who runs the Khmer project in Cambodia: he uses our translation tools and says that in fact they're the only viable way to translate Mozilla into Khmer and other languages that use Indic scripts. Was fantastic to see the depth and breadth of the project - translated Firefox and Thunderbird, nearly finished OpenOffice.org, has developed a public domain Khmer Unicode font, and is writing lots of documentation on the localization process, using our tools as a basis. Were staying in the same place and on the first afternoon, managed to bash out a final version of moz2po / po2moz so that you can do a round-trip from xpi to po and back again.

Pavel Jan�k has been absolutely fantastic doing our OpenOffice.org builds for us, so it was nice to meet him finally and say thank you, though I'm still not sure he realizes what a help it is. Since the conference we're now working on getting our tools working with OOo 2.0 and generating PO files automatically for all languages...

Various people from Sun were there, including Eike Rathke, Joerg Barfurth, Ingo Schmidt, and others... Was great to hear about new features for OOo 2.0, especially the new installers and how they will be able to produce language packs etc.

Michael Meeks did a great session on building the developer community, at which I signed a JCA, thus making all the patches I'd previously had integrated legal

Dan Williams did a talk looking at various possibilities of migrating the OpenOffice.org resource system to be a loadable-on-demand XML based system, possibly with caching. This would be great and could make localization a lot simpler by splitting out the text strings into a separate file for each language that could be replaced on demand.

Caolan McNamara ran through debugging/profiling techniques for OpenOffice.org and how to make them work as God intended.

Chris Halls, the Debian maintainer for OpenOffice.org, suggested ways to simplify the build process - breaking down the source into manageable parts, not including external libraries, providing the build tools as binaries, etc. Man, I want all that yesterday!

Had a great discussion at a Chinese restaurant with cph and Jacqueline McNally on ways to make it easier for new developers to get involved. Going to have READMEs for each module - yay. Also suggested having janitorial / TODO lists like the Wine project does.

All in all, was a fantastic conference, and really glad I was able to go.