Gentoo Presentation preparation

For 10 years, there now is a meeting called Lugcamp, where Linux geeks in and around germany meet. As we have 10th anniversary here, some special location was needed. So the LUGA, which was the LUG hosting it this year, aquired a youth hostel in AllgΓ€u.

Smaller talks have some kind of tradidion at these camps, namingly being heavily dynamic. You start preparing a talk about an hour or two before it starts.

When the first talk was about start, people noticed a small bug… Missing canvas. 2 people were ordered to pick one up but they did not arrive yet, so the dynamic part this time would be actually having some kind of canwas.

Here’s a pic of what we came up with πŸ˜‰
Canvas

All in all it’s fun every year. If you speak german, you can get some additional info about it at LUG Camp 2009 site

difficulties with proxy maintenance

Some of us devs do so-called proxy maintenance with some users who just like to get 2-3 packages updated in our tree, yet for them it hardly makes sense to become dev.

I have always been a fan of this, as it involves more people and I have nothing to do except reviewing patches and committing that. 5-10 packages are easy to handle for any dev, so no big drama.

Though when I joined our php team as commitmonkey, things started to become a bit hairy. There are like 200 packages who want regular revisit. Soonish there is a missing file on commit or whatever lil’ fault you can think of as grepping changes from one dir and applying them to another is not that userfriendly when cvs is part of that game.

So some really bad scripts magically appeared to do that task yet they failed as well sometimes due to not recognizing same versions and whatnot. End of story: A GUI was needed to get a visual impression about what is going on there. Doing copy of selected files and run some commands after the copy (like cvs add and whatnot).

2 1/2 days of coding later, here it is πŸ™‚

It’s part of the upcoming maintainer-helper release (yah I know, I was slacking at that project), so keep fingers tight or grep the live version πŸ™‚

git-svn, copies and empty dirs

Up to now, when working on sunrise with git-svn, I faced two minor issues that it didn’t handle nicely.

a) When copying stuff from sunrise/ to reviewed/ so that it goes out via layman, the file was added although it was merged.

b) It left empty dirs behind when I kicked an app (as it went over to gentoo-x86)

After reading a bunch of mailinglists and forums posts, I finally had a working solution. Just add these lines to .git/config file:

[diff]
renames = copies
[svn]
findcopiesharder = 1
rmdir = 1

to view packages or not to view packages — that is, urm, a gentoo problem?

So you read it in the title. Yes it sounds weird, by intention. When our primary site for package viewing went down some time ago, I didn’t notice much. It was more a good friend of mine asking me if I know when it will be back. Okay, it was down for three days so no big deal if something serious happened. You know the rest of the story or can read it up on the bug (just don’t comment there unless real help offered).

But why am I writing about this here? Well, also some time ago, araujo and me started something called gui project. Main target was obviously getting some nice gui for our day-to-day tasks. I don’t really know where (maybe some mailinglist) it came to my ears that our obviously loved packages site was down more a month. So hey, we’re a gui project, I more or less have finished my previous tinderbox thingy (just minor stuff open). So some spare time available. I never did a webpage yet with a python based framework so I decided to give some of them a try and if I would find a nice and not bloated one, this could be worth a try.

Two days of trying different ones later, I finally got settled with cherrypy and genshi (there’s a nice tutorial on the genshi homepage if you’re curious now)..

After some thinking about databases and stuff, finally got a db generator going resulting in data useful for such a site. That’s the moment where I started to git the code and let the show begin. In the meantime I was on holiday for more than a week so you might notice some inactivity in the timeline. Anyway, at least startpage and some filtering works. See http://tinyurl.com/2fq9dw for working features of the live version at http://packages.gentooext.net

It’s a starting project, mind you, though basics are working now, more stuff to come. If you have questions or comments, feel free to join irc.freenode.net #gentoo-guis , cookies are on topic πŸ˜‰

Tinderbox up and running

To be honest, I didn’t expect the thing to last that long though it was worth it. As the topic suggests, the beast is working now.

While the actual build script was done in one day, getting all involved things to play nice and friendly was really a hard job. Involved were cachings, optimizations for different VCSs and many minor corner cases.
Facing facts I’m abusing a system which was designed to fetch source from some repository then run tests on it and if multiple requests come in, just work on latest one. This is partially what I want and the other part is the opposite. So I had to dig deep into BuildBot to solve all outstanding issues.

Job management was more like coding top down (no really hard task after all) though needed tweaks along the road as well.

But as I said it was worth it. The thing has no dark corners anymore for me πŸ˜‰

And: there are pre-stable tests now for the incoming requests πŸ™‚

Tinderbox Screenshot

Note for interested people:
Repo of it has moved to dev.gentoo.org/~jokey/tinderbox-bzr
URL of the BuildBot Status Page: http://buildbot.gentooext.net/

stable requests with a special outcome

Back on Tuesday I finally got into the mental feeling of holidays (yey, first holidays in one and a half year) and decided to spend the whole day for Gentoo work. So what comes to mind first? Check for open bugs..

As member of x86 arch team, I decided to start there. After open the site it felt a bit like bad news. ~30 requests being normal, 67 that day. :crazy:
So either there were just some slackers over the weekend or some herd woke up and filed requests like drinking coffee. Neither was the case though.
Only possible solution: boot up all boxes and have them compile…

All boxes.. my pc, my desktop,my errrrr… ok that’s it. (As weird as it may sound to people who’ve never done AT stuff, even with two devices it’s much to do). Our “Gentoo Arch Tester Tool” (aka gatt) does a good job there by guessing packages just via bugzie ids though it’s still a lot of work and quite a lot of time. Only to find out $package doesn’t even build. Worked through ~20 bugs (and 3 failures) in about 8 hours. Not that efficient from what it felt like. πŸ™„
Luckily the php guys needed some help on getting stuff over from the overlay to portage and the day had a positive ending.

But I would like a tool that does a build on request to verify it atleast compiles on a default stable profile, maybe with a slightly polished make.conf. A quick look around returned only a “hey, let’s build whole portage tree” script and some non-working or “I run emerge via cron” scripts. Both won’t clearly do the job here.
Result: It’s open source environment — “Don’t moan, code yourself”.
First starting as a base on the build whole tree script I mentioned but then it quickly turned out that it wouldn’t do the job for me. So more or less, inspired by the stuff seen in that script, start coding from scratch. πŸ˜‰

After like 12 hours of intensive coding work on wednesday evening I finally had this thing done: Tinderbox Output
What does it do?

  1. Update system and world
  2. clean out unneeded build and non-world packages (world only contains 4 pkgs there)
  3. calculate minimum requirements for package X
  4. build those and make binpkgs of them for later use

What are the next steps? Obviously the request handling. Though another thing comes to mind: if there is more than one request at a time, parallel building on more than one box would be cool. Also there has do be at least a minimalistic bit of status overview.
Tried some build environments for the job, finally good old BuildBot was chosen as candidate. The setup is easy, all of the build boxes already have Python on it so not much overhead and I know the lang so can extend it if needed later on.

After reading through some docs to get the deeper know-how about how to distribute a buildjob, the thing looked like this:

buildbot-overview buildbot-overview

RL wanted some attention today so I didn’t put that much energy into it. Tomorrow will be our next LUG event so I think Sunday will be the time for more steps towards a request tinderbox (as only request handling stuff is left now).

If you want the (not yet documented) tinderbox script, “bzr get http://dev.gentooexperimental.org/~jokey/tinderbox/”

New virtualization around… And GPL’ed

Okay, today there was a nice announcement on heise… “Open-source competitor takes on VMware and Co.”

So we have an open-source virtualizer now? Yes we do πŸ™‚ The test on the page showed that it’s only a bit slower compared to the other mentioned virtualizer, so I gave it a run and it worked okay from the first moment. Stats indicated that there were some binaries around but we don’t like binaries if we can get the source, right? πŸ˜‰

I’ve put a masked live ebuild in the tree and in fact we’re the first distro that has virtualbox in the official repository. So give it a run if you want to see the power of open source πŸ™‚
There are good docs available on the homepage but if everything fails, you can stop by in #vbox on freenode (upstream irc channel)

G15 working (again) YEY

So it has been a pretty long time since I blogged something but this is something important πŸ˜€

As I’m back from a big Christmas party of one of our great local LUGs here, I’m happy to announce that being two days late, we have libg15-1.1.1 in the tree. For those who are thinking “why is that so important?” here’s the reason:

For all those people who have these nice G15 keyboards from Logitech, as of today we have back working G and M keys with latest kernels :). The keys stopped working starting with kernel 2.6.18 as it introduced some USB API rewrite. With some (detailed) input from some of our users and developers, we helped the upstream maintainer to track the issue down and it’s fixed now.

So a big THANK YOU to all involved to make these fancy G15 keyboards working with linux.

LugCamp is over now :(

One of the craziest but coolest events ended on Sunday with a big tombola and something called “Improvortag”, some nonsense but really funny speeches.

For those of you that don’t know about it, it is an event where 4 things are mixed together:
– Meeting other Lug’s members
– Having some speeches (only some, not the whole 4 days)
– One local event (here it was visiting brewery and an alpine dairy)
– Fun fun fun….
It really has a non-commercial/non-advertising trait, and feels more like a big LUG-LAN things than anything else. As an addon you have a “coffee flatrate” and get at least one meal a day for free.

The announced Open Radio live event didn’t take place because of some problems with the dialup internet connection we were offered there. We’re really sad about that but we still had a lot of fun without it.

Someone once said “geeks always find other geeks in random places”. This should be sed s/geeks/devs/ here πŸ˜‰ I met genstef there and we took a nice picture with The Alps in background πŸ™‚

genstef (left) and jokey (right)
left: genstef, right: jokey

I’m really looking forward to LugCamp @ Father’s Day 2007 as these ~150 people there are just like a big family πŸ˜€