So, apparently, the recent Gentoo/Seeds flare up and my own thoughts about it have sparked a little bit of speculation out there.
There’s been an inevitable (retrospectively) culture shift happening in Gentoo over the last few years. Gentoo has been steadily moving towards Debianisation. I know there are people out there who will read that and go “yeah? so what, Debian’s great.” And to you I say, “yes, Debian is great, but it’s not Gentoo. Debian is great because it was able to spawn things like Knoppix and Ubuntu.” Debian is also great because you can pick a CD (stable please) and throw it onto a server (not too new please) and be relatively sure that things will work as expected when you install it. Becoming a Debian developer is not great (from the anectdotes I’ve heard). There’s simply too much bureaucracy, too much of a waiting period, and too much of a niche you have to fill to become one.
Similarly, the dark clouds over gentoo really boil down to how fun it is(n’t) to be a Gentoo developer these days.
Even before Daniel left Gentoo, the culture shift had started. There was a small (and vocal) minority — they always tend to be hugely vocal — that was pushing the line that Daniel had “too much power.” Three years later, nobody seems to have enough power. The buck doesn’t stop, it just keeps getting passed. The council vows this year to make some changes, but honestly, the closest thing to a buck-stop we’ve had in the last few years is SpanKY. He happily lays the smack down and issues finalities. And I’ll tell you this: we need that in developer-land.
Part of the reason for the Seeds throw up (and prior to that, the overlays massacre) is that there is no focus for the project as a whole. We’re a multiheaded snake (where each head thinks the others a nuisance) trying to go in as many different directions. There’s no focal point where projects as a whole can point toward; there are no goals to which we can aim.
Now that the board of trustees is beginning to slim down (from an unhealthy and obese 13 people, to a slimmer 5), I hope that sees more of a change. I hope that the council will be a lot more proactive.
Probably the healthiest thing would be for Gentoo to die and then re-emerge (natch!) from its ashes. By this I mean that the project should get rid of all of its cruft. Start with people: get rid of the people who do not add any value to the project, people who have half-baked ideas with no follow-through and a list of unresolved bugs a mile long. Then, get rid of packages. Everything that has a bug to maintainer-wanted should just go. Then find everything else that isn’t maintained and get it out. Let the overlays handle them.
The ideal model would be to slim gentoo down to just a handful of developers working on the core system (base system, compilers, userlands, some editors, livecds, installers, etc). Get everything else into an overlay. There can be official overlays and unofficial overlays with a defined set of standards that determine what becomes deemed “an official overlay.” Those ones get mentioned in make.conf.example as a source of packages. Then you’d have an X overlay, a gnome overlay, a kde overlay, a java overlay, a clustering overlay, a science overlay, a graphics overlay, etc ad nauseum.
This would be a painful process, and it would create very bad blood between a lot of folks. It would also cleanse Gentoo.
There’s a lot more I could say on the subject, but I’ll let you, dear readers, say some stuff first. Feel free to flame me, insult me, whatever, just please put one constructive thing in your responses, that’s all I ask. I don’t ever censor comments (except for the spam-get-this-rolex type ones), so do what you do.
Kulleen, out.
as a long time user, i always felt that gentoo was heading in that direction due to the ease of use of the portage system in creating meta distributions
now we’ll actually be able to create them 🙂
> everything else into an overlay
You’re missing something by saying that. It’s not possible to keep dependencies between independent overlays coherent. By doing so you’d throw a number of problems out of Gentoo (but definitely only half of the pressing ones). It wouldn’t solve a single problem, but would create even more.
It would be indeed the end of Gentoo as a (usable) distro. You’re probably talking about the final coffin nail here.
Why is it not possible? Independent overlays should be encouraged to co-operate, surely? And, if there are issues with them, then why not make coherence one of the standards for acceptance into being recognised as an official overlay?
Gentoo isn’t a distro: I think that’s my point. Gentoo is a meta-distro — a vehicle and a platform to create your own distro. The final nail in the coffin is a dark analogy, and in some ways you might be right: it would be the end of a certain way of doing things — a paradigm shift.
I do believe we need to get back to the basics and focus on the core features Gentoo should sport (to please the users and please ourselves that we are making a top-notch distro). But I can see some future problems with pushing things out to overlays.
What happens when two packages in two separate overlays require the same library which is unmaintained? The overlay maintainers will have to collaborate inorder to not duplicate work or craft greatly-differing ebuilds. The other issue is how will portage handle a dependency of a package that is available in multiple overlays? Which overlay should be favored?
Now I’m not too knowledgable on the internal workings of portage, but those are the initial issues that come to my mind.
On the same note as overlays, I think it’s time to encourage 3rd party Gentoo specific projects. I keep running into underground Gentoo related projects that are eclipsed by Gentoo approved work. I think the promotion of such work would help increase diversity of the Gentoo code-space.
I also believe we should not panic here and merely accept that the sky is falling on Gentoo. We should relax, think about what needs to happen and then make it so (which of course is happening now if you look at the posts on planet.gentoo.org)
I know the idea of gentoo as a meta-distribution but I do not agree with it, from a user point of view.
From a developer point of view it’s great: “I’d like to have a new distribution for this or that, why not use gentoo because it already has everything in place for my new distro.” Great.
But think of the (desktop)users. A meta-distribution is not a distribution. Users want to rely on a distribution for distributing things. Like software they want. But a metadistribution is not centered around things users want, it seems more centered around things developers want. Embedded gentoo: nice idea, but how many users?
Users just want a working desktop, but e.g. KDE currently is not up to date in Gentoo. Not in stable. How many distributions are in this situation? I think a few. If developers are going to be divided among more projects then resources will be scarce. Gentoo will just group linux distro developers, it won’t be a distribution.
No, I don’t think you are going to have enough developers to manage all the overlays. Gentoo would litterally break into pieces.
My personal view on better solutions would be refocusing Gentoo. Step away from the meta-distribution idea and go for something general. What do you want to become with your distribution?
A desktop distribution? Fine, but then throw out older packages. There’s no use in supporting all versions of KDE and Gnome. Try going for the latest version of both of them, and one older version. (This is an idea used in ArchLinux which I really like.)
A server distribution? Fine, but then stop putting time in the desktop part. Gentoo has good init scripts, improve those and start putting more effort in the SELinux part.
Gentoo should wonder why users love or loved it. If 90% of the users loved it for being a good desktop distribution, there’s no good in starting with the FreeBSD port. (Altough it’s cool, that’s not the point here.) My feeling is that the project has passed its dotcom-bubble period. The clouds are starting to disappear, see how many developers are left and focus with them.
Bart
Bart,
So the idea then is that this planetary model for gentoo could lead to desktop distributions quite easily. As long as overlays are high quality, nothing stops anyone from putting together desktop distributions, embedded distributions, hardened distributions, etc. It even makes Stuart’s idea of vertical markets being targetted (see donnie’s post) that much closer to reality (ie, the Seeds thing).
In fact, I agree with you about this: a single overlay should be a single branch — ie, stable KDE is one overlay, testing KDE is another. And always, guaranteed to be so, etc.
I only have one comment i would like to add to the table of if gentoo is going in the right direction or not. I have found no other disto that has the ability to teach anyone the base linux kernel, the base bsd system or give the end-user the ability to design or build what they want. For the past two years i have used gentoo, freebsd, openbsd, and all of the other distros out there i could get my hands on, and personally I think gentoo is way ahead of all others. I have compiled well over 1300 plus source programs stable and unstable in less than five days with only one pause. Now you find me another distro that can take so many developer codes and compile them without failure. gentoo may be more advanced than other distros and sometimes harder to work with, but it is like a porche not a pinto. If that is what the masses want then fine, but if you read all the forums they all point out the fact that it is not the programs but the people installing them and using them as the cause of issues. some programs don’t copile is the issue I see all the time in the forums, but it is not the programs exclusively, it is the end-user. they all work for the most part on other distros, even thought they strip them down. With gentoo people need to understand the system and that dependancies need to be put in, and in what particular order in order to compile and work. when i took pascal in school i learned that in order to comile and run any program your code had to be 100% correct otherwise it would not run. nothing has changed. Gentoo gives you the power to do it or not. so , the secret is learn more and things will be better understood, because without all of the end-users learning there would be no gentoo. I like to learn fast so that is why i use gentoo.
Yeah, I’m pretty sure this would drive users away in droves. I’m pretty sure if this happened *I* would jump ship, but maybe not; layman has come a long way.
Disadvantges:
Cross-overlay dependency is even harder to get right and manage then single-tree dependency. Portage (and other tools) have dependency problems now, this will end up highlighting them. QA in Gentoo has it’s ups and downs, I just don’t think there’s enough effort consistently available to maintain 1/2 dozen separate but smaller trees.
More user effort. It’s hard enough to find the package you need in portage right now; emerge –search sucks so much we have at least two replacements in the tree. Now the user is going to have to search a list of official and unoffical overlays and hope one of them has their package. Now, what overlay do I add to get that KDE application for generating and viewing performance graphs of BLAS clusters (unbeknownst to me, it’s written in KDE binding for java so it’s in the java overlay).
Advantages:
Has anybody considered this might be the natural progression for a mature meta-distro? There are a lot of distros that have come and gone, and then there’s Debian: The rock of Gibraltar among distributions. Keep in mind that Debian is still around because of what they did right. While I don’t want Gentoo to become Debian, I think we could take a lesson or two from them.
Cooperation is what we do already in the tree – what doesn’t work is that people keep adding packages, but do not clean out or care for the unmaintained stuff. I’ve been already flamed for pointing out possible ways to condemn the tree polluting, months ago. It’s a matter of developer behaviour and reasonable tree_size/#developer relation, imho.
Why it’s not possible to fell the tree?! Of course it’s possible. But the overhead of keeping it all together would be a lot higher, since you’d need to care that the sets the overlays represent don’t overlap – otherwise you’d face weird keywording/stabilization related problems. Also every dev needed to use every overlay as he wouldn’t notice, if his bumped lib would break another overlay. Not to speak about stuff to be profile masked or to be added to use.{,local.}desc., etc.. Ask the security team what it thinks about not being able to tell the user, which package from which overlay is vulnerable or not. Last but not least: It wouldn’t mean, the overlays would be maintained better. The tree is too interdependent. You can pluck the leaves or even carefully seperate a single knot (and everything what depends on it), but the overall situation would be _much_ worse, if we try to bind a bunch of olive, oak, … limbs together and tell everyone that it is a living tree.
A venesection doesn’t help against pestilance, even if the quack advises. ;p
That doesn’t mean that I disagree about the need to cut off unmaintained stuff and take more care about what we allow to get into the tree. Also that we need to be more clear about what Gentoo does want to provide and what the self-chosen constraints should be (within the gentoo.org umbrella). We sincerly need that and I expect much from the new council in this regard. Both in the sense of organizing a strictly moderated discussion to gather views and ideas as well as decisions.
Let me add a bit more: Bad press and user feedback about “rushed” releases because of an installer that kills user partitions, been shipped with 2006.0 and (known to be broken!) 2006.1. This is totally unrelated to the tree status, but killing user data is the worst we can do. I don’t understand that we ship something that’s known to be that broken. This would have happened – even with a stump, instead a tree.
Posting this comment on behalf of: Alexandre Rostovtsev (tetromino) whose email address was rejected by b2evolution (why, ffs), and so emailed it to me instead:
Putting everything in overlays is a horribly bad idea. First, because
Gentoo’s massive package tree is one of the distro’s major selling
points. One of the main, if not THE main, reason I use Gentoo is that
I can just type “emerge random-package-I-just-read-about”, and 9 times
out of 10 it will emerge. But if I needed to hunt down a dozen
overlays for all the packages I use, then I might as well switch to
Fedora.
The second reason is that the dependency situation on the Linux
desktop has gotten much hairier in recent years. Fundamental packages
regularly break their previous APIs. As a result, you need “flag days”
to switch everything over to a new version of dbus, hal, ffmpeg and
the like. In the current Gentoo dev process, this is handled by
setting a bug in bugzilla to track all the packages that need patching
or testing with the new version. But now imagine the world where
everything is an overlay. Overlay authors obviously can’t use the main
Gentoo bugzilla for their own trees, since that would breeed massive
confusion. So they don’t find out about the flag day, their overlay
breaks, and quarter of Gentoo users (the ones who subscribe to the
overlay) freak out. Alternatively, they update the fundamental package
themselves before the flag day because they don’t realize all the side
effects, and again users freak out. Things like this are hard enough
to get right in a single tree, with a single bugzilla, and a single
mailing list. But when a typical user is running a dozen overlaid
trees, developed by different teams in very loose communication, the
“Gentoo desktop” will likely be broken at any one moment in time.
A third reason is the duplication of effort. When most things are in
one tree, you can invest in improving and adding eclasses, qa tools,
various system management utilities. For instance, consider the new
Java system. Would such a changeover have been at all possible if
popular Java packages had been split up among multiple overlays? I
doubt it. In the “everything is an overlay” model, eventually overlay
authors will quickly write (buggy) eclasses that other overlays are
already working on, in an incompatible manner, and in general
studiously reinvent square-wheeled bycicles.
In short: yes, the tree might need a small bit of pruning. However,
forcing people to turn to overlays for important stuff will KILL the
distro.
What about security? I would not install anything not official on a productive server.
I hope as much people as needed disagree to this idea, that’s not the way to go. Better get more developers maintaining “their” packages, even if it’s only one at the start and remove only packages from the tree that have security bugs.
From a newbs point of view, I’ve tried Debian, and I don’t care how popular it is, once you get into adding overlays, and making the average joe user run around looking for something, you screw with the beauty of portage. I maybe reading all of this wrong, but the thought of becoming “Debianlike” almost throws me into convulsions. Don’t forget…windows is popular too…and I’d toss my box out of the window if linux headed that direction.
Please don’t kill what makes Gentoo great.
As a reply to Timo; do you enjoy emerging a package only to find it is broken, or out of date, or has a bad digest, or has bugs? I do not like any of these things; this is why I remove packages from the tree (it is; my only real useful contribution as a developer these days). As much as I love keeping old packages around (cause saying the tree has 8000 packages in it is cool) they are NOT all maintained; hundreds of packages lie dead from people who retired years ago: packages with no upstream, open bugs, no revbumps for three years, etc…
I could probably prune a thousand packages from the tree and 98% of users would never notice. The 2% of users who do can get their digs ar Sunrise IMHO.
Hi,
I think to concentrate all work in gentoo as being a one and only “meta distribution” is the wrong way.
I think problems will then become worse. I think its problem are, that actually gentoo fritters itself on dozens of project.
Do one thing right and stop wasting your time on various frontiers.
For me portage and the user documentation was and is the key to gentoos success. If now and in the future more packages won’t emerge and I start wasting my time because gentoo breaks sooner or later, I will stop using it.
Quality is the key to its future.
Being gentoo only the “meta distribution” is not the answer.
It is the Question. And I would say No.