Avoid the spam on the gentoo.org mail with procmail

I use it since a long time, so since it works pretty good for me, I want to share how to handle the spam for your @gentoo.org address with procmail.

First, you need to say that procmail will filter your email(s):
echo "| /usr/bin/procmail" > /home/${USER}/.forward

Then create a simple /home/${USER}/.procmailrc with this content:
:0:
* ^X-Spam-Status: Yes
/dev/null

:0:
* ^X-Spam-Level: \*\*\*
/dev/null/

:0:
* ! ^List-Id
* ^X-Spam-Level: \*\*
/dev/null/

:0:
* ^Subject:.*viagra*
/dev/null

:0:
* ^Subject:.*cialis*
/dev/null

:0:
* ^Subject:.*money*
/dev/null

:0:
* ^Subject:.*rolex*
/dev/null

:0:
* ^Subject:.*scount*
/dev/null

:0:
* ^Subject:.*Viagra*
/dev/null

:0:
* ^Subject:.*Cialis*
/dev/null

:0:
* ^Subject:.*Marketing*
/dev/null

:0:
* ^Subject:.*marketing*
/dev/null

:0:
* ^Subject:.*Money*
/dev/null

:0:
* ^Subject:.*Rolex*
/dev/null

:0:
* ^Subject:.*Scount*
/dev/null

:0:
* ^Subject:.*glxgug*
/dev/null

:0:
* ^Subject:.*offizielle sieger*
/dev/null

:0:
* ^Subject:.*educational*
/dev/null

:0 B:
* $ content-[^:]+:${WS}*.+(\<)*(file)?name${WS}*=${WS}*\/.+\.(pif|scr|com|cpl|vbs|mim|hqx|bhx|uue|uu|b64)\"?$
/dev/null

:0 B:
* ^Content-Type: .*;$[ ]*(file)?name=\"?.*\.(pif|scr|com|cpl|vbs)\"?$
/dev/null

:0 B:
* ^Content-Type: .*; [ ]*(file)?name=\"?.*\.(pif|scr|com|cpl|vbs)\"?$
/dev/null

With the filter for X-Spam-Status and X-Spam-Level you will avoid the majority of the incoming spam.
Some mails that does not have any Spam flag, contains subject like viagra, cialis ( which I absolutely don’t need :D ), rolex and scount.
Yes, I could you the (c|C)ase syntax, but I had problems, so I prefer to write twice the rules instead of have any sort of troubles.
Note: with this email address I’m not subscribed to any newsletter or any sort of offers/catalogs so I filtered scount, markerting, money.

Sometimes I receive mails from people that are not spammer, with the X-Spam-Level flag with one star, so I decided to move these email into a folder, they will be double-checked with naked eye:

:0:
* ^X-Spam-Level: \*
/home/ago/.maildir/.INBOX.pspam/

To avoid confusion I always prefer to use a complete path here.

After a stabilization you will always see the annoying mail from the bugzilla which contains ${arch} stable, so if you want to drop them:

:0 B
* ^*(alpha|amd64|arm|hppa|ia64|m68k|ppc|ppc64|s390|sh|sparc|x86) stable*
/dev/null

Now, if you are using more email clients, on more computers, you may need to set the filters here instead of on all clients you are using, so for example:

:0
* ^From.*bugzilla-daemon@gentoo.org
* ^TO.*amd64@gentoo.org
/home/ago/.maildir/.INBOX.amd64/

And so on….
These, hints obviously are valid on all postfix-based mailserver; if you are using e.g. qmail, you need to move the .procmailrc, but this is still valid.
I hope this will help :)

EDIT:
If you need a particular set of rules, you can write it if you take a look at the source/header of the message, so If for example I don’t like to see the mails from bugzilla of the bugs that I reported:

the header says: X-Bugzilla-Reporter: ago@gentoo.org
so:

:0
* ^From.*bugzilla-daemon@gentoo.org
* ^X-Bugzilla-Reporter.*ago@gentoo.org
/dev/null

New repoman option “–include-arches”: the benefit for an arch team member

Since a long time I realized that is a pita every time that I keyword, receive a repoman failure for dependency.bad(mostly) that does not regard the arch that I’m changing.
So, checking in the repoman manual, I realized that –ignore-arches looks bad for my case and I decided to request a new feature: –include-arches.
This feature, as explained in the bug, checks only for the arches that you write as argument and should be used only when you are keywording/stabilizing.

Some examples/usage:

First, it saves time, the following example will try to run repoman full in the kdelibs directory:
$ time repoman full > /dev/null 2>&1
real 0m12.434s

$ time repoman full --include-arches "amd64" > /dev/null 2>&1
real 0m3.880s

Second, kdelibs suffers for a dependency.bad on amd64-fbsd, so:
$ repoman full
RepoMan scours the neighborhood...
>>> Creating Manifest for /home/ago/gentoo-x86/kde-base/kdelibs
dependency.bad 2
kde-base/kdelibs/kdelibs-4.10.2.ebuild: PDEPEND: ~amd64-fbsd(default/bsd/fbsd/amd64/9.0) ['>=kde-base/nepomuk-widgets-4.10.2:4[aqua=]']

$ repoman full --include-arches "amd64"
RepoMan scours the neighborhood...
>>> Creating Manifest for /home/ago/gentoo-x86/kde-base/kdelibs

Now when I will keyword the packages I can check for specific arches and skip the unuseful checks since they causes, in this case, only a waste of time.
Thanks to Zac for the work on it.

Looking for KDE users on ARM

I received few requests to make KDE stable for ARM. Unfortunately I can’t do a complete test but I’m able to compile on both armv5 and armv7.

Before stabilize, I may set a virtual machine on qemu to test better, but I’d prefer to receive some feedback from the users.

So, if you are running KDE on arm, feel free to comment here, send me an e-mail or add a comment in the stabilization bug.

If you want to partecipate, look at kde-stable project.

MAKEOPTS=”-j${core} +1″ is NOT the best optimization

Many times, when I had to set the make.conf on systems with particular architectures, I had a doubt on which is the best –jobs value.
The handook suggest to have ${core} + 1, but since I’m curious I wanted to test it by myself to be sure this is right.

To make a good test we need a package with a respectable build system that respects the make parallelization and takes at least few minutes to compile. Otherwise with packages that compile in few seconds we are unable to track the effective difference.
kde-base/kdelibs is, in my opinion, perfect.

If you are on architecture which kde-base/kdelibs is unavailable, just switch to another cmake-based package.

Now, download best_makeopts from my overlay. Below an explanation on what the script does and various suggestions.

  • You need to compile the package on a tmpfs filesystem and, I’m assuming you have /tmp mounted as a tmpfs too;
  • You need to have the tarball of the package on a tmpfs because if you have a slow disk, it may takes more time.
  • You need to switch your governor to performance.
  • You need to be sure you don’t have strange EMERGE_DEFAULT_OPTS.
  • You need to add ‘-B’ because we don’t want to include the time of the installation.
  • You need to drop the existent cache before compile.

As you can see, the for will emerge the same package with makeopts from 1 to 10. If you have, for example, a single core machine, just try the for from 1 to 4 is enough.

Please, during the test, don’t use the cpu for other purposes, and if you can, stop all services and make the test from the tty; you will see the time for every merge.

The following is an example on my machine:
-j1 : real 29m56.527s
-j2 : real 15m24.287s
-j3 : real 13m57.370s
-j4 : real 12m48.465s
-j5 : real 12m55.894s
-j6 : real 13m5.421s
-j7 : real 13m13.322s
-j8 : real 13m23.414s
-j9 : real 13m26.657s

The hardware is:
Intel(R) Core(TM) i3 CPU 540 @ 3.07GHz which has 2 CPUs and 4 threads.
After -j4 you can see the regression.

Another example from an Intel Itanium with 4 CPUs.
-j1 : real 4m24.930s
-j2 : real 2m27.854s
-j3 : real 1m47.462s
-j4 : real 1m28.082s
-j5 : real 1m29.497s

I tested this script on ~20 different machines and in the majority of the cases, the best optimization was ${core} or more exactly ${threads} of your CPU.

Conclusion:
From the handbook:

A good choice is the number of CPUs (or CPU cores) in your system plus one, but this guideline isn’t always perfect.

I don’t know who, years ago, suggested in the handbook ${core} + 1 and I don’t want to trigger a flame. I’m just saying, ${core} + 1 is not the best optimization for me and the test confirms the part:“but this guideline isn’t always perfect”

In all cases ${threads} + ${X} is slower than only ${threads}, so don’t use -j20 if you have a dual-core cpu.

Also, I’m not saying to use ${threads}, I’m just saying feel free to make your tests to watch what is the best optimization.

If you have suggestions to improve the functionality of the script or you think that this script is wrong, feel free to comment or leave an email.

Repoman-check a big list before file a STABLEREQ or KEYWORDREQ

Unfortunately, all times we have a big list to keyword or stabilize, repoman complains about missing packages. So, in this post I will give you the solution to avoid this problem.

First, please download the batch-stabilize-pretend script from my overlay.
I’m not a python programmer but I was able to edit the script made by Paweł Hajdan. I just deleted the bugzilla commit part, and I make the script able to print repoman full if the list is not complete.
This script works only with =www-client/pybugz-0.9.3 and you need >=sys-apps/portage-2.1.11.63

Now, to check if repoman will complain about your list, you need to do:
./batch-stabilize-pretend.py --arch amd64 --repo /home/ago/gentoo-x86 -i /tmp/yourlist

where:

  • batch-stabilize-pretend.py is the script (obviously);
  • amd64 is the arch that you want to check. You will use ~amd64 for the keywordreq;
  • /home/ago/gentoo-x86 is the local copy of the CVS;
  • /tmp/yourlist is the list which contains the packages;

Few useful notes:

If you want to check on some arches, you can use a simple for:
for i in amd64 x86 sparc ppc ; do
./batch-stabilize-pretend.py --arch "${i}" --repo /home/ago/gentoo-x86 -i /tmp/yourlist
done

The script will run ekeyword, so it will touch your local CVS copy of gentoo-x86. If this is not your intention, please make another copy and work there or don’t forget to run cvs up -C.

Before doing this work, you need to run cvs up in the root of your gentoo-x86 local CVS.

The list must be structured in this mode:
# bug #445900
=app-portage/eix-0.27.4
=www-client/pybugz-0.9.3
=dev-vcs/cvs-1.12.12-r6
#and so on..

How to find issues related to LINGUAS

Usually, I want to find all possible issues with the LINGUAS variable, so in my arch testing environment I have enabled all linguas that the main tree uses.
To keep my make.conf more ‘clear’ I’m using source and another file called linguas.conf.

So, this is my /etc/portage/linguas.conf:
LINGUAS="am fil zh af ca cs da de el es et gl hu nb nl pl pt ro ru sk sl sv uk bg cy en eo fo ga he id ku lt lv mk ms nn sw tn zu ja zh_TW en_GB pt_BR ko zh_CN ar en_CA fi kk oc sr tr fa wa nds as be bn bn_BD bn_IN en_US es_AR es_CL es_ES es_MX eu fy fy_NL ga_IE gu gu_IN hi hi_IN is ka kn ml mr nn_NO or pa pa_IN pt_PT rm si sq sv_SE ta ta_LK te th vi ast dz km my om sh ug uz ca@valencia sr@ijekavian sr@ijekavianlatin sr@latin csb hne mai se es_LA fr_CA zh_HK br la no es_CR et_EE sr_CS bo hsb hy mn sr@Latn lb ne bs tg uz@cyrillic xh be_BY brx ca_XV dgo en_ZA gd kok ks ky lo mni nr ns pap ps rw sa_IN sat sd ss st sw_TZ ti ts ve mt ia az me tl ak hy_AM lg nso son ur_PK it fr nb nb_NO hr nan ur tk cs_CZ da_DK de_1901 de_CH en_AU lt_LT pl_PL sa sk_SK th_TH ta_IN tt sco ha mi ven ar_SY el_GR ro_RO ru_RU sl_SI uk_UA vi_VN ar_SY te_IN de_DE es_VE fa_IR fr_FR hu_HU id_ID it_IT ja_JP ka_GE nl_NL sr_BA sr_RS ca_ES fi_FI he_IL jv ru_gold yi eu_ES chr jp"

Now you need to set in your make.conf:
source /etc/portage/linguas.conf

I will update this post if there will be new linguas/languages in the future.

[01-31-2013]: Add chr and jp.

Kernel: vanilla-sources maintenance

In the last time I’m helping the kernel team with the bump of vanilla-sources.

It does not take much time because I’m doing it with a script. So, personally, I will continue to bump the following series:

  • 2.6.32
  • 3.0
  • 3.2
  • 3.4
  • 3.6

I will remove the EOL series as soon as possible.

If you have requests, please let me know.

How -g0 may be useful

Usually I use -g0 as CFLAGS/CXXFLAGS; it will be useful to find wrong buildsystem behavior.
ago@arcadia ~ $ portageq envvar CFLAGS
-march=native -O2 -g0

Here is an example where the buildsystem sed only ‘-g‘, leaves ‘0‘ and causes compile failure:

x86_64-pc-linux-gnu-gcc -DNDEBUG -march=native -O2 0 -m64 -O3 -Wall -DREGINA_SHARE_DIRECTORY=\"/usr/share/regina\" -DREGINA_VERSION_DATE=\""31 Dec 2011"\" -DREGINA_VERSION_MAJOR=\"3\" -DREGINA_VERSION_MINOR=\"6\" -DREGINA_VERSION_SUPP=\"\" -DHAVE_CONFIG_H -DHAVE_GCI -I./gci -I. -I. -I./contrib -o funcs.o -c ./funcs.c
x86_64-pc-linux-gnu-gcc: 0: No such file or directory
./funcs.c: In function '__regina_convert_date':
./funcs.c:772:14: warning: array subscript is above array bounds
make: *** [funcs.o] Error 1
emake failed

So add it to your CFLAGS/CXXFLAGS may be a good idea.

Documentazione Italiana: chiamata per nuovi traduttori

I’m writing this post in italian language because it is intended only for italian people.

E’ da tempo che abbiamo messo su l’idea di lavorare su git per quanto riguarda la traduzione della documentazione gentoo da inglese a italiano.
Siamo già in tanti, ma se avessimo altri traduttori potremmo produrre molto di più.
Non sono richeste specifiche tecniche, se non un minimo di conoscenza della lingua inglese.

Riferimenti:
http://dev.gentoo.org/~ago/trads-it.xml
http://dev.gentoo.org/~ago/howtohelp.xml
http://www.gentoo.org/doc/it/xml-guide.xml

Se in questi documenti c’è qualcosa di poco chiaro, non esitate a contattarmi.

Chi è interessato a collaborare può scrivermi via mail all’indirizzo ago@gentoo.org aggiungendo possibilmente il tag [docs-it] ad inizio oggetto o semplicemente cliccando qui.