Usually, I want to find all possible issues with the LINGUAS variable, so in my arch testing environment I have enabled all linguas that the main tree uses.
To keep my make.conf more ‘clear’ I’m using source and another file called linguas.conf.

So, this is my /etc/portage/linguas.conf:
LINGUAS="am fil zh af ca cs da de el es et gl hu nb nl pl pt ro ru sk sl sv uk bg cy en eo fo ga he id ku lt lv mk ms nn sw tn zu ja zh_TW en_GB pt_BR ko zh_CN ar en_CA fi kk oc sr tr fa wa nds as be bn bn_BD bn_IN en_US es_AR es_CL es_ES es_MX eu fy fy_NL ga_IE gu gu_IN hi hi_IN is ka kn ml mr nn_NO or pa pa_IN pt_PT rm si sq sv_SE ta ta_LK te th vi ast dz km my om sh ug uz ca@valencia sr@ijekavian sr@ijekavianlatin sr@latin csb hne mai se es_LA fr_CA zh_HK br la no es_CR et_EE sr_CS bo hsb hy mn sr@Latn lb ne bs tg uz@cyrillic xh be_BY brx ca_XV dgo en_ZA gd kok ks ky lo mni nr ns pap ps rw sa_IN sat sd ss st sw_TZ ti ts ve mt ia az me tl ak hy_AM lg nso son ur_PK it fr nb nb_NO hr nan ur tk cs_CZ da_DK de_1901 de_CH en_AU lt_LT pl_PL sa sk_SK th_TH ta_IN tt sco ha mi ven ar_SY el_GR ro_RO ru_RU sl_SI uk_UA vi_VN ar_SY te_IN de_DE es_VE fa_IR fr_FR hu_HU id_ID it_IT ja_JP ka_GE nl_NL sr_BA sr_RS ca_ES fi_FI he_IL jv ru_gold yi eu_ES chr jp"

Now you need to set in your make.conf:
source /etc/portage/linguas.conf

I will update this post if there will be new linguas/languages in the future.

[01-31-2013]: Add chr and jp.

Usually I use -g0 as CFLAGS/CXXFLAGS; it will be useful to find wrong buildsystem behavior.
ago@arcadia ~ $ portageq envvar CFLAGS
-march=native -O2 -g0

Here is an example where the buildsystem sed only ‘-g‘, leaves ‘0‘ and causes compile failure:

x86_64-pc-linux-gnu-gcc -DNDEBUG -march=native -O2 0 -m64 -O3 -Wall -DREGINA_SHARE_DIRECTORY=\"/usr/share/regina\" -DREGINA_VERSION_DATE=\""31 Dec 2011"\" -DREGINA_VERSION_MAJOR=\"3\" -DREGINA_VERSION_MINOR=\"6\" -DREGINA_VERSION_SUPP=\"\" -DHAVE_CONFIG_H -DHAVE_GCI -I./gci -I. -I. -I./contrib -o funcs.o -c ./funcs.c
x86_64-pc-linux-gnu-gcc: 0: No such file or directory
./funcs.c: In function '__regina_convert_date':
./funcs.c:772:14: warning: array subscript is above array bounds
make: *** [funcs.o] Error 1
emake failed

So add it to your CFLAGS/CXXFLAGS may be a good idea.

Many times in the past, a lot of people found bugs for packages actually in a STABLEREQ.

Every time my answer was: make it as a block only if is a regression; and every time the next question was: “What is a regression?”.

Well, seems that word is not familiar for all, let me explain this concept.; first, see the word regression as a synonymous of worsening.

I guess, for this concept, an example should give you ‘the basic idea’

Imagine we are testing app-arch/tar-1.26-r1 (at this moment it is really ~arch).

During the testing we find 4 issue:

1)the CFLAGS variable is not respected.
2)there is a sed failure in the ebuild.
3)there is a test failure.
4)tar fails to extract some archives in a specific mode.

I chose these four problems as an example for a reason. The regressions must block the stabilization, independently of the type of problem.
Each of them represents a problem of a “different nature”.

1)means builsystem issue
2)means ebuild issue
3) and 4) means specific software problem

What you should do now? You should test the last stable version of tar (1.26 in this case) and check if you are able to reproduce these problems.

From the subsequent tests, you can see that tar-1.26 fails to respect CFLAGS(1), fails to sed one or more files(2), there are no test failures, and you can reproduce the extract issue.

Now, go to our bugzilla, and check if there are open bugs about these problems. If not, please open the bugs but pay attention about the blocks.

Since the first is reproducible in the last stable, is not a regression, it means no block.
Since the second is reproducible in the last stable, is not a regression, it means no block.
Since the third is not reproducible in the last stable, is a regression, it means block.
Since the fourth is reproducible in the last stable, is not a regression, it means no block.

For this case, you should open a new bug about the test failures and make it as a block for the current stabilization. Obviously, if there are open bugs that need a block, do it instead of open new(duplicate) bugs.

Now, apart the test failures and ignoring the failures 1 and 2, the obvious question is: “Why we should mark stable tar-1.26-r1 if it fails to extract stuff?”.
Here you should learn the regression concept; imagine you are an user, you are using tar-1.26 and you can’t extract some archives; we mark stable 1.26-r1 and you can’t do it too. There are no changes for you and no worsening. You can’t do before and you can’t do now again.

Probably this post is documented elsewhere, but I hope that can helps.

Seems obviously for many of you, but a lot of people in the past, have asked me how to test toolchain components and be sure to don’t break anything.

Ok, as usual we will start to test the singular package with multiple USE combinations. Don’t expect that should work with USE=”vanilla”; for who don’t know:

Do not add extra patches which change default behaviour; DO NOT USE THIS ON A GLOBAL SCALE as the severity of the meaning changes drastically

If the various tests passes without build failure, now, you should try to recompile your system, to check if the new package could break something, so:
emerge -e system

If there are failures, check if them are related to what you are testing(e.g. don’t post zlib failures when you test binutils). If you don’t know how to understand who causes the failure, just poke a developer on irc.

If you have a powerful machine, instead of re-emerge system please do:
emerge -e world

Obviously for failures, file a bug and block the stabilization bug or the tracker.

In the past, a lot of people ask me how test a new kernel. This tip could help new arch tester.

First, emerge the new sources ( 3.4.5 is just an example, replace it with your ${version} ):
echo "=sys-kernel/gentoo-sources-3.4.5" >> /etc/portage/package.keywords
emerge -av =sys-kernel/gentoo-sources-3.4.5

Now go to kernel directory, try to enable all modules and check if them compile:
cd /usr/src/linux
make allyesconfig
make # don't forget to add '-j'

Might seem strange, but in the past, with allyesconfig, I found bug like this, not reproducible with normal config.

The next step is clean the past build and make your custom kernel.
make distclean
make menuconfig
make
make modules_install # if you use modules

Now try to boot with new kernel, and check if there are not bad message with:
dmesg

Now, try to reach a bit of uptime and if all is ok, please give a feedback.

This is a base guide to test {vanilla,gentoo}-sources. If you are testing a kernel with other/special features ( e.g. hardened/zen/tuxonice ), make sure that these features work perfectly.