{"id":127,"date":"2013-01-14T18:38:22","date_gmt":"2013-01-14T16:38:22","guid":{"rendered":"http:\/\/blogs.gentoo.org\/ago\/?p=127"},"modified":"2018-06-20T09:29:29","modified_gmt":"2018-06-20T07:29:29","slug":"makeopts-jcore-1-is-not-the-best-optimization","status":"publish","type":"post","link":"https:\/\/blogs.gentoo.org\/ago\/2013\/01\/14\/makeopts-jcore-1-is-not-the-best-optimization\/","title":{"rendered":"MAKEOPTS=&#8221;-j${core} +1&#8243; is NOT the best optimization"},"content":{"rendered":"<p>Many times, when I had to set the make.conf on systems with particular architectures, I had a doubt on which is the best &#8211;jobs value.<br \/>\nThe handook suggest to have ${core} + 1, but since I&#8217;m curious I wanted to test it by myself to be sure this is right.<\/p>\n<p>To make a good test we need a package with a respectable build system that respects the make parallelization and takes at least few minutes to compile. Otherwise with packages that compile in few seconds we are unable to track the effective difference.<br \/>\n<em>kde-base\/kdelibs<\/em> is, in my opinion, perfect.<\/p>\n<p>If you are on architecture which kde-base\/kdelibs is unavailable, just switch to another cmake-based package.<\/p>\n<p>Now, download <a href=\"https:\/\/gitweb.gentoo.org\/dev\/ago.git\/tree\/script\/best_makeopts.sh\">best_makeopts<\/a>\u00a0from my overlay. Below an explanation on what the script does and various suggestions.<\/p>\n<ul>\n<li>You need to compile the package on a tmpfs filesystem and, I&#8217;m assuming you have <em>\/tmp<\/em> mounted as a tmpfs too;<\/li>\n<li>You need to have the tarball of the package on a tmpfs because if you have a slow disk, it may takes more time.<\/li>\n<li>You need to switch your governor to <em>performance.<\/em><\/li>\n<li>You need to be sure you don&#8217;t have strange <em>EMERGE_DEFAULT_OPTS<\/em>.<\/li>\n<li>You need to add &#8216;-B&#8217; because we don&#8217;t want to include the time of the installation.<\/li>\n<li>You need to drop the existent cache before compile.<\/li>\n<\/ul>\n<p>As you can see, the for will emerge the same package with makeopts from 1 to 10.\u00a0If you have, for example, a single core machine, just try the for from 1 to 4 is enough.<\/p>\n<p>Please, during the test, don&#8217;t use the cpu for other purposes, and if you can, stop all services and make the test from the tty; you will see the time for every merge.<\/p>\n<p>The following is an example on my machine:<br \/>\n<code>-j1 : real    29m56.527s<br \/>\n-j2 : real    15m24.287s<br \/>\n-j3 : real    13m57.370s<br \/>\n-j4 : real    12m48.465s<br \/>\n-j5 : real    12m55.894s<br \/>\n-j6 : real    13m5.421s<br \/>\n-j7 : real    13m13.322s<br \/>\n-j8 : real    13m23.414s<br \/>\n-j9 : real    13m26.657s<\/code><\/p>\n<p>The hardware is:<br \/>\n<code>Intel(R) Core(TM) i3 CPU         540  @ 3.07GHz<\/code> which has 2 CPUs and 4 threads.<br \/>\nAfter -j4 you can see the regression.<\/p>\n<p>Another example from an Intel Itanium with 4 CPUs.<br \/>\n<code>-j1 : real    4m24.930s<br \/>\n-j2 : real    2m27.854s<br \/>\n-j3 : real    1m47.462s<br \/>\n-j4 : real    1m28.082s<br \/>\n-j5 : real    1m29.497s<\/code><\/p>\n<p>I tested this script on ~20 different machines and in the majority of the cases, the best optimization was ${core} or more exactly ${threads} of your CPU.<\/p>\n<p><strong>Conclusion:<\/strong><br \/>\nFrom the handbook:<\/p>\n<blockquote><p>A good choice is the number of CPUs (or CPU cores) in your system plus one, but this guideline isn&#8217;t always perfect.<\/p><\/blockquote>\n<p>I don&#8217;t know who, years ago, suggested in the handbook ${core} + 1 and I don&#8217;t want to trigger a flame. I&#8217;m just saying, ${core} + 1 is not the best optimization <strong>for me<\/strong> and the test confirms the part:<em>&#8220;but this guideline isn&#8217;t always perfect&#8221;<\/em><\/p>\n<p>In all cases ${threads} + ${X} is slower than only ${threads}, so don&#8217;t use -j20 if you have a dual-core cpu.<\/p>\n<p>Also, I&#8217;m not saying to use ${threads}, I&#8217;m just saying feel free to make your tests to watch what is the best optimization.<\/p>\n<p>If you have suggestions to improve the functionality of the script or you think that this script is wrong, feel free to comment or leave an email.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Many times, when I had to set the make.conf on systems with particular architectures, I had a doubt on which is the best &#8211;jobs value. The handook suggest to have ${core} + 1, but since I&#8217;m curious I wanted to &hellip; <a href=\"https:\/\/blogs.gentoo.org\/ago\/2013\/01\/14\/makeopts-jcore-1-is-not-the-best-optimization\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":140,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[3],"tags":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p2EaBc-23","_links":{"self":[{"href":"https:\/\/blogs.gentoo.org\/ago\/wp-json\/wp\/v2\/posts\/127"}],"collection":[{"href":"https:\/\/blogs.gentoo.org\/ago\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.gentoo.org\/ago\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/ago\/wp-json\/wp\/v2\/users\/140"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/ago\/wp-json\/wp\/v2\/comments?post=127"}],"version-history":[{"count":30,"href":"https:\/\/blogs.gentoo.org\/ago\/wp-json\/wp\/v2\/posts\/127\/revisions"}],"predecessor-version":[{"id":2385,"href":"https:\/\/blogs.gentoo.org\/ago\/wp-json\/wp\/v2\/posts\/127\/revisions\/2385"}],"wp:attachment":[{"href":"https:\/\/blogs.gentoo.org\/ago\/wp-json\/wp\/v2\/media?parent=127"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/ago\/wp-json\/wp\/v2\/categories?post=127"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/ago\/wp-json\/wp\/v2\/tags?post=127"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}