emerge --version<\/code>.<\/p>\nThe problem.<\/h2>\n
emerge\u2019s \u2013version command takes a bit longer than most programs. For example,<\/p>\n
$ time gcc --version\r\nExecuted in 1.07 millis\r\n\r\n$ time python --version\r\nExecuted in 2.70 millis\r\n\r\n$ time black --version\r\nExecuted in 84.64 millis<\/code><\/pre>\nGuess how much time emerge --version<\/code> takes.<\/p>\n$ time emerge --version\r\nExecuted in 709.19 millis<\/code><\/pre>\nThere is something sinister going on. So we decided to profile the the code. The live profile results\u00a0 can be found here<\/a>.<\/p>\nHere is the image version ![\"profile\"](\"https:\/\/gist.githubusercontent.com\/berinaniesh\/4805eaaec0f7ffd41766d864f13c582f\/raw\/c6237cb0d29c5282e89533c1d3605577d45a332f\/profile.png\")
<\/p>\n
Diving into the profile results<\/h2>\n
Here, the total run takes 750 milliseconds.<\/p>\n
\n- Till
emerge_main<\/code>, it takes 71 milliseconds – That\u2019s for some imports, command line arguments parsing etc, cannot be avoided.<\/li>\n- From
emerge_main<\/code> to run_action<\/code> takes 167 milliseconds. This is mainly due to the creation of emerge_config<\/code> object, which is needed to assess different variables (emerge \u2013version outputs more information than only a version number). This can be reduced with significant code changes, but it will lead to a lot of code restructuring and code duplication. We still got 500 milliseconds to account for, lets look into that.<\/li>\n- Notice that there is no difference between
run_action\u00a0<\/code>and getportageversion<\/code>. This means that getting the portage version takes around 1 millisecond. It is true because, it
\nPORTAGE_VERSION<\/code> is just a variable defined in lib\/portage\/__init__.py<\/code>.<\/li>\ngetgccversion()<\/code> takes 460 milliseconds. That\u2019s concerning because we just noticed that gcc --version<\/code> takes around 1 millisecond.<\/li>\n<\/ul>\nDiving into getgccversion()<\/code><\/h2>\nTurns out getgccversion<\/code> gets gcc\u2019s version in two ways. gcc --dumpversion<\/code> or gcc-config -c<\/code>. If the former code path is taken, getgccversion<\/code> takes a couple milliseconds, but if the latter code path is taken, getgccversion<\/code> takes 450 milliseconds. I tried to find when the former codepath is taken, but it is almost never and it seemed like gcc-config -c<\/code> is unnecessary. So, I avoided the call and created a new code path just for --version<\/code> (so that no other part of portage is affected).<\/p>\nPatch<\/h2>\n
Together with a new code path for --version<\/code>, with the help of my mentor Sam James, we also refactored a few big functions into smaller ones, and added a few quality of life changes like f-strings etc. The pull request can be found here<\/a> With this change, the emerge --version<\/code> goes from 750ms to 240ms.<\/p>\nBut, we did not consider the edge cases where the CHOST of the system where packages are\u00a0 compiled might be different than the system the binpkgs would be used. Though it does not affect the functionality of portage in any way, it could<\/em> provide wrong information to the
\nend user when he\/she types emerge --version<\/code>. So, the pull request is not merged yet and we are working on solutions to the problem. One of the main reasons for delay of resolution is the fact
\nthat I don\u2019t completely understand what exactly gcc-config <\/code>does. It is a bash script and I have very little knowledge of bash. We are working on a solution and will try to get the changes merged into master.<\/p>\nSide benefit<\/h2>\n
Studying the portage codebase for emerge \u2013version has been indeed a fruitful one. I am getting more familiar with the codebase and I was able to find an unreachable code block (duplicated logic). I submitted a pull request<\/a> and it was merged. Sam commented\u00a0<\/a>
\nthat I am getting familiar with the codebase. That felt good.<\/p>\nNext week<\/h2>\n
I need more understanding of portage\u2019s internals. Sam suggested that I add docstrings and type annotations to the codebase. That\u2019ll help new developers as well as help me understand the codebase more. So, the next week will most probably be spent type annotating and adding docstrings. I\u2019ll also spent a bit of time learning bash so that I can work on gcc-config and many more as portage\/gentoo relies a lot on bash.<\/p>\n
Conclusion<\/h2>\n
Overall, the first week was a productive one. Though the pull request is not merged yet, it has good changes with respect to refactoring. If the new codepath is not sucessful, we\u2019d drop those commits to merge in the rest, hopefully we fix the patch to work on all CHOSTs. See you next week! Have a good one!<\/p>\n","protected":false},"excerpt":{"rendered":"
Week 1 – Modernization of Portage Coding period starts So, it\u2019s the first week of the official coding period and I wanted to write some code and get it\u00a0 merged into the master branch (I understand it\u2019s a bit over … Continue reading