{"id":15,"date":"2006-07-03T04:22:40","date_gmt":"2006-06-12T02:32:32","guid":{"rendered":""},"modified":"2016-11-26T07:49:56","modified_gmt":"2016-11-26T07:49:56","slug":"cvs2git_parsecvs_gentoo_x86_conversion","status":"publish","type":"post","link":"https:\/\/blogs.gentoo.org\/solar\/2006\/07\/03\/cvs2git_parsecvs_gentoo_x86_conversion\/","title":{"rendered":"cvs2git\/parsecvs gentoo-x86 conversion"},"content":{"rendered":"<p>Recently thanks to the Oregon State University Open Source Labs I&#8217;ve been given access to an <a href=\"http:\/\/powerdev.osuosl.org\/node\/4\">IBM OpenPower 720<\/a>. This thing is a beast like no other box which I have access to. The specs are simply amazing. Anyway I noticed that Alec Warner\/antarus@gentoo was having problems with running a cvs2git conversion of the gentoo-x86 tree, every box which he attempted it on ran out of memory. I figured ok well I&#8217;ve got access to the mothership and should not have any problems doing a run for him. We talked for a little while and he provided me with a quick little script to fire off the conversion process. Well it took 21 hrs consumed 100% of the CPU the entire time and then it failed, towards the end right before it died with an Out of Memory: Killed process 14671 (parsecvs) error. It had consumed 70.1G of virtual memory and 30G RSS as well as all the swap. The gentoo-x86 tree is about 1.4G worth cvs data, the parsecvs util had managed to convert that into 4.1G of git data before it got killed. Gotta say from an admin\/infra point of view going from a 1.4G to +4.1G backend repo leaves little room to be desired.<\/p>\n<p>None the less I don&#8217;t see us switching to git any time soon unless the backend tools for conversion get a rewrite\/update so they can process the full repo as incremental parts or learn how to use the existing memory more efficiently.<\/p>\n<p>In this graph you can see where I started about at 21:00 and ran till about 18:00 the following day, at about 14:00 the basic conversion process was done and parsecvs started allocating memory here pretty quickly for another 4 hours. The final spike is when it started swapping to disk before it got killed.<\/p>\n<p><img loading=\"lazy\" src=\"http:\/\/planet.gentoo.org\/developers\/media\/cvs2git-24h-load.png\" width=\"597\" height=\"241\" alt=\"24h CPU Usage\" \/><\/p>\n<p>Unfortunately the snmpd version running on the box does not appear to support 64bit counters so all the memory graphs are\/were nil.<\/p>\n<p>In the end I had fun helping him with this, and it really gave the power5 a workout \ud83d\ude42<br \/>\nSometime later this week I&#8217;ll start setting up ppc64 binrepos, cross compilers etc..<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Recently thanks to the Oregon State University Open Source Labs I&#8217;ve been given access to an IBM OpenPower 720. This thing is a beast like no other box which I have access to. The specs are simply amazing. Anyway I noticed that Alec Warner\/antarus@gentoo was having problems with running a cvs2git conversion of the gentoo-x86 &hellip; <a href=\"https:\/\/blogs.gentoo.org\/solar\/2006\/07\/03\/cvs2git_parsecvs_gentoo_x86_conversion\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">cvs2git\/parsecvs gentoo-x86 conversion<\/span><\/a><\/p>\n","protected":false},"author":36,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[3,4],"tags":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blogs.gentoo.org\/solar\/wp-json\/wp\/v2\/posts\/15"}],"collection":[{"href":"https:\/\/blogs.gentoo.org\/solar\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.gentoo.org\/solar\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/solar\/wp-json\/wp\/v2\/users\/36"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/solar\/wp-json\/wp\/v2\/comments?post=15"}],"version-history":[{"count":1,"href":"https:\/\/blogs.gentoo.org\/solar\/wp-json\/wp\/v2\/posts\/15\/revisions"}],"predecessor-version":[{"id":36,"href":"https:\/\/blogs.gentoo.org\/solar\/wp-json\/wp\/v2\/posts\/15\/revisions\/36"}],"wp:attachment":[{"href":"https:\/\/blogs.gentoo.org\/solar\/wp-json\/wp\/v2\/media?parent=15"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/solar\/wp-json\/wp\/v2\/categories?post=15"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/solar\/wp-json\/wp\/v2\/tags?post=15"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}