{"id":251,"date":"2018-04-17T23:15:38","date_gmt":"2018-04-18T06:15:38","guid":{"rendered":"http:\/\/blogs.gentoo.org\/zmedico\/?p=251"},"modified":"2018-05-17T14:21:45","modified_gmt":"2018-05-17T21:21:45","slug":"portage-api-asyncio-event-loop","status":"publish","type":"post","link":"https:\/\/blogs.gentoo.org\/zmedico\/2018\/04\/17\/portage-api-asyncio-event-loop\/","title":{"rendered":"portage API now provides an asyncio event loop policy"},"content":{"rendered":"<p>In portage-2.3.30, portage&#8217;s python API provides an\u00a0<a href=\"https:\/\/docs.python.org\/3\/library\/asyncio-eventloops.html#asyncio.set_event_loop_policy\">asyncio event loop policy<\/a> via a DefaultEventLoopPolicy class. For example, here&#8217;s a little program that uses portage&#8217;s\u00a0DefaultEventLoopPolicy to do the same thing as emerge &#045;&#045;regen, using an async_iter_completed function to implement the &#045;&#045;jobs and &#045;&#045;load&#045;average options:<\/p>\n<pre>#!\/usr\/bin\/env python\r\n\r\nfrom __future__ import print_function\r\n\r\nimport argparse\r\nimport functools\r\nimport multiprocessing\r\nimport operator\r\n\r\nimport portage\r\nfrom portage.util.futures.iter_completed import (\r\n    async_iter_completed,\r\n)\r\nfrom portage.util.futures.unix_events import (\r\n    DefaultEventLoopPolicy,\r\n)\r\n\r\n\r\ndef handle_result(cpv, future):\r\n    metadata = dict(zip(portage.auxdbkeys, future.result()))\r\n    print(cpv)\r\n    for k, v in sorted(metadata.items(),\r\n        key=operator.itemgetter(0)):\r\n        if v:\r\n            print('\\t{}: {}'.format(k, v))\r\n    print()\r\n\r\n\r\ndef future_generator(repo_location, loop=None):\r\n\r\n    portdb = portage.portdb\r\n\r\n    for cp in portdb.cp_all(trees=[repo_location]):\r\n        for cpv in portdb.cp_list(cp, mytree=repo_location):\r\n            future = portdb.async_aux_get(\r\n                cpv,\r\n                portage.auxdbkeys,\r\n                mytree=repo_location,\r\n                loop=loop,\r\n            )\r\n\r\n            future.add_done_callback(\r\n                functools.partial(handle_result, cpv))\r\n\r\n            yield future\r\n\r\n\r\ndef main():\r\n    parser = argparse.ArgumentParser()\r\n    parser.add_argument(\r\n        '--repo',\r\n        action='store',\r\n        default='gentoo',\r\n    )\r\n    parser.add_argument(\r\n        '--jobs',\r\n        action='store',\r\n        type=int,\r\n        default=multiprocessing.cpu_count(),\r\n    )\r\n    parser.add_argument(\r\n        '--load-average',\r\n        action='store',\r\n        type=float,\r\n        default=multiprocessing.cpu_count(),\r\n    )\r\n    args = parser.parse_args()\r\n\r\n    try:\r\n        repo_location = portage.settings.repositories.\\\r\n            get_location_for_name(args.repo)\r\n    except KeyError:\r\n        parser.error('unknown repo: {}\\navailable repos: {}'.\\\r\n            format(args.repo, ' '.join(sorted(\r\n            repo.name for repo in\r\n            portage.settings.repositories))))\r\n\r\n    policy = DefaultEventLoopPolicy()\r\n    loop = policy.get_event_loop()\r\n\r\n    try:\r\n        for future_done_set in async_iter_completed(\r\n            future_generator(repo_location, loop=loop),\r\n            max_jobs=args.jobs,\r\n            max_load=args.load_average,\r\n            loop=loop):\r\n            loop.run_until_complete(future_done_set)\r\n    finally:\r\n        loop.close()\r\n\r\n\r\n\r\nif __name__ == '__main__':\r\n    main()\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>In portage-2.3.30, portage&#8217;s python API provides an\u00a0asyncio event loop policy via a DefaultEventLoopPolicy class. For example, here&#8217;s a little program that uses portage&#8217;s\u00a0DefaultEventLoopPolicy to do the same thing as emerge &#045;&#045;regen, using an async_iter_completed function to implement the &#045;&#045;jobs and &#045;&#045;load&#045;average options: #!\/usr\/bin\/env python from __future__ import print_function import argparse import functools import multiprocessing import &hellip; <a href=\"https:\/\/blogs.gentoo.org\/zmedico\/2018\/04\/17\/portage-api-asyncio-event-loop\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">portage API now provides an asyncio event loop policy<\/span><\/a><\/p>\n","protected":false},"author":65,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3,21],"tags":[23,9,22],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blogs.gentoo.org\/zmedico\/wp-json\/wp\/v2\/posts\/251"}],"collection":[{"href":"https:\/\/blogs.gentoo.org\/zmedico\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.gentoo.org\/zmedico\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/zmedico\/wp-json\/wp\/v2\/users\/65"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/zmedico\/wp-json\/wp\/v2\/comments?post=251"}],"version-history":[{"count":11,"href":"https:\/\/blogs.gentoo.org\/zmedico\/wp-json\/wp\/v2\/posts\/251\/revisions"}],"predecessor-version":[{"id":263,"href":"https:\/\/blogs.gentoo.org\/zmedico\/wp-json\/wp\/v2\/posts\/251\/revisions\/263"}],"wp:attachment":[{"href":"https:\/\/blogs.gentoo.org\/zmedico\/wp-json\/wp\/v2\/media?parent=251"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/zmedico\/wp-json\/wp\/v2\/categories?post=251"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/zmedico\/wp-json\/wp\/v2\/tags?post=251"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}