Author: aeroniero

Hello,

This is the last week of the Gentoo-GPG Google Summer of Code project. It has been a fun ride where I learned a whole lot of things from little Python tricks to how Portage verifies GnuPG signatures. It will be an unforgettable experience that has helped me make an entrance to the open source community and shown me the success and the difficulties that can be found there. Now, for the summary of the work that I have done, I have written it below in two segments, Gentoo-Keys and MetaManifest.

Gentoo-Keys

In my proposal which can be found here, I promised to deliver on many things, most which I was able to achieve, while, in the end, one thing I and my mentors decided not to prioritize for that time frame, which I replaced with deliverables outside the proposal. More specifically, due to the complexity of pyGPG and Gentoo-Keys, I was unable to make unit tests for Gkeys because of time constraints. In return, I spent more time working the email reminder generator and fixing older Gkeys functions. All the other deliverables were accomplished. A combined PR of all my Gkeys code can be found in the branch here.

1. Count Incorrect Qualified IDs and present them in the Spec Check summary.  (link) (done in the bonding period)
2. Added a requirements file that includes required packages for Gkeys to run. (link) (done in the 1st week)
3. Fixed the move-seed function that was broken due to more recent patches. (link) (done in the 1st week)
4. Created the update key function that downloads and installs the seeds of a selected Gkeys category or all categories. (link) (done in the 2nd and 3rd week)
5. Add automatic update seed check whenever Gkeys runs, that checks if the seeds match with the ones from the server and if not, it gives the user the option to use update-seed. (link same PR as above) (done in the 4th week and 5th week)
6. Made the default GPG config file over-ridable in Gkeys-Gen during the generation of a new key. (link) (done in the 4th week)
7. Fixed a small bug in the verify function of Gkeys. (link) (done in the 4th week)
8. Created the send-key function (dubbed upload key in the proposal) that send the selected key/s in the selected server of the config file. In order to get that to work though, I had to make a small patch in pyGPG as well. (link, link) (done in the 5th week)
9. Created an email reminder generation script that works along with spec-check and using the Spec Check tuple, checks if a key has expired or is expiring in a selected time frame from the config file and if it is, it sends an email to the key’s owner that includes all the information needed along with resources on how to update the key. It priorities emails with a preferred address found in the config file and has the option to login in either email account found again in the config file. (link) (done in the 10th and 11th week)

Meta Manifest System

In my proposal, I also promised to implement the new Meta Manifest system according to GLEP:58 which is a system that provides a more efficient way to verify the integrity of Gentoo distribution. It works is by creating many different Manifest files for all important directories of a tree (ex. categories, profiles etc.) and then adding the hash sums of those Manifest files to create a master Meta Manifest file in the root directory, which is afterwards GPG signed by an official Gentoo key. That way, the user only needs to verify the master meta manifest file to check the integrity of the tree, which will automatically go through every single manifest file and verify its hash sum. If a package from a category is fully missing, then it is ignored, since partial downloads of categories are allowed as long as the downloaded packages include everything. Here is the link to the PR. The project was divided in two parts:

1. The creation and signing  process of the Meta Manifest files by using the logic above. (done in the 6th to 8th week)
2. The verification and signature validation process of the Meta Manifest files.  (done in the 9th to 11th week)

Plans for the future

My plan for the future is to continue supporting my code for Portage like making minor fixes and reworking some functions from the original manifest code that seem a little inefficient. I would also like to become part of the team developing Gentoo-Keys and prepare for the next release of the platform.

Conclusion

Finally, I would like to thank all those people that helped me with the project. Firstly my mentors, Pavlos ‘dastergon’ Ratis, Brian ‘dol-sen’ Dolbec and Kristian ‘K_F’ Fiskerstrand for their guidance and suggestions, and always being there whenever I had questions. Also, I would like to thank Doug ‘dwfreed’ Freed, Zac ‘zmedico’ Medico and Gilles ‘EvaSDK’ Dartiguelongue for reviewing my code and giving useful tips.

-aeroniero

Hello,

This was my twelfth week in the GSoC for the project Gentoo-GPG. For the second half, I will be developing the new MetaManifest system that will provide an efficient and secure way to verify the integrity of Gentoo distribution.

My work for this week was to:

• Do a few more optimization and fixes for the email generation script.
• Finish up with metamanifest script by doing an elaborate PR, which can be found here. (The testing failed because Gkeys is not implemented in portage yet but will be before the commits are merged)
• Fix any issues that might appear during runtests for the metamanifest.
• Add more documentation and prepare for the evaluation.

My plan for next week is to:

• Document everything I have done so far in my blog with links to each individual commit.
• Finish any final fixes and prepare for evaluation.

– aeroniero

Hello,

This was my eleventh week in the GSoC for the project Gentoo-GPG. For the second half, I will be developing the new MetaManifest system that will provide an efficient and secure way to verify the integrity of Gentoo distribution.

My work for this week was to:

• Finish optimizing and debugging the metamanifest script according to Zac’s feedback.
• Test for all metamanifest cases.
• Start rebasing all the changes in the previous commits.
• Finish the email generation script and do the PR, which can be found here.

My plan for next week is to:

• Finish up with metamanifest script by doing an elaborate PR.
• Fix any issues that might appear during runtests.
• Add more documentation and prepare for the evaluation.
• Possibly continue making the metamanifest test for runtests but most likely I will need to finish it after GSoC.

-aeroniero

Hello,

This was my tenth week in the GSoC for the project Gentoo-GPG. For the second half, I will be developing the new MetaManifest system that will provide an efficient and secure way to verify the integrity of Gentoo distribution.

My work for this week was to:

• Show the code to the other Gentoo devs for feedback.
• Continue optimizing and debugging according to feedback.
• After finishing the debugging of the Meta-Manifest system, generate email reminders for keys nearing expiry.

My work for next week is to:

• Continue optimizing and debugging the metamanifest script according to Zac’s  feedback.
• Finish the email generation script.
• Add metamanifest tests in the portage test section for creating, verifying and possibly signing metamanifest files.

– aeroniero

Hello,

This was my ninth week in the GSoC for the project Gentoo-GPG. For the second half, I will be developing the new MetaManifest system that will provide an efficient and secure way to verify the integrity of Gentoo distribution.

My work for this week was to:

• Add package manifest verification before the category MetaManifest generation to ensure the integrity of the package manifest files.
• Add more test cases for the verification.
• Rewrite the create category and master manifest functions to make them more efficient and less resource heavy.
• Rewrite other minor methods for efficiency.

My plan for next week is to:

• Show the code to the other Gentoo devs for feedback
• Continue optimizing and debugging according to feedback.
• After finishing the debugging of the Meta-Manifest system, generate email reminders for keys nearing expiry.

– aeroniero

Hello,

This was my eight week in the GSoC for the project Gentoo-GPG. For the second half, I will be developing the new MetaManifest system that will provide an efficient and secure way to verify the integrity of Gentoo distribution.

My work for this week was to:

• Add signature verification to my MetaManifest script using the Gkeys interface from Gentoo Keys
• Add manifest verification that passes through all repo levels of the tree to my script.
• Add the option for the user to verify the package manifests that include the DIST files by excluding them from the verification.
• Made some test cases and added them to my test tree.
• Debugged and optimized the script.

My plan for next week is to:

• Add more test cases according to my mentors’ feedback.
• Show my script to other Gentoo devs, as well, for more feedback.
• Optimize, debug and fix according to the suggestions given from the above.

– aeroniero

Hello,

This was my seventh week in the GSoC for the project Gentoo-GPG. For the second half, I will be developing the new MetaManifest system that will provide an efficient and secure way to verify the integrity of Gentoo distribution.

This week my work was to:

• Added eclass, profile and a master type to the types of MetaManifest files that can be created.
• Added a GPG signing option to the creation process.
• Added a way to test the script using a parser.

My plan for next week is to:

• Add signature verification to the script.
• Add manifest verification to the script that goes through all repo levels.
• Continue optimizing the script and prepare for the next milestone of the project.

– aeroniero

• Continue planning on how to implement the new MetaManifest system.
• Create a test overlay to be used to test all the work I do.
• Start the first milestone by creating a script that creates a MetaManifest file in a test overlay.
• Finish and bug fix all the previous Gentoo-Keys work.

• Add GPG signing to the MetaManifest file that the script creates.
• Make another script that verifies the MetaManifest file and its signature.
• Optimize and test all the scripts I made and plan for the next milestone of the project.

Hello,

This was my fifth week in the GSoC for the project Gentoo-GPG. For the first half I will be working on improving and expanding the capabilities of Gentoo-Keys, whose code can be found in this Github repository. For the second half, I will be developing the new MetaManifest system that will provide an efficient and secure way to verify the integrity of Gentoo distribution. My work for the fifth week was to:

• Implement upload key functionality
• Fix the import key function of gkeys
• Bug test and fix my previous work
• Start planning for the development of the meta-manifest system for the 2nd half of GSoC.

My goal for next week is to:

• Continue planning on how to implement the new MetaManifest system.
• Start the first milestone by creating a script that creates a MetaManifest file in a test tree.
• Finish and bug fix all the previous Gentoo-Keys work.

– aeroniero

Hello,

This was my fourth week in the GSoC for the project Gentoo-GPG. For the first half I will be working on improving and expanding the capabilities of Gentoo-Keys, whose code can be found in this Github repository. My work for the fourth week was to:

• Expand on the function’s capabilities by making it inform the user about which keys have been updated each time the function is used.
• Display a warning message to the user and give him the opportunity to do an update check if a key has been revoked or expired recently.
• Do further bug testing on everything I have done on Gentoo-Keys so far to prepare for the midterm evaluation.
• Make the GPG default config file over-ridable for gkeys-gen during key creation the code of which can be found here
• Fix a bug in the verify function of gkeys, which can be found here.

My plan for next week is to:

• Implement upload key functionality
• Fix the import key function of gkeys
• Bug test and fix my previous work
• Start planning for the development of the meta-manifest system for the 2nd half of GSoC.

– aeroniero