Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update OSS-Fuzz integration #2511

Open
correctmost opened this issue Aug 13, 2024 · 6 comments
Open

Update OSS-Fuzz integration #2511

correctmost opened this issue Aug 13, 2024 · 6 comments
Labels
Maintenance Discussion or action around maintaining astroid or the dev workflow Work in progress

Comments

@correctmost
Copy link
Contributor

Description

An OSS-Fuzz integration was set up for astroid last year. The build has been failing for a few months and it seems to lack active maintenance.

Would you mind if I fixed the build and assigned myself as a co-maintainer? I maintain the librsvg integration and I am familiar with the OSS-Fuzz platform.

Having a functioning OSS-Fuzz integration could help catch regressions on the main branch before releases are tagged.

Action items

  1. Fix the build by pinning the version to astroid 3.2.4 for now
    • The OSS-Fuzz images are currently stuck on Python 3.8, but there is work being done to support 3.10
    • Fixing the build now will allow the corpus to grow, which will benefit future runs against main (once Python 3.10 support lands)
  2. Add myself to the list of people who are automatically CC'd on astroid bug reports

Considerations

  1. Are any of the current maintainers interested in being set as the primary contact for the OSS-Fuzz integration?
    • To get full access to the system, you will need a Google/Gmail account
      • Note: The associated email address will be listed in the OSS-Fuzz repo without any obfuscation (example)
    • Other maintainers can also be automatically CC'd, but there can only be one primary
  2. OSS-Fuzz can generate a decent amount of bug reports
    • I can help triage the issues, but some maintainers do not want to get notifications from a totally separate system
  3. Updating the astroid project files on OSS-Fuzz requires the signing of a Google CLA
    • I have already signed the CLA, so I can help with PRs if you are uncomfortable with that process
  4. Google offers monetary rewards for improving the code coverage of existing integrations. I am interested in fixing the build and helping maintain the integration independently of that, but I also have ideas for increasing coverage that might qualify for said rewards.

Google has documentation for the OSS-Fuzz system, but I can also help answer any questions. Thanks!

@jacobtylerwalls
Copy link
Member

Thanks for this @correctmost, I'll ask around and get an answer on the primary contact, but I anticipate that the answers to your other questions will be "for sure".

@jacobtylerwalls
Copy link
Member

I got a response from @Pierre-Sassoulas that it's okay to list me (@jacobtylerwalls) as primary: jacobtylerwalls [ at ] gmail.com

Would you mind if I fixed the build and assigned myself as a co-maintainer?

Yes, go ahead, and thanks for the help!

@correctmost
Copy link
Contributor Author

Awesome, thanks! I submitted a PR to fix the build and update the maintainers list:

DavidKorczynski pushed a commit to google/oss-fuzz that referenced this issue Aug 21, 2024
I have received permission to update the maintainers list here:
pylint-dev/astroid#2511 (comment)

This commit also fixes the build by pinning astroid to the latest
version with Python 3.8 support. This will allow the corpus to grow
until Python 3.9+ is supported on OSS-Fuzz.
@correctmost
Copy link
Contributor Author

The OSS-Fuzz PR was merged, so you should now have access to the following items with your Google account:

If you don't have access yet, it may take a day or two for everything to sync.

We also got CC'd on ~15 existing bug reports. Here's my plan for those:

  • Let the automated reproduction tasks run tomorrow with a fixed build
    • The build was not fixed for today's run (8-21) because the PR got merged after the run
  • Triage any reports that are still marked as reproducible to see if the bug still exists on main

This is my overall plan for the integration:

  • Verify access to the system
  • Triage all existing reports
  • Submit developer docs to the astroid (or Pylint) repo
  • Increase coverage after a steady state has been reached with the triage efforts

Let me know if you encounter any issues!

@correctmost
Copy link
Contributor Author

correctmost commented Aug 27, 2024

Updates for the above tasks:

Verify access to the system

Hopefully you have access now :)

Triage all existing reports

All issues have been triaged. Here's the full list of bugs:

Submit developer docs to the astroid (or Pylint) repo

PR submitted: pylint-dev/pylint#9896

Increase coverage after a steady state has been reached with the triage efforts

Coverage builds are currently broken. I am trying to get more debug info with this PR: google/oss-fuzz#12502.

I will revisit this item after the coverage builds are fixed.

@correctmost
Copy link
Contributor Author

Coverage builds are currently broken. I am trying to get more debug info with this PR: google/oss-fuzz#12502.

Coverage builds were fixed by these changes:

I submitted an additional PR to try to increase coverage, which is currently at 66%:

Remaining work:

@Pierre-Sassoulas Pierre-Sassoulas added Maintenance Discussion or action around maintaining astroid or the dev workflow Work in progress labels Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Maintenance Discussion or action around maintaining astroid or the dev workflow Work in progress
Projects
None yet
Development

No branches or pull requests

4 participants
@Pierre-Sassoulas @jacobtylerwalls @correctmost and others