Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-enable pylint CI checker #5096

Merged
merged 1 commit into from
Oct 25, 2023

Conversation

matetokodi
Copy link
Contributor

This patch re-enables the pylint CI checker

Adjust pylintrc to account for new checkers introduced since updating from ubuntu-18.04 to ubuntu-22-04: ignore imports, disable some style-based checks: e.g: consider-using-f-string, no-else-continue, among others.

Update test262-harness.py to use argparse instead of the now deprecated optparse.

Rename variables in jerry_client_main.py that redefined python builtins or shadowed variables from an outer scope

@@ -18,6 +18,7 @@ jobs:
- run: sudo apt update
# TODO: update checkers to current versions available in ubuntu 22.04
# - run: sudo apt install doxygen clang-format-10 cppcheck pylint python-serial
- run: sudo apt install pylint
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would keep the rest in the comment. I suspect they will be needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking they would the added back as needed, also some of their names are different in ubuntu-22.04.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment shows the missing tools, regardless of the new name. If we delete it, we don't know which tools are still not added back. Instead, we should delete them one by one when they are added back.

@@ -59,7 +59,7 @@ confidence=
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use"--disable=all --enable=classes
# --disable=W"
disable=import-star-module-level,old-octal-literal,oct-method,unpacking-in-except,parameter-unpacking,backtick,old-raise-syntax,old-ne-operator,long-suffix,dict-view-method,dict-iter-method,metaclass-assignment,next-method-called,raising-string,indexing-exception,raw_input-builtin,long-builtin,file-builtin,execfile-builtin,coerce-builtin,cmp-builtin,buffer-builtin,basestring-builtin,apply-builtin,filter-builtin-not-iterating,using-cmp-argument,useless-suppression,range-builtin-not-iterating,suppressed-message,no-absolute-import,old-division,cmp-method,reload-builtin,zip-builtin-not-iterating,intern-builtin,unichr-builtin,reduce-builtin,standarderror-builtin,unicode-builtin,xrange-builtin,coerce-method,delslice-method,getslice-method,setslice-method,input-builtin,round-builtin,hex-method,nonzero-method,map-builtin-not-iterating,missing-docstring,locally-disabled
disable=import-star-module-level,old-octal-literal,oct-method,unpacking-in-except,parameter-unpacking,backtick,old-raise-syntax,old-ne-operator,long-suffix,dict-view-method,dict-iter-method,metaclass-assignment,next-method-called,raising-string,indexing-exception,raw_input-builtin,long-builtin,file-builtin,execfile-builtin,coerce-builtin,cmp-builtin,buffer-builtin,basestring-builtin,apply-builtin,filter-builtin-not-iterating,using-cmp-argument,useless-suppression,range-builtin-not-iterating,suppressed-message,no-absolute-import,old-division,cmp-method,reload-builtin,zip-builtin-not-iterating,intern-builtin,unichr-builtin,reduce-builtin,standarderror-builtin,unicode-builtin,xrange-builtin,coerce-method,delslice-method,getslice-method,setslice-method,input-builtin,round-builtin,hex-method,nonzero-method,map-builtin-not-iterating,missing-docstring,locally-disabled,consider-using-f-string,unspecified-encoding,consider-using-from-import,consider-using-with,consider-using-in,no-else-return,no-else-continue,no-else-break,consider-using-sys-exit,chained-comparison,useless-object-inheritance,no-else-raise,singleton-comparison,redundant-u-string-prefix,import-outside-toplevel
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there new additions here?

Copy link
Contributor Author

@matetokodi matetokodi Sep 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the following ones are new additions:

  • consider-using-f-string
  • unspecified-encoding
  • consider-using-from-import
  • consider-using-with
  • consider-using-in
  • no-else-return
  • no-else-continue
  • no-else-break
  • consider-using-sys-exit
  • chained-comparison
  • useless-object-inheritance
  • no-else-raise
  • singleton-comparison
  • redundant-u-string-prefix
  • import-outside-toplevel

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How much these affect our code? Linting looses its purpose if we disable everything. If there are not much changes, we could enable them.

Copy link
Contributor Author

@matetokodi matetokodi Sep 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The big one is consider-using-f-string, that affects most places strings are formatted, the rest only affect a couple lines each.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have an exact minimum requirement for python version? f-strings have "only" been supported since py36. It may not be a big deal if we consider that officially even py37 is EOL, but some developers my be stuck on older systems where old python versions may only be available. (As I recall, that used to be the reason for using py2 for a very long time.)

Copy link
Contributor Author

@matetokodi matetokodi Sep 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think we have a set minimum, and for now I left consider-using-f-string and 2 others in specific places disabled, and fixed the rest.

@@ -297,7 +297,7 @@ ignore-comments=yes
ignore-docstrings=yes

# Ignore imports when computing similarities.
ignore-imports=no
ignore-imports=yes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the effect of this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the newer version of pylint the thresholds on duplicate code are different, and in two files the imports are flagged as duplicate code

as far as I know there is now way to disable the duplicate-code checker for only the relevant sections of the files, and I did not want to disable checking for duplicate code entirely

************* Module jerry_client_main
jerry-debugger/jerry_client_main.py:1:0: R0801: Similar lines in 2 files
==run-test-suite:[19:26]
==run-unittests:[20:28]
import os
import subprocess
import sys

import util

def get_arguments(): (duplicate-code)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could rewrite the code to avoid duplicates, if it is not too ugly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renaming the get_arguments function in one of the files is enough.

tools/runners/test262-harness.py Outdated Show resolved Hide resolved
tools/pylint/pylintrc Outdated Show resolved Hide resolved
@matetokodi matetokodi force-pushed the update_pylint branch 2 times, most recently from 8474bbe to 68cf4d9 Compare September 22, 2023 13:43
Copy link
Member

@akosthekiss akosthekiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize that some of my comments are not really related to pylint-induced changes but more to py2-vs-py3 differences. Perhaps they could be separated into another patch that deals with py3 transition.

tools/build.py Show resolved Hide resolved
tools/runners/run-test-suite.py Outdated Show resolved Hide resolved
tools/runners/run-test-suite.py Show resolved Hide resolved
jerry-debugger/jerry_client_websocket.py Show resolved Hide resolved
tools/runners/test262-harness.py Outdated Show resolved Hide resolved
tools/runners/test262-harness.py Show resolved Hide resolved
@matetokodi matetokodi force-pushed the update_pylint branch 3 times, most recently from 3190607 to 65045cb Compare October 2, 2023 12:14
@matetokodi
Copy link
Contributor Author

Over time this became an overall Python modernization patch, with the following additions:

  • Use f-strings
  • Remove from __future__ imports
  • Remove shebangs from non-executable files
  • Re-enable all pylint checkers except missing-docstring
  • Set minimum python version to 3.6

Copy link
Member

@zherczeg zherczeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

tools/build.py Outdated Show resolved Hide resolved
tools/runners/run-test-suite-test262.py Outdated Show resolved Hide resolved
Update code to conform to the newer version of pylint available in
ubuntu-22.04, with few exceptions:
    - disabled `import-outside-toplevel` for `main()` in
      `jerry_client.py`
    - disabled `consider-using-with` for the logfile of `TestSuite` in
      `test262-harness.py` as using `with` is not practical in that case

Update test262-harness.py to use argparse instead of the now deprecated
optparse

Rename variables in jerry_client_main.py that redefined python builtins
or shadowed variables from an outer scope

Update python files to use f-stirngs

Add minimum python versions (3.6 and 3.8) to the CI jobs: without it the
default python version did not support the `with` statement for
`subprocess.Popen` used in `build.py` on macos, or in some cases f-stirngs

Remove `from __future__` imports that are no-ops in python 3

Remove shebang from non executable files

Re-enable most pylint checkers, except `missing-docstring`

JerryScript-DCO-1.0-Signed-off-by: Máté Tokodi [email protected]
Copy link
Member

@akosthekiss akosthekiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@akosthekiss akosthekiss merged commit bc408b1 into jerryscript-project:master Oct 25, 2023
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants