Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix parsing of csv template files #219

Merged
merged 5 commits into from
Sep 22, 2021
Merged

Fix parsing of csv template files #219

merged 5 commits into from
Sep 22, 2021

Conversation

gmischler
Copy link
Collaborator

@gmischler gmischler commented Sep 18, 2021

As of issue #215, here's the pull request:

Values of csv files are now converted by position, instead of content, so that no kind of text can cause an error.
I also added a converter to make sure that hex and oct color values are accepted for foreground and background in csv files.
Updated the test data to include an entry with only digits as text, to check for the regression (adequately failed before the code fix).
Updated the documentation and examples to reflect the changes and match the test data again.
While I was at it, I also included the recently improved multiline functionality in the documentation examples and in the tests.

Looks like I forgot to actually remove util.try_to_type() though. Is that used anywhere else?

*Values of csv files are converted by position, instead of content
* Updated tests to check for regression
* Updated documentation and tests to include multiline text.
fpdf/template.py Outdated
Comment on lines 90 to 93
handlers = (
("name", str.strip),
("type", str.strip),
("x1", float),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i like this approach. "in the name of deterministic behavior" why strip the fields? but this is a good place to put the actual conversions, and less lines of code than what i was considering when i put the util function in.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything got stripped in the original version, so I just left it that way in order not to change the behaviour. The alternative would be to just use str as a NOP.

fpdf/template.py Outdated
Comment on lines 79 to 81
s = s.strip()
if not s:
raise ValueError('Foreground and Background must be numeric')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe consider putting a different error message as s is "not empty" here, not "not numeric". (unless i am missing something?)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I phrased it in this way to inform the user about what's supposed to go there. "Not empty" doesn't give that information.
The documentation could also be more specific about this, but fleshing that out is a project for another time...

restrict decimal seperator replacement to float fields
fpdf/template.py Outdated
def varsep_float(s):
"""Convert to float with given decimal seperator"""
# glad to have nonlocal scoping...
return float(s.replace(decimal_sep, '.'))
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hadn't payed enough attention to decimal seperator replacement at first.
It previously was done on any field where the content didn't start with a ' (single quote).
Those are not at all garanteed to be present in text fields, so we need to restrict replacement to actual float value fields.
Fortunately a nested function sees the surrounding scope where the seperator is defined, which makes this easy.

@@ -28,6 +28,7 @@ def test_template_nominal_hardcoded(tmp_path):
"align": "I",
"text": "logo",
"priority": 2,
"multiline": 0,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did multiline become mandatory?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only changed the tests in this area. Previously multiline was not tested at all, so I added it there.
But if it's not mandatory (obviously, since it passed before), I guess both cases should be tested, so I'll remove some of them agein.
Btw.: Are ther any other fields that are optional? The documentation only mentions a default in passing, but doesn't explicitly say which are or aren't mandatory.

@codecov
Copy link

codecov bot commented Sep 21, 2021

Codecov Report

Merging #219 (bc0883f) into master (dbd4118) will increase coverage by 0.14%.
The diff coverage is 91.30%.

❗ Current head bc0883f differs from pull request most recent head f1d7802. Consider uploading reports for the commit f1d7802 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master     #219      +/-   ##
==========================================
+ Coverage   84.88%   85.03%   +0.14%     
==========================================
  Files          16       16              
  Lines        3527     3542      +15     
  Branches      710      713       +3     
==========================================
+ Hits         2994     3012      +18     
+ Misses        360      358       -2     
+ Partials      173      172       -1     
Impacted Files Coverage Δ
fpdf/template.py 65.94% <85.71%> (+2.46%) ⬆️
fpdf/image_parsing.py 93.75% <100.00%> (+0.69%) ⬆️
fpdf/syntax.py 94.25% <100.00%> (ø)
fpdf/util.py 72.00% <0.00%> (-12.00%) ⬇️
fpdf/fpdf.py 82.86% <0.00%> (+0.41%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dbd4118...f1d7802. Read the comment docs.

@Lucas-C Lucas-C merged commit a7bae5e into py-pdf:master Sep 22, 2021
@Lucas-C
Copy link
Member

Lucas-C commented Sep 22, 2021

Thank you for your contribution @gmischler !

I removed util.try_to_type & updated the CHANGELOG.md to mention it:
https:/PyFPDF/fpdf2/blob/master/CHANGELOG.md#244---not-released-yet

@Lucas-C
Copy link
Member

Lucas-C commented Sep 22, 2021

@all-contributors please add @gmischler for code

@allcontributors
Copy link

@Lucas-C

I've put up a pull request to add @gmischler! 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants