Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor PHPUnit bootstrap #96

Merged
merged 5 commits into from
Apr 6, 2018

Conversation

PhrozenByte
Copy link
Contributor

@PhrozenByte PhrozenByte commented Sep 5, 2016

This can be used as a template for Parsedown extensions (like Parsedown Extra) to reuse Parsedown's original tests. You can either extend tests (e.g. ParsedownExtraTest extends ParsedownTest) or reuse them unchanged (e.g. CommonMarkTest). Parsedown extensions simply have to implement their own TestParsedown class (file test/TestParsedown.php) and all original tests will run with a instance of this class, rather than a instance of the original Parsedown class.

This PR is a follow-up to erusev/parsedown#423, i.e. you must merge erusev/parsedown#423 first, otherwise this doesn't work.


Here are the phpunit runs of CommonMarkTest and CommonMarkTestWeak:

$ phpunit -c phpunit.xml.dist vendor/erusev/parsedown/test/CommonMarkTest.php
Output: http://pastebin.com/ZVBjdmvK

FFF.FFFFF........FF.....F.F..F...F....FF....FF..F....FFF.F..FFF  63 / 616 ( 10%)
....FFFF....F..FFF.FFFFFFFFFFF.FFF.FFFFFFFFFFFFFF.FFFFFFFFFFF.. 126 / 616 ( 20%)
FFFFFFF...FFFF.FFFFFFFF.F.F.FF.FFFF.F.F.F.FF..FFF.FF......FF... 189 / 616 ( 30%)
.F..FFFFF...F...F..F..FFFF.F.....FFFFFF.FFFFF.F.FFFFFFF.FFFFF.. 252 / 616 ( 40%)
.FFFF.FFFF.F.FF.FFFFFFFFFFF.FF.F.FFF.F.FFFFFFFFFFF.FFFFF.F...FF 315 / 616 ( 51%)
..FFF..FF..FFF...FFF.F...FFFF.F.F..F..FF..FFFFFFF.FF....FF.FFF. 378 / 616 ( 61%)
...FFF.....FF.....FFF...............F..FF.......F..FF......F... 441 / 616 ( 71%)
FFFFFFFFFFFF..FF..FFF.FFFF.FFFFF.FF....F..FFFF..FFF..F..FFF.FFF 504 / 616 ( 81%)
.FFFFF.FFF.......F...F...F.....FFFFF.FF....F.F.FF.......FF.F.F. 567 / 616 ( 92%)
.FE.FF....FF.F..FF.....FFF..F.F..F.F.....F.......

FAILURES!
Tests: 616, Assertions: 615, Failures: 333, Errors: 1.

$ phpunit -c phpunit.xml.dist vendor/erusev/parsedown/test/CommonMarkTestWeak.php
Output: http://pastebin.com/G4cBLfCT

FFF.F.F...........F.....F.F..F...F.....F....FF..F.....FF....FFF  63 / 616 ( 10%)
....F.FF........FF..........FF.FF..FFF...F.F...FF...FFFFFFFFF.. 126 / 616 ( 20%)
FFFFFFF...FFFF.FFFFFF.F.F.F..F.FFFF.F.F.F.FF..F.F.FF........... 189 / 616 ( 30%)
....FFFFF...F...F..F..F..F.F......FFFFF.FF.FF.F.FFF...F.FFF.F.. 252 / 616 ( 40%)
.FFFF.FFFF.F.FF.FFFFFFF.F.F....F.FFF.F...FFFFFFFFF.FFFFF......F 315 / 616 ( 51%)
..FFF..FF..FFF...FFF.F...FFFF.F.F..F..FF..FFFFFFF.FF....FF.FFF. 378 / 616 ( 61%)
...FFF.....FF.....FFF...............F..FF.......F..FF......F... 441 / 616 ( 71%)
FFFFFFFFFFFF..FF..FFF.FFFF.FFFFF.FF....F..FFFF..FFF..F..FFF.FFF 504 / 616 ( 81%)
.FFFFF.FFF.......F...F...F.....FFFFF.FF....F.F.FF.......FF.F.F. 567 / 616 ( 92%)
.FE.FF....FF.F..FF.....FFF..F.F..F.F.............

FAILURES!
Tests: 616, Assertions: 615, Failures: 281, Errors: 1.

This can be used as a template for Parsedown extensions (like Parsedown Extra) to reuse Parsedown's original tests. You can either extend tests (like ParsedownExtraTest) or reuse them unchanged (like CommonMarkTest). Parsedown extensions simply have to implement their own TestParsedown class (test/TestParsedown.php) and all original tests will run with a instance of this class, rather than a instance of the original Parsedown class.

This PR is a follow-up to erusev/parsedown#423, i.e. you must merge erusev/parsedown#423 first, otherwise this doesn't work.
PhrozenByte added a commit to PhrozenByte/parsedown that referenced this pull request Sep 5, 2016
This allows Parsedown extensions (like Parsedown Extra) to reuse existing Parsedown tests. See erusev/parsedown-extra#96 for details.
Failing tests don't break builds on purpose, Parsedown Extra doesn't fully comply with the CommonMark specs at the moment. We should switch to test/CommonMarkTest.php of erusev/parsedown later, see erusev/parsedown#423 for details.
.travis.yml Outdated
- 5.6
- 5.5
- 5.4
- 5.3
- nightly
- hhvm
- hhvm-nightly
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hhvm-nightly is not supported by travis anymore.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... what is just temporary until Travis updates its infrastructure - something that will happen sooner rather than later, Ubuntu Precise's end-of-life is in about 6 months.

.travis.yml Outdated
fast_finish: true
allow_failures:
- php: nightly
- php: hhvm-nightly

install:
- composer install
- composer install --prefer-dist
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll want to prefer source here aidantwoods/SecureParsedown@3693641

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aaaaaah, and I was wondering why the files are missing 😆 Thanks!

@PhrozenByte
Copy link
Contributor Author

PhrozenByte commented Mar 1, 2018

All fixed now. However, the build still fails. It's IMO a good idea to merge #85 before merging this PR (just with a updated version constraint for erusev/parsedown, should be ^1.7.1 or ^1.8.0, depending on the next release's tag). There are two reasons why the build still fail:

  1. The new CommonMarkTest is in Parsedown's dev-master, but wasn't tagged yet, i.e. composer still loads v1.7.0 which simply doesn't ship the new tests yet. You can try it locally by temporarily requiring dev-master

  2. There seems to be two regression (unrelated to this PR) about footnotes (see below; the tests didn't work for years, so...); you can ignore the other four errors showing up on Travis, they disappear when using Parsedown's dev-master

$ vendor/bin/phpunit 
PHPUnit 4.8.36 by Sebastian Bergmann and contributors.

...........................................................F.F..

Time: 193 ms, Memory: 6.50MB

There were 2 failures:

1) ParsedownExtraTest::test_ with data set #59 ('footnote', '.../data/')
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
 <li id="fn:1">
-<p>one&#160;<a href="#fnref1:1" rev="footnote" class="footnote-backref">&#8617;</a> <a href="#fnref2:1" rev="footnote" class="footnote-backref">&#8617;</a></p>
+&lt;p&gt;one&amp;#160;&lt;a href="#fnref1:1" rev="footnote" class="footnote-backref"&gt;&amp;#8617;&lt;/a&gt; &lt;a href="#fnref2:1" rev="footnote" class="footnote-backref"&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
 </li>
 <li id="fn:2">
-<p>two&#160;<a href="#fnref1:2" rev="footnote" class="footnote-backref">&#8617;</a></p>
+&lt;p&gt;two&amp;#160;&lt;a href="#fnref1:2" rev="footnote" class="footnote-backref"&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
 </li>
 <li id="fn:a">
-<p>one&#160;<a href="#fnref1:a" rev="footnote" class="footnote-backref">&#8617;</a></p>
+&lt;p&gt;one&amp;#160;&lt;a href="#fnref1:a" rev="footnote" class="footnote-backref"&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
 </li>
 <li id="fn:b">
-<p>two&#160;<a href="#fnref1:b" rev="footnote" class="footnote-backref">&#8617;</a></p>
+&lt;p&gt;two&amp;#160;&lt;a href="#fnref1:b" rev="footnote" class="footnote-backref"&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
 </li>
 </ol>
 </div>'

./parsedown-extra/vendor/erusev/parsedown/test/ParsedownTest.php:55

2) ParsedownExtraTest::test_ with data set #61 ('compound_footnote', '.../data/')
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
 '<p>footnote <sup id="fnref1:1"><a href="#fn:1" class="footnote-ref">1</a></sup> and another one <sup id="fnref1:2"><a href="#fn:2" class="footnote-ref">2</a></sup></p>
 <div class="footnotes">
 <hr />
 <ol>
 <li id="fn:1">
-<p>line 1
-line 2</p>
-<blockquote>
-<p>quote</p>
-</blockquote>
-<p>another paragraph&#160;<a href="#fnref1:1" rev="footnote" class="footnote-backref">&#8617;</a></p>
+&lt;p&gt;line 1
+line 2&lt;/p&gt;
+&lt;blockquote&gt;
+&lt;p&gt;quote&lt;/p&gt;
+&lt;/blockquote&gt;
+&lt;p&gt;another paragraph&amp;#160;&lt;a href="#fnref1:1" rev="footnote" class="footnote-backref"&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
 </li>
 <li id="fn:2">
-<p>paragraph</p>
-<p>another paragraph&#160;<a href="#fnref1:2" rev="footnote" class="footnote-backref">&#8617;</a></p>
+&lt;p&gt;paragraph&lt;/p&gt;
+&lt;p&gt;another paragraph&amp;#160;&lt;a href="#fnref1:2" rev="footnote" class="footnote-backref"&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
 </li>
 </ol>
 </div>'

./parsedown-extra/vendor/erusev/parsedown/test/ParsedownTest.php:55

FAILURES!
Tests: 64, Assertions: 68, Failures: 2.

@PhrozenByte
Copy link
Contributor Author

Here's our latest build with Parsedown 1.7.1: https://travis-ci.org/erusev/parsedown-extra/builds/350738061

Naturally the two unrelated regressions are still there 😅

@aidantwoods aidantwoods merged commit 402fed4 into erusev:master Apr 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants