Skip to content
This repository has been archived by the owner on Sep 5, 2024. It is now read-only.

Resolve Jenkins fails with Angular Material and Angular 1.3.x #6699

Closed
ThomasBurleson opened this issue Jan 15, 2016 · 10 comments
Closed

Resolve Jenkins fails with Angular Material and Angular 1.3.x #6699

ThomasBurleson opened this issue Jan 15, 2016 · 10 comments
Assignees
@topherfangio
Labels
P0: critical Critical issues that must be addressed immediately.
Milestone
1.0.5

Comments

@ThomasBurleson
Copy link
Contributor

Jenkins AngularJS Material Testings

Safari 6.2.8 (Mac OS X 10.8.5) md-input-container directive should show error on $submitted and $invalid FAILED
    Expected '<md-input-container class=""><input ng-model="foo" class="ng-pristine ng-untouched ng-valid md-input" id="input_113" tabindex="0" aria-invalid="true"><div class="md-errors-spacer"></div><label for="input_113"></label></md-input-container>to have class 'md-input-invalid'.
    /Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/src/components/input/input.spec.js:74:54
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
......................................................................
Safari 6.2.8 (Mac OS X 10.8.5) <md-toolbar> works with ng-show FAILED
    Expected '0px' not to equal '0px'.
    /Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/src/components/toolbar/toolbar.spec.js:167:48
    invoke@/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/node_modules/angular/angular.js:4219:22
    workFn@/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/node_modules/angular-mocks/angular-mocks.js:2475:26
Safari 6.2.8 (Mac OS X 10.8.5) <md-toolbar> works with ng-hide FAILED
    Expected '-32px' to equal '0px'.
    /Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/src/components/toolbar/toolbar.spec.js:219:44
    invoke@/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/node_modules/angular/angular.js:4219:22
    workFn@/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/node_modules/angular-mocks/angular-mocks.js:2475:26
................................................................................
................................................................................
................................................................................
....................................................
Chrome 47.0.2526 (Mac OS X 10.8.5) md-input-container directive should show error on $submitted and $invalid FAILED
    Expected '<md-input-container class=""><input ng-model="foo" class="ng-pristine ng-untouched ng-valid md-input" id="input_113" tabindex="0" aria-invalid="true"><div class="md-errors-spacer"></div><label for="input_113"></label></md-input-container>to have class 'md-input-invalid'.
        at Object.<anonymous> (/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/src/components/input/input.spec.js:74:43)
................................................................................
................................................................................
......................
Safari 6.2.8 (Mac OS X 10.8.5): Executed 1840 of 1849 (3 FAILED) (skipped 9) (38.579 secs / 37.777 secs)
................................................................................
................................................................................
................................................................................
............................................................
Chrome 47.0.2526 (Mac OS X 10.8.5) <md-toolbar> works with ng-show FAILED
    Expected '0px' not to equal '0px'.
        at Object.<anonymous> (/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/src/components/toolbar/toolbar.spec.js:167:41)
        at Object.invoke (/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/node_modules/angular/angular.js:4219:17)
        at Object.workFn (/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/node_modules/angular-mocks/angular-mocks.js:2475:20)
Chrome 47.0.2526 (Mac OS X 10.8.5) <md-toolbar> works with ng-hide FAILED
    Expected '-32px' to equal '0px'.
        at Object.<anonymous> (/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/src/components/toolbar/toolbar.spec.js:219:37)
        at Object.invoke (/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/node_modules/angular/angular.js:4219:17)
        at Object.workFn (/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/node_modules/angular-mocks/angular-mocks.js:2475:20)
...........
Firefox 43.0.0 (Mac OS X 10.8.0) md-input-container directive should show error on $submitted and $invalid FAILED
    Expected '<md-input-container class=""><input aria-invalid="true" tabindex="0" id="input_113" class="ng-pristine ng-untouched ng-valid md-input" ng-model="foo"><div class="md-errors-spacer"></div><label for="input_113"></label></md-input-container>to have class 'md-input-invalid'.
    @/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/src/components/input/input.spec.js:74:5
................................................................................
................................................................................
...........................................
Chrome 47.0.2526 (Mac OS X 10.8.5): Executed 1840 of 1849 (3 FAILED) (skipped 9) (50.363 secs / 49.607 secs)
................................................................................
..................
Firefox 43.0.0 (Mac OS X 10.8.0) <md-toolbar> works with ng-show FAILED
    Expected '0px' not to equal '0px'.
    @/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/src/components/toolbar/toolbar.spec.js:167:5
    invoke@/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/node_modules/angular/angular.js:4219:14
    workFn@/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/node_modules/angular-mocks/angular-mocks.js:2475:11
Firefox 43.0.0 (Mac OS X 10.8.0) <md-toolbar> works with ng-hide FAILED
    Expected '-24px' to equal '0px'.
    @/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/src/components/toolbar/toolbar.spec.js:219:5
    invoke@/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/node_modules/angular/angular.js:4219:14
    workFn@/Users/jenkins/JenkinsHome/workspace/angularjs-material-version-testing/node_modules/angular-mocks/angular-mocks.js:2475:11
.......................................
Firefox 43.0.0 (Mac OS X 10.8.0): Executed 1840 of 1849 (3 FAILED) (skipped 9) (57.329 secs / 55.872 secs)
TOTAL: 9 FAILED, 5511 SUCCESS
[02:07:04] Karma exited with the following exit code: 1
@ThomasBurleson ThomasBurleson added the P0: critical Critical issues that must be addressed immediately. label Jan 15, 2016
@ThomasBurleson ThomasBurleson self-assigned this Jan 15, 2016
@ThomasBurleson ThomasBurleson added this to the 1.0.3 milestone Jan 15, 2016
@topherfangio
Copy link
Contributor

@ThomasBurleson Pretty sure this is just waiting on PRs to get merged: #6550 and #6276.

@ThomasBurleson
Copy link
Contributor Author

Odd issue still causing Jenkins Failures in Angular 1.3.2:

Chrome 47.0.2526 (Mac OS X 10.8.5): Executed 1848 of 1857 (skipped 9) SUCCESS (48.347 secs / 47.611 secs)
Chrome 47.0.2526 (Mac OS X 10.8.5) ERROR
  Some of your tests did a full page reload!
Chrome 47.0.2526 (Mac OS X 10.8.5): Executed 1848 of 1857 (skipped 9) ERROR (48.367 secs / 47.611 secs)
................................................................................
................................................................................
....
Firefox 43.0.0 (Mac OS X 10.8.0): Executed 1848 of 1857 (skipped 9) SUCCESS (55.863 secs / 54.373 secs)
[02:07:09] Karma exited with the following exit code: 1


--- Finished Testing AngularMaterial against AngularJS (1.3.20) ---
STATUS: FAILED

See Stack Overflow Help for head hook for debugging.

Attn: @topherfangio

@topherfangio
Copy link
Contributor

Digging in now.

@topherfangio topherfangio added the in progress Mainly for in progress PRs, but may be used for issues that require multiple PRs label Jan 18, 2016
@ThomasBurleson ThomasBurleson removed their assignment Jan 18, 2016
@topherfangio
Copy link
Contributor

@matsko Could use your feedback on this Jenkins issue. Have two potential solutions:

  1. We are thinking of adding concurrency: 1 to the karma.conf so it only runs a single browser instance at a time. I was seeing issues far more often when running Safari and Chrome together and setting this alleviated those. You have any feedback on this, good or bad?
  2. It looks like the Jenkins task isn't copying of the proper versions of angular-sanitize and angular-touch (it simply doesn't copy them). I'm not sure if it's currently causing an issue, but I can definitely see some issues down the road.

From the console output:

--- Testing AngularMaterial against AngularJS (1.3.20) ---

[copy] copied over ./tmp/angular-1.3.20/files/angular.js to ./node_modules/angular/angular.js
[copy] copied over ./tmp/angular-1.3.20/files/angular.min.js to ./node_modules/angular/angular.min.js
[copy] copied over ./tmp/angular-1.3.20/files/angular-animate.js to ./node_modules/angular-animate/angular-animate.js
[copy] copied over ./tmp/angular-1.3.20/files/angular-animate.min.js to ./node_modules/angular-animate/angular-animate.min.js
[copy] copied over ./tmp/angular-1.3.20/files/angular-route.js to ./node_modules/angular-route/angular-route.js
[copy] copied over ./tmp/angular-1.3.20/files/angular-route.min.js to ./node_modules/angular-route/angular-route.min.js
[copy] copied over ./tmp/angular-1.3.20/files/angular-aria.js to ./node_modules/angular-aria/angular-aria.js
[copy] copied over ./tmp/angular-1.3.20/files/angular-aria.min.js to ./node_modules/angular-aria/angular-aria.min.js
[copy] copied over ./tmp/angular-1.3.20/files/angular-messages.js to ./node_modules/angular-messages/angular-messages.js
[copy] copied over ./tmp/angular-1.3.20/files/angular-messages.min.js to ./node_modules/angular-messages/angular-messages.min.js
[copy] copied over ./tmp/angular-1.3.20/files/angular-mocks.js to ./node_modules/angular-mocks/angular-mocks.js
[copy] copied over ./tmp/angular-1.3.20/files/angular-mocks.min.js to ./node_modules/angular-mocks/angular-mocks.min.js

[02:05:41] Using gulpfile ~/JenkinsHome/workspace/angularjs-material-version-testing/gulpfile.js
[02:05:41] Starting 'build-scss'...

Think you could fix point 2 above and provide some feedback on point 1?

@ThomasBurleson ThomasBurleson self-assigned this Jan 19, 2016
topherfangio added a commit that referenced this issue Jan 19, 2016
@topherfangio
build: Attempt to fix some Jenkins build failures.
ef71080 
Our Jenkins CI build occassionally fails with disconnects from
browsers. Setting `concurrency: 1` in the Karma config appears
to alleviate this by ensuring that the browsers run one at a time.

References #6699.
@topherfangio
Copy link
Contributor

Fixed by ef71080.

@matsko matsko mentioned this issue Jan 20, 2016
Closed
@topherfangio
Copy link
Contributor

Reopening because we're still getting page reloads occasionally.

@topherfangio topherfangio reopened this Jan 20, 2016
ThomasBurleson pushed a commit that referenced this issue Jan 21, 2016
@matsko @ThomasBurleson
test(versions): ensure that angular-touch and angular-sanitize ar…
8ccb518 
…e included

Related #6699. Closes #6771
@ThomasBurleson ThomasBurleson modified the milestones: 1.0.3, 1.0.4 Jan 22, 2016
@topherfangio
Copy link
Contributor

Okay, so we're still getting some reload issues, but the strange thing is that they happen AFTER the tests have run AND succeeded. Based off of a little bit of digging, I think it might have something to do with the way that Karma clears the context and it's interfering with Jasmine.

Karma recently added a clearContext config option which we could set to false to see if that helps (karma-runner/karma@5fc8ee7), however, it looks like Jenkins is running an old version of Karma (0.13.15) and that config isn't available until 0.13.18 (the latest is 0.13.19).

@matsko Do you know how would we go about updating the npm packges on Jenkins?

@ThomasBurleson ThomasBurleson modified the milestones: 1.0.4, 1.0.5 Jan 30, 2016
@topherfangio
Copy link
Contributor

After the recent update of our npm modules on the CI server, the last 7 days have passed successfully, so I am going to go ahead and close this ticket.

@topherfangio
Copy link
Contributor

Reopening due to new CI failures. cc @ThomasBurleson @EladBezalel from the Slack conversation. I am going to create a PR which adds the clearContext mentioned above to see if that fixes it. I will also see if we can have a separate Karma config for the CI server.

@topherfangio topherfangio reopened this Feb 9, 2016
topherfangio added a commit to profoundry-us/material that referenced this issue Feb 9, 2016
@topherfangio
docs: Potential fix for failing CI server.
e3a3f35 
 - Add the `clearContext: false` option to hopefully fix the random
   "some tests reloaded" error.

 - Create separate CI karma config for all CI-related changes.

 - Update `test-versions.sh` to use CI Karma config and fix typos.

References angular#6699.
@topherfangio topherfangio mentioned this issue Feb 9, 2016
Closed
ThomasBurleson pushed a commit that referenced this issue Feb 12, 2016
@topherfangio @ThomasBurleson
fix(tests): improve karma config for failing CI server.
df268e8 
 - Add the `clearContext: false` option to hopefully fix the random
   "some tests reloaded" error.
 - Create separate CI karma config for all CI-related changes.
 - improve karma error capture for karma.js
 - reduce disconnectTimeout 500 mSecs
 - Update `test-versions.sh` to use CI Karma config and fix typos.

References #6699.  Closes #7094
@topherfangio
Copy link
Contributor

@ThomasBurleson I'm going to go ahead and close this as it appears that all of the recent failures (viewable at https://ci.angularjs.org/job/angular-material1-minor-versions-testing/) seem to be valid failures, not related to the issues.

@ThomasBurleson ThomasBurleson removed the in progress Mainly for in progress PRs, but may be used for issues that require multiple PRs label Apr 25, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@topherfangio topherfangio

Labels
P0: critical Critical issues that must be addressed immediately.
Projects
None yet
Milestone
1.0.5
Development

No branches or pull requests
2 participants
@topherfangio @ThomasBurleson