Web Benchmarks failing

[shihaohong]

Unsure what started this issue, but the web benchmarks seemed to have stopped working.

I submitted an almost empty PR to reproduce: #410. Thinking it might've been some dependency problem issue, I ran flutter pub get which updated the version for the transitive webdriver dependency. This caused the Mac bot to fail similarly to how the Windows bot was failing when I first spotted the issue (see my next comment for what shows on the Windows bot without any changes to pubspec.lock).

cc/ @pennzht

This is the resulting error message:

00:10 +0: D:\a\gallery\gallery\code\test\benchmarks_test.dart: Can run a web benchmark                                                                                                                 
Starting web benchmark tests ...
Launching Chrome.
Launching Chrome...
Waiting for the benchmark to report benchmark profile.
[CHROME]: 
[CHROME]: DevTools listening on ws://127.0.0.1:10000/devtools/browser/81cac324-475c-443b-a818-129f5be37024
Connecting to DevTools: ws://localhost:10000/devtools/page/F83A9AA35D3498E85674D341B9613979
Connected to Chrome tab:  (http://localhost:9999/index.html)
Launching benchmark "gallery_v2_studies_perf"
[APP] Warming up.
[APP] Warm-up finished.
[APP] ==== List of demos to be run ====
[APP] shrine@study
[APP] rally@study
[APP] crane@study
[APP] fortnightly@study
[APP] reply@study
[APP] starter@study
[APP] ==== End of list of demos to be run ====
[APP] Running demo "shrine@study"
[APP] Running demo "rally@study"
[APP] Running demo "crane@study"
[APP] Running demo "fortnightly@study"
[APP] Running demo "reply@study"
[APP] Running demo "starter@study"
[APP] All demos finished.
Shell: Failed to interpret the Chrome trace contents. The trace was saved in ./chrome-trace.json
Extracted 269 measured frames.
Skipped 32 non-measured frames.

02:50 +0 -1: D:\a\gallery\gallery\code\test\benchmarks_test.dart: Can run a web benchmark [E]                                                                                                          
  NoSuchMethodError: The getter 'tdur' was called on null.
  Receiver: null
  Tried calling: tdur
  dart:core                                       Object.noSuchMethod
  package:web_benchmarks/src/browser.dart 452:15  _computeAverageDuration.<fn>
  dart:_internal                                  ListIterable.fold
  package:web_benchmarks/src/browser.dart 451:8   _computeAverageDuration
  package:web_benchmarks/src/browser.dart 388:32  BlinkTraceSummary.fromJson
  package:web_benchmarks/src/runner.dart 159:35   BenchmarkServer.run.<fn>
  ===== asynchronous gap ===========================
  dart:async                                      _Completer.completeError
  package:web_benchmarks/src/runner.dart 219:23   BenchmarkServer.run.<fn>
  ===== asynchronous gap ===========================
  dart:async                                      _asyncThenWrapperHelper
  package:web_benchmarks/src/runner.dart          BenchmarkServer.run.<fn>
  package:shelf/src/cascade.dart 74:54            Cascade.handler.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:async                                      Future.then
  package:shelf/src/cascade.dart 73:58            Cascade.handler.<fn>
  package:shelf/src/cascade.dart 73:47            Cascade.handler.<fn>.<fn>
  dart:async                                      new Future.sync
  package:shelf/src/cascade.dart 73:21            Cascade.handler.<fn>
  package:shelf/shelf_io.dart 101:29              handleRequest
  package:shelf/shelf_io.dart 63:34               serveRequests.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:_http                                      _HttpServer.listen
  package:shelf/shelf_io.dart 63:14               serveRequests.<fn>
  package:shelf/src/util.dart 21:20               catchTopLevelErrors
  package:shelf/shelf_io.dart 62:3                serveRequests
  package:web_benchmarks/src/runner.dart 237:7    BenchmarkServer.run
  ===== asynchronous gap ===========================
  dart:async                                      _asyncThenWrapperHelper
  package:web_benchmarks/server.dart 61:5         serveWebBenchmark
  test\benchmarks_test.dart 27:30                 main.<fn>
  test\benchmarks_test.dart 24:35                 main.<fn>
  

02:50 +0 -1: loading D:\a\gallery\gallery\code\test\demo_descriptions_test.dart    

[shihaohong]

The error logs is slightly different on windows in #409:

00:12 +0: D:\a\gallery\gallery\code\test\benchmarks_test.dart: Can run a web benchmark                                                                                                                 
Starting web benchmark tests ...
Launching Chrome.
Launching Chrome...
Waiting for the benchmark to report benchmark profile.
[CHROME]: 
[CHROME]: DevTools listening on ws://127.0.0.1:10000/devtools/browser/105bd672-44c8-42b1-8229-593a17b6eff6
Connecting to DevTools: ws://localhost:10000/devtools/page/6DB323E3309ADD70AB680AE3099EDED6
Connected to Chrome tab:  (http://localhost:9999/index.html)
Launching benchmark "gallery_v2_studies_perf"
[APP] Warming up.
[APP] Warm-up finished.
[APP] ==== List of demos to be run ====
[APP] shrine@study
[APP] rally@study
[APP] crane@study
[APP] fortnightly@study
[APP] reply@study
[APP] starter@study
[APP] ==== End of list of demos to be run ====
[APP] Running demo "shrine@study"
[APP] Running demo "rally@study"
[APP] Running demo "crane@study"
[APP] Running demo "fortnightly@study"
[APP] Running demo "reply@study"
[APP] Running demo "starter@study"
[APP] All demos finished.
Shell: Failed to interpret the Chrome trace contents. The trace was saved in ./chrome-trace.json
Extracted 293 measured frames.
Skipped 41 non-measured frames.

02:49 +0 -1: D:\a\gallery\gallery\code\test\benchmarks_test.dart: Can run a web benchmark [E]                                                                                                          
  NoSuchMethodError: The getter 'tdur' was called on null.
  Receiver: null
  Tried calling: tdur
  dart:core                                       Object.noSuchMethod
  package:web_benchmarks/src/browser.dart 452:15  _computeAverageDuration.<fn>
  dart:_internal                                  ListIterable.fold
  package:web_benchmarks/src/browser.dart 451:8   _computeAverageDuration
  package:web_benchmarks/src/browser.dart 388:32  BlinkTraceSummary.fromJson
  package:web_benchmarks/src/runner.dart 159:35   BenchmarkServer.run.<fn>
  ===== asynchronous gap ===========================
  dart:async                                      _Completer.completeError
  package:web_benchmarks/src/runner.dart 219:23   BenchmarkServer.run.<fn>
  ===== asynchronous gap ===========================
  dart:async                                      _asyncThenWrapperHelper
  package:web_benchmarks/src/runner.dart          BenchmarkServer.run.<fn>
  package:shelf/src/cascade.dart 74:54            Cascade.handler.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:async                                      Future.then
  package:shelf/src/cascade.dart 73:58            Cascade.handler.<fn>
  package:shelf/src/cascade.dart 73:47            Cascade.handler.<fn>.<fn>
  dart:async                                      new Future.sync
  package:shelf/src/cascade.dart 73:21            Cascade.handler.<fn>
  package:shelf/shelf_io.dart 101:29              handleRequest
  package:shelf/shelf_io.dart 63:34               serveRequests.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:_http                                      _HttpServer.listen
  package:shelf/shelf_io.dart 63:14               serveRequests.<fn>
  package:shelf/src/util.dart 21:20               catchTopLevelErrors
  package:shelf/shelf_io.dart 62:3                serveRequests
  package:web_benchmarks/src/runner.dart 237:7    BenchmarkServer.run
  ===== asynchronous gap ===========================
  dart:async                                      _asyncThenWrapperHelper
  package:web_benchmarks/server.dart 61:5         serveWebBenchmark
  test\benchmarks_test.dart 27:30                 main.<fn>
  test\benchmarks_test.dart 24:35                 main.<fn>