-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[build_ts_refs] improve caches, allow building a subset of projects #107981
[build_ts_refs] improve caches, allow building a subset of projects #107981
Conversation
…entionally-expire-extracted-caches
Pinging @elastic/kibana-operations (Team:Operations) |
Sorry for the massive number of teams pinged, this PR edits a ton of TS config files so I'm not going to wait for all reviewers as we know these updates are going to work. |
💚 Build Succeeded
Metrics [docs]Async chunks
Page load bundle
History
To update your PR or re-run it, just comment with: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
appservices changes lgtm
Pinging @elastic/apm-ui (Team:apm) |
Pinging @elastic/fleet (Team:Fleet) |
…lastic#107981) * [build_ts_refs] improve caches, allow building a subset of projects * cleanup project def script and update refs in type check script * rename browser_bazel config to avoid kebab-case * remove execInProjects() helper * list references for tsconfig.types.json for api-extractor workload * disable composite features of tsconfig.types.json for api-extractor * set declaration: true to avoid weird debug error * fix jest tests Co-authored-by: spalger <[email protected]> # Conflicts: # .gitignore # examples/hello_world/tsconfig.json # tsconfig.json # tsconfig.refs.json
…107981) (#108139) * [build_ts_refs] improve caches, allow building a subset of projects * cleanup project def script and update refs in type check script * rename browser_bazel config to avoid kebab-case * remove execInProjects() helper * list references for tsconfig.types.json for api-extractor workload * disable composite features of tsconfig.types.json for api-extractor * set declaration: true to avoid weird debug error * fix jest tests Co-authored-by: spalger <[email protected]> # Conflicts: # .gitignore # examples/hello_world/tsconfig.json # tsconfig.json # tsconfig.refs.json Co-authored-by: Kibana Machine <[email protected]>
…-png-pdf-report-type * 'master' of github.com:elastic/kibana: (101 commits) [ML] APM Latency Correlations: Field/value candidates prioritization (elastic#107370) [Reporting] Add lenience to a test on the order of asserted logs (elastic#108135) [Lens] fix do not submit invalid query in filtered metric (elastic#107542) skip flaky test (elastic#108043) fix newly introduced type error (elastic#107593) [Reporting] server side code clean up (elastic#106940) [build_ts_refs] improve caches, allow building a subset of projects (elastic#107981) [APM] Add new ftr_e2e to kibana CI and remove current e2e tests. (elastic#107593) add manage rules link to alerts dropdown (elastic#107950) [ML] Enable Index data visualizer document count chart to update time range query (elastic#106438) [Security Solutions][Detection Engine] Fixes "undefined" crash for author field by adding a migration for it (elastic#107230) [Actions UI] Fixed Jira Api token label. (elastic#107776) [Alerting UI] Fixed display permissions for edit/delete buttons when user has read only access. (elastic#107996) [Maps] fix code owners (elastic#108106) Update EMS landing page url (elastic#108102) Do not render page header for loading domains (elastic#108078) Update dependency @elastic/charts to v33.2.2 (elastic#107939) [APM] Display throughput as tps (instead of tpm) when bucket size < 60 seconds (elastic#107850) [Fleet] Fix all category count (elastic#108089) [Security Solution][Bug] - Disable alert table RBAC until fields sorted (elastic#108034) ... # Conflicts: # x-pack/plugins/reporting/server/export_types/common/generate_png.ts # x-pack/plugins/reporting/server/lib/screenshots/index.ts # x-pack/plugins/reporting/server/lib/screenshots/observable.test.ts # x-pack/plugins/reporting/server/lib/screenshots/observable.ts
node scripts/build_ts_refs
has a major flaw in the way it handles caches: we extract caches to the output directory whenever we're building on a new mergeBase and in doing so set the modified times of the output files to the moment the cache is extracted, buttsc
will only build types for a project if the output files are older than the input files.To fix this we need to make sure that the output files do not have newer timestamps than the input files, as long as the input files have changes since the cache was created. To accomplish this we use
git diff --name-only {sha-cache-was-made-on}
to get the list of files which have updated, map those to TS projects, and then set the modified time of the files from those caches tonew Date(0)
. Doing this forces TS to use the.tsbuildinfo
file to determine which files have changed and how to update the types, which is a little more than twice as fast as running a fresh build without caches in my limited testing.This PR also migrates all TS projects to be "composite" and "incremental", which means that all references outside of the project are pointing to another composite project, which builds its own types, and build info is written to a
.tsbuildinfo
file to support incrementally updating types based on specific file changes. This structure was implemented in most of the TS projects in the repo but not completed until this PR. Now any project that isn't built by bazel extendstsconfig.base.json
and is composite+incremental. Any project that is built in bazel uses thetsconfig.bazel.json
base (or thetsconfig.browser_bazel.json
variant) and is not composite or incremental.This PR integrates #107804 without requiring users to commit the updated file, and instead allows users to put
// SELF MANAGED
at the top of the file to control which projects we build types for withscripts/build_ts_refs
. Some teams have had success limiting the project scope, and it seems like we should be able to support that usage pattern better this way.Finally, this PR reworks
scripts/type_check
to only check non-composite projects asbuild_ts_refs
is executed before it runs and validates the types of all composite projects, sotype_check
only needs to check the non-composite projects. Also updated the script to not uselistr
so that it renders better.