Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Rx.NET 2023 roadmap, and tooling update ADR #1879

Merged
merged 4 commits into from
Mar 8, 2023
Merged

Conversation

idg10
Copy link
Collaborator

@idg10 idg10 commented Mar 1, 2023

As #1868 states, one of the steps in bringing Rx.NET 'back to life' is to "Publish a Roadmap for 2023 (covering the .NET 8.0 LTS release train) & beyond".

This PR addresses that by setting out a road map. It also provides an Architectural Decision Record (ADR) describing the first proposed step (updating the Rx.NET) in more detail, along with all the context that informs that part of our proposals.

Resolves #1880

@idg10
Copy link
Collaborator Author

idg10 commented Mar 1, 2023

I don't know to what extent any of Rx's former contributors continue to have any interest in this project, but I thought it would be worth drawing these proposals to their attention. There might be historical folk knowledge that I don't have that would point against some of these suggestions.

So to that end, I am tagging @bartdesmet @danielcweber @akarnokd @clairernovotny @shiftkey @bash @quinmars @RxDave @Daniel-Svensson @mattpodwysocki @ghuntley @LeeCampbell @SimonCropp @anaisbetts @NewellClark @RyotaMurohoshi @M-Zuber @briangru @marodev @aelij @theodorzoulias @ryanwersal @cdmihai @cabauman @dotMorten @joelhoisko @ltrzesniewski @hez2010 @PJB3005 @jkotas @eugbaranov @NickDarvey @ts2do @olevett @alfredmyers @TobBrandt-Work @NickCraver @tcberryoxinst @rytswd @Dixin @khellang @jamesdoran @MichalStrehovsky @najuqvt @martsyn @sblom @georgis in case any of them have an opinion on this.

(I think that's everyone who ever made a commit in this repo that had a technical impact on Rx. Apologies if I missed anyway.)

@danielcweber
Copy link
Collaborator

I don't know to what extent any of Rx's former contributors continue to have any interest in this project

I'm planning to adopt Rx.Async in some form once it gets a release and would happily pick up contributing to this repo again (bug fixes, etc.). I still have write access to this repo from my batch of work back in 2018/9.


Over two years have passed since Rx 5.0 was released. Microsoft actively discourages the use of UWP, so why do we still target it? One obvious answer is that there has been no new release of Rx in the intervening time. However, it's not as simple as that.

Although Microsoft pushes people strongly in the direction of WinUI v3, the fact is that if you want to write a .NET Windows Store application today, certain Windows features are only available to you if you target UWP. For example, full access to the camera (including direct access to the raw pixels from captured images) is currently (February 2023) only possible with UWP.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious, which camera API specifically? CameraCaptureUI?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was using CaptureElement in XAML and a whole bunch of types from Windows.Media.Capture.

@HowardvanRooijen
Copy link
Collaborator

HowardvanRooijen commented Mar 4, 2023

See PR #1882

@HowardvanRooijen
Copy link
Collaborator

See PR #1883

@idg10 idg10 merged commit a89f60b into main Mar 8, 2023
@idg10 idg10 deleted the planning/roadmap-2023 branch March 8, 2023 15:47
@Fijo
Copy link
Contributor

Fijo commented May 6, 2023

About platform independent rx, test sharing and Unity

I just wanted to leave some information here in regards to IL2CPP compatibility and testing with Unity. In an attempt to reuse as much of the test code from this repository as possible and I've created a pretty unorthodox automation toolchain as part of the Rx.Unity repo.

The main reason I am making this post is because I saw you, talking about sharing tests so adoptions for different platforms (like UWP) can keep running them after they've been moved out of the main repository. Thus I thought I'd share my approach to give you some insight, inspiration and in hopes for a good solution that'll be easily maintainable for everyone.

Before I forget thank you so much for the work you've been putting in to bring Rx.Net back on track. I really appreciate it.

While that (Rx.Unity) needs updating again, which I am still committed to getting back to.
I'm also planning to do some more work on there once it's possible to use it along with the official System.Reactive NuGet Packages.

Why did I have to run all the tests?
Mostly to figure out what works with IL2CPP and in Unity in general.

How the build works (only focusing on what's relevant for Rx.net):

  1. I clone the Rx.Net sources into a folder.
  2. Patching the sources (described below).
  3. Custom System.Reactive Build (described below).
  4. Sideloading partial classes from tests (described below).
  5. Including migrated UniRx-Tests to verify support for people migrating
  6. Building and running tests in Unity (described below).

Patching the sources

Build on gulp I've scripted out a few tasks to transform the tests from the dotnet/reactive repo so they run within Unity.

Here are some of the things being done to rx sources themselves:

Patch System.Reactive.csproj

  • TargetFramework => netstandard2.0
  • LangVersion => 9
  • DefineConstants => NO_NULLABLE_ATTRIBUTES
  • SignAssembly => false

Patch './Rx.NET/Source/src/System.Reactive/TaskObservable.cs'

  • Removing the [AsyncMethodBuilder(typeof(TaskObservableMethodBuilder<>))]-Attribute (Apparently this was unsupported / caused issues - I'll have to look into that again to find out more.)

Patch System.Reactive - Entire File replacements

I replace the following files with different ones or my own (Links lead to the replaced file):

Patch XUnit to NUnit

Unitys tooling, as well as the runtime test runner I'm using only really works with NUnit. Adding too many additional dependencies is kind of painful (it would propobly work for assertations and I think I'll look if that's an option for when I'm adapting the now MS Test based sources.). Luckily a few replacements were enough to cover everything being used within the tests. Asyncronous Tests the trickiest here.
For more see xunit2nunitTransformer-Method.

Patches on Tests

  • ThreadPoolScheduler.Instance is replaced with Rx.Unity.Concurrency.ThreadPoolOnlyScheduler.Instance in most test sources, as that's generally preferred when using Rx.Unity.
  • public class is replaced with public partial class in most test sources, to allow additional setup / teardown logic to be provided by additional files.
  • Various Tests are blacklisted by either making those methods private (via search and replace) or by excluding those files. The goal is ofc to get as much as possible to work in Unity but certain things just aren't there yet or would be a lot more difficult to troubleshoot. Alot of them cause the Tests to end up hanging up as well.

Custom System.Reactive Build

Building is done with dotnet build in ./Rx.NET/Source/src/System.Reactive.

=> Once the whole SchedulerDefaults.cs thing is fixed I think the whole InternalsVisibleTo.cs required for the tests is the only real thing preventing me from just using the NuGet Package of System.Reactive.

I'm only using the netstandard2.0 build for now. The latest version of Unity supports up to netstandard2.1, today. They are however working on a [https://forum.unity.com/threads/net-6-support.1059992/](full move to dotnet6) AFAIK.

Sideloading partial classes from tests

Alot of the test classes are being made partial and they are sideloaded with a custom SetUp/ TearDown-Methods specified in additional sources.
The main purpose of this is generally to switch the static instances within SchedulerDefaults to the more Unity compatible ones and to change them back on teardown.
In a few instances additional static methods are also provided (which are never actually used) but provide a workaround to force the AOT Linker to generate certain generic classes with the required specific type parameters as in rare cases it fails to make correct predictions. Having all the generic type combination being generated used to be a requirement for IL2CPP to work. Recently Unity have added support for generic type creation at runtime in IL2CPP but it is slow.

Building and running tests in Unity

Only test code and the source for Rx.Unity is build within Unity itself but not System.Reactive itself. The required files are copied into the appropriate folder by my script.

Unity tests are built and run in a via github actions. To ensure correct behaviour with IL2CPP an actual game is built including some framework that enables just running tests in that game. Then those tests are run via console. For Mac, iOS, Android and WebGL this isn't possible (or in the case of Mac I didn't get it to run that - I don't have a Mac so I couldn't troubleshoot it easily). Here an artefact with the build and a limited expiration is generated which can be manually downloaded, installed and run to verify all tests are working.

@idg10

@HowardvanRooijen
Copy link
Collaborator

@Fijo thanks very much for sharing that. We're very keen to enable better support for Unity, but it's an ecosystem we're unfamiliar with. If you're happy, we'll reach out to get your feedback when we get to those backlog items (most likely as part of v7 / v8)

@Thaina
Copy link

Thaina commented Sep 29, 2024

@HowardvanRooijen Are there any progress about changing default scheduler or anything to support other platform like unity at all?

@HowardvanRooijen
Copy link
Collaborator

Hi @Thaina

There are a number of things in flight at the moment, that we're trying to land in time for the .NET 9 launch (and .NET 6.0 going out of support, as well as giving Ix some TLC #2135 #2131 #2102) and general tidy up #2113 #2092 #2134

There was a big discussion around packaging... which would solve some of the problems we're facing #2038 - but I don't think we got to a consensus.

We've been blocked for a couple of months as the .NET Foundation has mandated a move to Managed Identities for code signing auth. This is a great idea, but since Rx was the first project to attempt this, it took a while to work out how to get the config right (dotnet-foundation/projects#387). Ian got that working this week (#2167), with the help of Chris Sfanos from the DNF, so progress is moving ahead again.

Ian has been looking at getting Async Rx into a better state:

We're hoping to do another yearly update either as part of .NET Conf or released independently at the same time.

@Thaina
Copy link

Thaina commented Sep 29, 2024

Thank you for your response

@HowardvanRooijen
Copy link
Collaborator

@Thaina - you're welcome! There's lots of stuff going on, to try and get all the bits ready for the next release. Hopefully we'll be able to make this a little clearer and more cohesive over the next month as we head towards the November milestone.

@khellang
Copy link
Member

Thanks for the hard work you're putting into this, @HowardvanRooijen and team ❤️

@HowardvanRooijen
Copy link
Collaborator

@khellang it's all @idg10 hard work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[New Maintainers] Publish a Roadmap for 2023