Test failure System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true

[v-wenyuxu]

Run: runtime-coreclr libraries-jitstress 20221019.1

Failed test:

net7.0-windows-Release-arm64-CoreCLR_checked-jitstress1_tiered-Windows.10.Arm64v8.Open

- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true

net7.0-windows-Release-arm64-CoreCLR_checked-jitminopts-Windows.10.Arm64v8.Open

- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true

Error message:

Half-way through waiting for remote process.
Memory load: 50

Image Name                     PID Services

========================= ======== ============================================
svchost.exe                   1552 BrokerInfrastructure, DcomLaunch, Power,

SystemEventsBroker

svchost.exe                   1720 RpcEptMapper, RpcSs

svchost.exe                   1800 LSM

svchost.exe                   2028 BTAGService

svchost.exe                   2036 BthAvctpSvc

svchost.exe                   2044 nsi

svchost.exe                      8 TermService

svchost.exe                   1220 bthserv

svchost.exe                   2120 EventLog

svchost.exe                   2124 NcbService

svchost.exe                   2140 Wcmsvc

svchost.exe                   2148 Schedule

svchost.exe                   2168 TimeBrokerSvc

svchost.exe                   2300 ProfSvc

svchost.exe                   2396 Dhcp

svchost.exe                   2480 DisplayEnhancementService

svchost.exe                   2488 UserManager

svchost.exe                   2504 hidserv

svchost.exe                   2512 SEMgrSvc

svchost.exe                   2684 DeviceAssociationService

svchost.exe                   2692 WinHttpAutoProxySvc

svchost.exe                   2700 UmRdpService

svchost.exe                   2708 TabletInputService

svchost.exe                   2716 NlaSvc

svchost.exe                   3132 LanmanWorkstation

svchost.exe                   3140 StateRepository

svchost.exe                   3152 Dnscache

svchost.exe                   3220 netprofm

svchost.exe                   3384 SessionEnv

svchost.exe                   3644 CertPropSvc

svchost.exe                   3576 EventSystem

svchost.exe                   3848 Themes

svchost.exe                   3864 SysMain

svchost.exe                   3876 SensrSvc

svchost.exe                   4200 CoreMessagingRegistrar

svchost.exe                   4208 SENS

svchost.exe                   4320 AudioEndpointBuilder

svchost.exe                   4328 FontCache

svchost.exe                   4432 DispBrokerDesktopSvc

svchost.exe                   4832 RmSvc

svchost.exe                   4840 winmgmt

svchost.exe                   4620 Audiosrv

svchost.exe                   5204 WwanSvc

svchost.exe                   5212 DusmSvc

svchost.exe                   5268 WlanSvc

svchost.exe                   5384 lfsvc


Stack trace
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 224
   at System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true() in /_/src/libraries/System.Transactions.Local/tests/OleTxTests.cs:line 549
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs:line 69

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @roji
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run: runtime-coreclr libraries-jitstress 20221019.1

Failed test:

net7.0-windows-Release-arm64-CoreCLR_checked-jitstress1_tiered-Windows.10.Arm64v8.Open

- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true

net7.0-windows-Release-arm64-CoreCLR_checked-jitminopts-Windows.10.Arm64v8.Open

- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true

Error message:

Half-way through waiting for remote process.
Memory load: 50

Image Name                     PID Services

========================= ======== ============================================
svchost.exe                   1552 BrokerInfrastructure, DcomLaunch, Power,

SystemEventsBroker

svchost.exe                   1720 RpcEptMapper, RpcSs

svchost.exe                   1800 LSM

svchost.exe                   2028 BTAGService

svchost.exe                   2036 BthAvctpSvc

svchost.exe                   2044 nsi

svchost.exe                      8 TermService

svchost.exe                   1220 bthserv

svchost.exe                   2120 EventLog

svchost.exe                   2124 NcbService

svchost.exe                   2140 Wcmsvc

svchost.exe                   2148 Schedule

svchost.exe                   2168 TimeBrokerSvc

svchost.exe                   2300 ProfSvc

svchost.exe                   2396 Dhcp

svchost.exe                   2480 DisplayEnhancementService

svchost.exe                   2488 UserManager

svchost.exe                   2504 hidserv

svchost.exe                   2512 SEMgrSvc

svchost.exe                   2684 DeviceAssociationService

svchost.exe                   2692 WinHttpAutoProxySvc

svchost.exe                   2700 UmRdpService

svchost.exe                   2708 TabletInputService

svchost.exe                   2716 NlaSvc

svchost.exe                   3132 LanmanWorkstation

svchost.exe                   3140 StateRepository

svchost.exe                   3152 Dnscache

svchost.exe                   3220 netprofm

svchost.exe                   3384 SessionEnv

svchost.exe                   3644 CertPropSvc

svchost.exe                   3576 EventSystem

svchost.exe                   3848 Themes

svchost.exe                   3864 SysMain

svchost.exe                   3876 SensrSvc

svchost.exe                   4200 CoreMessagingRegistrar

svchost.exe                   4208 SENS

svchost.exe                   4320 AudioEndpointBuilder

svchost.exe                   4328 FontCache

svchost.exe                   4432 DispBrokerDesktopSvc

svchost.exe                   4832 RmSvc

svchost.exe                   4840 winmgmt

svchost.exe                   4620 Audiosrv

svchost.exe                   5204 WwanSvc

svchost.exe                   5212 DusmSvc

svchost.exe                   5268 WlanSvc

svchost.exe                   5384 lfsvc


Stack trace
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 224
   at System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true() in /_/src/libraries/System.Transactions.Local/tests/OleTxTests.cs:line 549
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs:line 69

Author:	v-wenyuxu
Assignees:	-
Labels:	arch-arm64, area-System.Transactions, os-windows, JitStress, untriaged, blocking-clean-ci-optional
Milestone:	-

[roji]

I'm OOF for the next week so won't be able to look into this... But could this be related to the RemoteExecutor rather than to distributed transactions? Or is there a way to get more visibility on what's going on there?

/cc @stephentoub

[v-wenyuxu]

Failed again in: runtime-coreclr libraries-jitstress 20221027.1

Failed test:

net7.0-windows-Release-arm64-CoreCLR_checked-zapdisable-Windows.10.Arm64v8.Open

- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true

net7.0-windows-Release-arm64-CoreCLR_checked-jitstress2_tiered-Windows.10.Arm64v8.Open

- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true

net7.0-windows-Release-arm64-CoreCLR_checked-jitstress2-Windows.10.Arm64v8.Open

- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true

Error message:

Half-way through waiting for remote process.
Memory load: 47

Image Name                     PID Services

========================= ======== ============================================
svchost.exe                   1568 BrokerInfrastructure, DcomLaunch, Power,

SystemEventsBroker

svchost.exe                   1720 RpcEptMapper, RpcSs

svchost.exe                   1792 LSM

svchost.exe                    760 BDESVC

svchost.exe                   1104 TermService

svchost.exe                   1108 nsi

svchost.exe                   1112 BTAGService

svchost.exe                   1236 BthAvctpSvc

svchost.exe                   1228 bthserv

svchost.exe                   2068 NcbService

svchost.exe                   2076 TimeBrokerSvc

svchost.exe                   2252 Wcmsvc

svchost.exe                   2264 Schedule

svchost.exe                   2348 EventLog

svchost.exe                   2360 DisplayEnhancementService

svchost.exe                   2384 ProfSvc

svchost.exe                   2408 hidserv

svchost.exe                   2536 Dhcp

svchost.exe                   2592 TabletInputService

svchost.exe                   2816 UmRdpService

svchost.exe                   2832 DeviceAssociationService

svchost.exe                   2840 UserManager

svchost.exe                   2992 camsvc

svchost.exe                   3000 NlaSvc

svchost.exe                   3028 StateRepository

svchost.exe                   3328 LanmanWorkstation

svchost.exe                   3400 SEMgrSvc

svchost.exe                   3408 Dnscache

svchost.exe                   3444 netprofm

svchost.exe                   3600 SessionEnv

svchost.exe                   3716 CertPropSvc

svchost.exe                   3464 SensrSvc

svchost.exe                   2896 Themes

svchost.exe                   3968 SysMain

svchost.exe                   4108 EventSystem

svchost.exe                   4268 SENS

svchost.exe                   4308 AudioEndpointBuilder

svchost.exe                   4304 FontCache

svchost.exe                   4440 CoreMessagingRegistrar

svchost.exe                   4604 DispBrokerDesktopSvc

svchost.exe                   4804 Audiosrv

svchost.exe                   4812 winmgmt

svchost.exe                   4924 DusmSvc

svchost.exe                   4944 WwanSvc

svchost.exe                   4996 WlanSvc

svchost.exe                   5048 RmSvc


Stack trace
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 224
   at System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true() in /_/src/libraries/System.Transactions.Local/tests/OleTxTests.cs:line 549
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs:line 64

[roji]

Thanks, I'll be looking into this, but it may take a bit of time due to other priorities.

[v-wenyuxu]

Failed again in: runtime-coreclr libraries-jitstress2-jitstressregs 20221105.1

Failed tests:

net7.0-windows-Release-arm64-CoreCLR_checked-jitstress2_jitstressregs4-Windows.10.Arm64v8.Open
- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true
net7.0-windows-Release-arm64-CoreCLR_checked-jitstress2_jitstressregs2-Windows.10.Arm64v8.Open
- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true
net7.0-windows-Release-arm64-CoreCLR_checked-jitstress2_jitstressregs0x10-Windows.10.Arm64v8.Open
- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true
net7.0-windows-Release-arm64-CoreCLR_checked-jitstress2_jitstressregs0x80-Windows.10.Arm64v8.Open
- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true

Error message:

Half-way through waiting for remote process.
Memory load: 55

Image Name                     PID Services                                    
========================= ======== ============================================
svchost.exe                   1584 BrokerInfrastructure, DcomLaunch, Power,    
                                  SystemEventsBroker                          
svchost.exe                   1744 RpcEptMapper, RpcSs                         
svchost.exe                   1808 LSM                                         
svchost.exe                    616 nsi                                         
svchost.exe                   1008 TermService                                 
svchost.exe                   1164 BthAvctpSvc                                 
svchost.exe                   1424 bthserv                                     
svchost.exe                   1380 BTAGService                                 
svchost.exe                   2068 SEMgrSvc                                    
svchost.exe                   2076 Wcmsvc                                      
svchost.exe                   2084 NcbService                                  
svchost.exe                   2192 Schedule                                    
svchost.exe                   2240 ProfSvc                                     
svchost.exe                   2364 DisplayEnhancementService                   
svchost.exe                   2360 hidserv                                     
svchost.exe                   2436 Dhcp                                        
svchost.exe                   2432 EventLog                                    
svchost.exe                   2484 UserManager                                 
svchost.exe                   2600 UmRdpService                                
svchost.exe                   2608 TabletInputService                          
svchost.exe                   2760 NlaSvc                                      
svchost.exe                   2780 TimeBrokerSvc                               
svchost.exe                   3028 DeviceAssociationService                    
svchost.exe                   3116 LanmanWorkstation                           
svchost.exe                   3124 StateRepository                             
svchost.exe                   3292 Dnscache                                    
svchost.exe                   3384 netprofm                                    
svchost.exe                   3460 SessionEnv                                  
svchost.exe                   3608 CertPropSvc                                 
svchost.exe                   3720 CoreMessagingRegistrar                      
svchost.exe                   3828 DispBrokerDesktopSvc                        
svchost.exe                   3200 SensrSvc                                    
svchost.exe                   4140 FontCache                                   
svchost.exe                   4148 Themes                                      
svchost.exe                   4156 SysMain                                     
svchost.exe                   4164 EventSystem                                 
svchost.exe                   4364 SENS                                        
svchost.exe                   4420 AudioEndpointBuilder                        
svchost.exe                   4592 RmSvc                                       
svchost.exe                   4844 winmgmt                                     
svchost.exe                   4852 Audiosrv                                    
svchost.exe                   5008 DusmSvc                                     
svchost.exe                   4112 WlanSvc                                     
svchost.exe                   4116 lfsvc                                       
svchost.exe                   4212 WwanSvc                                     
svchost.exe                   5276 ShellHWDetec

Stack trace:

   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 224
  at System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true() in /_/src/libraries/System.Transactions.Local/tests/OleTxTests.cs:line 549
  at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
  at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs:line 64

[jakobbotsch]

@roji Can we disable the test in the meantime? We hit this frequently on JIT pipelies.

[roji]

Sorry for the delay on this, am currently away on vacation. Will fix or disable the test tomorrow.

[jakobbotsch]

Sorry for the delay on this, am currently away on vacation. Will fix or disable the test tomorrow.

No worries, enjoy your vacation.

[v-wenyuxu]

Failed in Run: runtime-coreclr libraries-jitstress 20221113.1

Failed tests:

net7.0-windows-Release-arm64-CoreCLR_checked-no_tiered_compilation-Windows.10.Arm64v8.Open
- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true

net7.0-windows-Release-arm64-CoreCLR_checked-jitstress1_tiered-Windows.10.Arm64v8.Open
- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true

net7.0-windows-Release-arm64-CoreCLR_checked-jitstress1-Windows.10.Arm64v8.Open
- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true

net7.0-windows-Release-arm64-CoreCLR_checked-tailcallstress-Windows.10.Arm64v8.Open
- System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true

Error message:

Half-way through waiting for remote process.
Memory load: 57

Image Name                     PID Services                                    
========================= ======== ============================================
svchost.exe                   1588 BrokerInfrastructure, DcomLaunch, Power,    
                                  SystemEventsBroker                          
svchost.exe                   1748 RpcEptMapper, RpcSs                         
svchost.exe                   1808 LSM                                         
svchost.exe                   1028 BDESVC                                      
svchost.exe                   1144 TermService                                 
svchost.exe                    964 BthAvctpSvc                                 
svchost.exe                   1184 nsi                                         
svchost.exe                   1412 bthserv                                     
svchost.exe                   1272 BTAGService                                 
svchost.exe                   2088 NcbService                                  
svchost.exe                   2096 TimeBrokerSvc                               
svchost.exe                   2268 Wcmsvc                                      
svchost.exe                   2276 Schedule                                    
svchost.exe                   2284 EventLog                                    
svchost.exe                   2372 ProfSvc                                     
svchost.exe                   2480 DisplayEnhancementService                   
svchost.exe                   2488 hidserv                                     
svchost.exe                   2584 Dhcp                                        
svchost.exe                   2580 DeviceAssociationService                    
svchost.exe                   2636 UserManager                                 
svchost.exe                   2652 UmRdpService                                
svchost.exe                   2740 TabletInputService                          
svchost.exe                   3008 WinHttpAutoProxySvc                         
svchost.exe                   3052 NlaSvc                                      
svchost.exe                   2596 LanmanWorkstation                           
svchost.exe                   2900 StateRepository                             
svchost.exe                   3432 Dnscache                                    
svchost.exe                   3440 netprofm                                    
svchost.exe                   3540 SessionEnv                                  
svchost.exe                   3688 CertPropSvc                                 
svchost.exe                   3756 CoreMessagingRegistrar                      
svchost.exe                   3836 SEMgrSvc                                    
svchost.exe                   3904 DispBrokerDesktopSvc                        
svchost.exe                   3092 SensrSvc                                    
svchost.exe                   4176 FontCache                                   
svchost.exe                   4324 Themes                                      
svchost.exe                   4332 EventSystem                                 
svchost.exe                   4340 SysMain                                     
svchost.exe                   4508 SENS                                        
svchost.exe                   4536 AudioEndpointBuilder                        
svchost.exe                   4628 RmSvc                                       
svchost.exe                   5032 Audiosrv                                    
svchost.exe                   5040 winmgmt                                     
svchost.exe                   4300 WwanSvc                                     
svchost.exe                   4388 DusmSvc                                     
svchost.exe                   5156 WlanSvc     

Stack trace:

   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 224
  at System.Transactions.Tests.OleTxTests.ImplicitDistributedTransactions_cannot_be_changed_after_being_read_as_true() in /_/src/libraries/System.Transactions.Local/tests/OleTxTests.cs:line 549
  at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
  at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs:line 64

[roji]

The failures reports above are all from jitress on arm64 on Windows, and seems to indicate a timeout from RemoteExecutor (see #74320 for a general discussion on unexplained RemoteExecutor timeouts, mostly on jitstress). I can't think of a specific reason for the test being hung, so for now I'll disable the test to avoid further failures, and to check whether the hanging/timeout moves to another test (in case the hanging depends on timing).