Bug: Admin - New Installs, Scheduled Jobs Will Not Load Default Installed Jobs On First Visit Flashing A Notification To Restart Stuck In A Loop (5.1.0)

[thabaum]

Issue

Description

Scheduled Jobs component in the Control Panel > Admin Dashboard has a notification that flashes repeatedly by clearing itself and then appearing again in a loop and does not stop until navigating to a new page and also the page needs to be refreshed as it will be keeping the output logs in VS busy until you do. This will make the Control Panel > Admin Dashboard UI button control inactive as well.

https:/oqtane/oqtane.framework/blob/dev/Oqtane.Client/Modules/Admin/Jobs/Index.razor

Screenshots

Additional Context

I have also noticed the double flashes (2 page UI reloads/refreshes) within the pages for static SSR and wonder if some caching or something needs to come into play to smooth that out to feel like it is one page load?

I just happen to be clicking through all of the admin pages today to discover this issue as I looked to see if any admin pages didn't double flash in Visual Studio IIS Express. Currently this is the only page with this odd of a behavior with this notification loop.

More info...

After investigating more it went away, so I almost closed the issue. But then I noticed how there where 2 scheduled jobs, and 0 showing before.

So after you stop the site in visual studio, or restart the app (only stable way is with stopping VS) then starting it again the problem goes away.

This was seen in all render modes. It is a fresh install issue only on first visit, then restart and it goes away displaying the two jobs shown below.

Out of all pages tested located in the admin control panel dashboard parent pages it felt like admin/system/Index.razor or "System Info" loaded the quickest without any double flashing going on, just the form and UI is loaded, then data in the form comes later as you would expect. Site Settings will load the entire UI twice compared to System Info, or at least it feels this way.

[zyhfish]

Hi @thabaum , is there a clean reproduce steps or any screencast to help to understand the issue? thx

[thabaum]

@zyhfish hello, here are some steps to reproduce.

1. Create a clean visual studio instance of Oqtane in a folder downloaded from the dev branch (5.1.0)
2. Build the solution
3. Install Oqtane from within visual studio IIS Express Oqtane.Server.
4. Login as host.
5. Click on the administration dashboard UI button control from inside of the administration control panel presented when clicking on the cog icon.
6. Click on Scheduled Jobs from within the Administration Dashboard.
7. Watch the behavior described (flashing notification to restart)
8. Close or stop the debugging.
9. Run the application again ( a restart for Oqtane since doing this in app while running visual studio will pretty much force you to do the same).
10. Notice this issue is no longer a problem and 2 scheduled jobs are installed that did not show up prior. As if they are being installed on the first visit.

Thanks for asking hopefully someone can reproduce. It is odd and only so far known to happen on a fresh installation first visit to Scheduled Jobs administration page. After that there are no issues as these get installed and the app is restarted.

The page I am visiting is the Admin Jobs page component.

This does in fact still present an issue. Again once you restart the server (visual studio) this is no longer an issue and 2 jobs are shown in the list.

If you want to ask how I was able to get those screenshots, I wonder that myself as they blink fast.

This error shows up but you cannot click on any links other than navigation menu links, control panel cog icon loses interactivity until you restart as well. Restart and all is working as expected and 2 jobs appear.

When you close the browser, normally it stops visual studio but this time it wont, it continues to do some task in background so you have to actually hit the stop in visual studio and then run it again in debug mode. Very strange behavior. If you close Visual Studio it is like it wants to keep running in the background. Almost like you have to short circuit it to get it to stop at times.

Error In Logs

Log Message: An Unexpected Error Has Occurred In Oqtane.Modules.Admin.Jobs, Oqtane.Client: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..<br /><br />Please visit http://localhost:44357/admin/log?id=39 for more information

System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
 ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.CreateException(SocketError error, Boolean forAsyncThrow)
   at System.Net.Sockets.NetworkStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnection.FillAsync(Boolean async)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.ReadAsyncCore(Memory`1 buffer, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Text.Json.Serialization.ReadBufferState.ReadFromStreamAsync(Stream utf8Json, CancellationToken cancellationToken, Boolean fillBuffer)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
   at System.Net.Http.Json.HttpContentJsonExtensions.ReadFromJsonAsyncCore[T](HttpContent content, JsonSerializerOptions options, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Net.Http.Json.HttpContentJsonExtensions.ReadFromJsonAsyncCore[T](HttpContent content, JsonSerializerOptions options, CancellationToken cancellationToken)
   at Oqtane.Services.ServiceBase.GetJsonAsync[T](String uri)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
   at System.Runtime.CompilerServices.TaskAwaiter.<>c.<OutputWaitEtwEvents>b__12_0(Action innerContinuation, Task innerTask)
   at System.Threading.Tasks.SynchronizationContextAwaitTaskContinuation.<>c__DisplayClass6_0.<GetActionLogDelegate>b__0()
   at Microsoft.AspNetCore.Components.Rendering.RendererSynchronizationContext.PostAsync[TState](Task antecedent, Action`1 callback, TState state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
   at System.Runtime.CompilerServices.TaskAwaiter.<>c.<OutputWaitEtwEvents>b__12_0(Action innerContinuation, Task innerTask)
   at System.Threading.Tasks.AwaitTaskContinuation.System.Threading.IThreadPoolWorkItem.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
--- End of stack trace from previous location ---

   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.FillAsync(Boolean async)
   at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.ReadAsyncCore(Memory`1 buffer, CancellationToken cancellationToken)
   at System.Text.Json.Serialization.ReadBufferState.ReadFromStreamAsync(Stream utf8Json, CancellationToken cancellationToken, Boolean fillBuffer)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
   at System.Net.Http.Json.HttpContentJsonExtensions.ReadFromJsonAsyncCore[T](HttpContent content, JsonSerializerOptions options, CancellationToken cancellationToken)
   at Oqtane.Services.ServiceBase.GetJsonAsync[T](String uri) in C:\Dev\Oqtane\Dev\5.1.0\021924-3\Oqtane.Client\Services\ServiceBase.cs:line 146
   at Oqtane.Services.JobService.GetJobsAsync() in C:\Dev\Oqtane\Dev\5.1.0\021924-3\Oqtane.Client\Services\JobService.cs:line 20
   at Oqtane.Modules.Admin.Jobs.Index.OnParametersSetAsync() in C:\Dev\Oqtane\Dev\5.1.0\021924-3\Oqtane.Client\Modules\Admin\Jobs\Index.razor:line 58
   at Microsoft.AspNetCore.Components.ComponentBase.CallStateHasChangedOnAsyncCompletion(Task task)
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.GetErrorHandledTask(Task taskToHandle, ComponentState owningComponentState)

[Error] [Oqtane.Infrastructure.PurgeJob] An Error Occurred Executing Scheduled Job: Purge Job - System.ArgumentNullException: Value cannot be null. (Parameter 'type')
   at System.ArgumentNullException.Throw(String paramName)
   at System.Activator.CreateInstance(Type type, Boolean nonPublic, Boolean wrapExceptions)
   at Oqtane.Repository.MasterDBContext.OnConfiguring(DbContextOptionsBuilder optionsBuilder) in C:\Dev\Oqtane\Dev\5.1.0\021924-3\Oqtane.Server\Repository\Context\MasterDBContext.cs:line 54
   at Microsoft.EntityFrameworkCore.DbContext.get_ContextServices()
   at Microsoft.EntityFrameworkCore.DbContext.get_Model()
   at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.get_EntityType()
   at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.CheckState()
   at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.get_EntityQueryable()
   at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.System.Collections.Generic.IEnumerable<TEntity>.GetEnumerator()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Oqtane.Repository.JobRepository.<GetJobs>b__3_0(ICacheEntry entry) in C:\Dev\Oqtane\Dev\5.1.0\021924-3\Oqtane.Server\Repository\JobRepository.cs:line 26
   at Microsoft.Extensions.Caching.Memory.CacheExtensions.GetOrCreate[TItem](IMemoryCache cache, Object key, Func`2 factory)
   at Oqtane.Repository.JobRepository.GetJobs() in C:\Dev\Oqtane\Dev\5.1.0\021924-3\Oqtane.Server\Repository\JobRepository.cs:line 23
   at Oqtane.Infrastructure.HostedServiceBase.ExecuteAsync(CancellationToken stoppingToken) in C:\Dev\Oqtane\Dev\5.1.0\021924-3\Oqtane.Server\Infrastructure\Jobs\HostedServiceBase.cs:line 67 

This are a few logs from System Info > Log tab, Event Log and another relating. You can notice some activity in Visual Studio as well once this page has been visited that continues until you shut down visual studio itself I believe.

Hope this helps! Thank you.

[zyhfish]

Hi @thabaum , thx so much for the information. I tried to analyze the issue and found several issues need to be fixed here:

1. RenderModeBoundary.AddModuleMessage will call to StateHasChanged to refresh the render, but this will trigger the module control component re-render as well, in this case there will be a infinite loop which cause the error, to verify this, you can add below code to avoid the infinite loop in Oqtane.Client\Modules\Admin\Jobs\Index.razor:

private List<Job> _jobs;
private bool _rendered = false;

public override SecurityAccessLevel SecurityAccessLevel { get { return SecurityAccessLevel.Host; } }

protected override async Task OnParametersSetAsync()
{
    _jobs = await JobService.GetJobsAsync();
    if (_jobs.Count == 0)
    {
        if(!_rendered)
        {
            _rendered = true;
            AddModuleMessage(string.Format(Localizer["Message.NoJobs"], NavigateUrl("admin/system")), MessageType.Warning);
        }
        
	}
}

We need to find out the root cause of this infinite loop issue and make a real fix there.
2. the jobs list is empty when first load the website.
3. if we delete all the jobs, then restart the application, the jobs will come back.

I will continue to check and see whether can provide a complete fix here.

[thabaum]

@zyhfish I have also verified that deleting the default jobs if they are already exist, revisit the page (F5 or hit refresh in browser), will trigger this issue. Restart the app and the jobs re-appear in the Scheduled Jobs.

This leads me to two issues existing, one with the which could be caused by recent core changes in our code base

1. What is triggering the message refreshing on/off?
   We may have to test this against 5.0.2 and prior to see when this was introduced for further clues.

2. How/Why these jobs get installed if not installed even if uninstalled (deleted) upon restart?
   Why are the jobs not installed with the site when first installation process takes place, or are they supposed to have been? If they are necessary then maybe this is a way to reinstall them? Then maybe the delete UI button control needs to be inactive for these two Jobs, if they are a must have to run Oqtane Framework, or are these scheduled jobs supposed to be able to be removed and Oqtane can continue to run?