flickering across video transfer

[CJHFUTURE]

Hi there is a patchy areas that jump around between images in video transfer.
This isnt so much temporal issues with the style transfer of features as much as little patches of luminance change that flicker around....Is this something to do with theWhiten-Color Transform as it doesnt happen when using the Adain method with your implementation?

[eridgd]

Deleted my earlier comment because I misread the question - in regards to flickering with video transfer, that's a common problem with style transfer approaches to varying degrees. See my response here #6 for other approaches that are more stable for video.

[CJHFUTURE]

I wonder if it's possible to add a -seed option, so can get repeatable results. Try running twice, with same -seed, eg -seed 123, and you should get the same output image each time? I see this has bee implemented in the original neuralStyle Johnson repo. What do you think?This may increase temporal coherence in video stylisation.

[eridgd]

Unfortunately, that won't solve the issue because the flickering is due to variability between image frames, not randomness in the net itself. For example, if you run stylize.py to stylize a single image file, then run it again with the same input, the outputs will be identical because the parameters are fixed and all ops are deterministic.

The jcjohnson repo is the original style transfer algorithm, which involves an optimization over a random noise image and so has a component that can be controlled with a seed. Optimization in the WCT algorithm occurs during the decoder training phase only.