Best practice for SurfaceView recreation (or avoiding it)

[andreimarinescu]

Hi everyone,

I'm having the following issue and I'm wondering if there's anything I could be doing differently. I'm implementing a fullscreen functionality by removing the layout containing the SurfaceView the player is drawing on and placing it into a Dialog window that is overlayed.

The issue that I'm seeing is that the video starts redrawing after a rather large amount of time (between 1.5 and 3 seconds), although the audio never stops. The SurfaceView is empty during that entire time. Restarting playback altogether would be a much less costly operation than what I'm seeing here. I've pasted below the logcat output, basically I'm only seeing that the surface gets recreated.

I'm running ExoPlayer 1.5.3 and testing on an HTC One M8 (running 6.0.1), but I've had this reported on other devices as well. The test video I'm using is the following: http://qa.jwplayer.com/support-demos/static/bunny.mp4. I've also noticed that testing over HLS with Apple's bipbop takes less, but the issue is still there.

The video re-appears a good while after the last line was outputted in logcat, but there's no further output from ExoPlayer. I've noticed the solution proposed in #1034 but I'm not really sure that that's the best way to handle this. Is there any best practice in working with this scenario?

12-22 18:53:07.241 6627-7200/com.testapp D/SurfaceUtils: set up nativeWindow 0xaee99508 for 480x368, color 0x7fa30c04, rotation 0, usage 0x42002900
12-22 18:53:07.267 6627-6627/com.testapp D/VastPlayer: Surface destroyed.
12-22 18:53:07.284 6627-6627/com.testapp W/InputEventReceiver: Attempted to finish an input event but the input event receiver has already been disposed.
12-22 18:53:07.284 6627-6627/com.testapp D/VastPlayer: Exiting fullscreen
12-22 18:53:07.285 6627-6627/com.testapp D/EventLogger: videoFormat [9.84, 3, 0]
12-22 18:53:07.303 6627-6627/com.testapp D/VastPlayer: Surface created
12-22 18:53:07.339 6627-7354/com.testapp I/OMXClient: Using client-side OMX mux.
12-22 18:53:07.347 6627-7353/com.testapp I/MediaCodec: [OMX.qcom.video.decoder.avc] setting surface generation to 6786051
12-22 18:53:07.400 6627-7354/com.testapp D/SurfaceUtils: set up nativeWindow 0xaee99508 for 480x360, color 0x7fa30c04, rotation 0, usage 0x42002900
12-22 18:53:07.412 6627-6627/com.testapp D/EventLogger: decoderInitialized [9.96, OMX.qcom.video.decoder.avc]
12-22 18:53:07.442 6627-7354/com.testapp D/SurfaceUtils: set up nativeWindow 0xaee99508 for 480x368, color 0x7fa30c04, rotation 0, usage 0x42002900

[andreimarinescu]

Any updates on this?

[raja-baz]

I have pretty much the exact same issue and would love some feedback on this. In my case, I set up a bare-bones test before even going into integrating ExoPlayer in the app I'm working on to test the viability of my approach to how I want to do fullscreen functionality. I took the demo app from the project and put in 2 surface views instead of 1. I then added a button that toggles between one and the other(by calling player.setSurface in the PlayerActivity).

It seems to take a random amount of time(though typically > 1s and sometimes as big as 4s) for the second surface view to start displaying anything which seems odd as intuitively it seems something like this should be instantaneous, no? Is there anything I'm missing? Is there a quick way to switch the output to a different surface view?

[andreimarinescu]

Confirmed, I've ran the same test (thinking that the Dialog's Window might have something to do with this), with the same results.

I'd love to have some feedback on this, at least some confirmation that this is the expected behaviour with ExoPlayer.

[raja-baz]

I managed to get this working(switching views where the video output is directed fast). I based the solution on this class from grafika

I posted a gist with the necessary modifications

Once you have that, you simply include a PlayerView in your layout, and whenever you want to direct the video output somewhere you call playerView.acquireVideoTexture() passing in a context and the DemoPlayer object. This works for me and I can implement a fast switch to a different activity that displays video in fullscreen. Hope this helps

[andreimarinescu]

I can confirm that this works nicely. Thanks, @raja-baz for sharing this!

[Android4MediaPlayer]

please pull request,thks.

[andreimarinescu]

@Android4MediaPlayer the changes discussed here only involve the app-side player implementation, not the ExoPlayer library itself. So a pull request isn't needed, you can just apply the modifications in the gist (basically, using a TextureView instead of a SurfaceView for rendering).

[raja-baz]

@Android4MediaPlayer I guess one could integrate the changes made to the DemoPlayer to demonstrate this feature, but I won't be doing a pull request for that(my current approach is too kludgy to be a general-purpose thing).

[ojw28]

It should also be noted that there are significant negative battery consumption implications around that kind of approach, so we don't really want to encourage widespread use of such techniques. In general:

• If your app is for playing long-form content this kind of approach really is best avoided. I know it's pretty annoying, but it's better for the user if you design around the limitation.
• If you app is for short-form content and you really need it, the approach can be useful.

It may be that setOutputSurface provides a more efficient solution from API level 23, although it's still slightly awkward if you don't have a surface at all for some period of time during the transition.

[raja-baz]

@ojw28 in our case the videos will be in the 3~6min range, don't know if you count that is long, but we really do need the flexibility of moving the video output around.

What are the battery implications? There is extra GPU work that needs to be done(render to texture, then render the texture to screen, instead of a one-step process) but I was under the impression that most of the battery cost when watching videos would come from the screen itself, the CPU cost of decoding, and the first render step(render video frames to texture/screen). Adding a step where a TextureSurface is rendered onto the screen shouldn't add much, should it?

[ojw28]

There are quite a few variables that make providing a single answer quite difficult. The implications are hardware dependent, and obviously variables like how bright the screen is make a big difference (e.g. if the screen is as bright as possible then the screen will account for a far greater % of battery consumption than if the screen is as dim as possible).

With the above in mind and therefore treating the numbers with some caution, I have seen data that suggests using TextureView rather than SurfaceView can result in +30% consumption during playback (the results showed ~390mA draw with TextureView v.s. ~295mA with SurfaceView). I can't say how controlled the experimental setup was. Note that the approach described above for solving this issue will be at least as bad as TextureView; probably worse.

I was mainly referring to TV shows / movies by "long-form content". Although I guess what's really important is the total time users are likely to be engaged with your application. If users are likely to be watching video for extended periods, they'd be better served if you preserve their device's battery life in preference to fancy UI transitions.

[raja-baz]

Fair enough, in our case this isn't avoidable without changing a lot of code for a single feature that isn't a core feature. We can monitor analytics and decide on whether the increased battery usage is acceptable or not, I guess.

In any case, this isn't just about fancy UI transitions. It's a question of functionality. Using the setSurface() call results in an unpredictable pause in video output. If the video is paused or buffering, for example, the screen stays black. If I had a guarantee that the video would become visible even if the time it took is as high as 5s then I'd be fine with that. But as things stand the amount of time is indeterminate and that's just not acceptable.

[ojw28]

If you do gather some data around battery consumption then it would be great if you could share it here, when you have something (ditto for anyone else on this thread :)).

[raja-baz]

I was thinking more along the lines of gathering data about usage patterns in terms of session length and if battery consumption is even a concern(if our average user watches 5min of video per day, say, and we don't have people watching more than 15min, then a relative difference of 30% in battery consumption isn't meaningful versus the experience itself).

In terms of the testing battery usage of various approaches, A/B testing, or something, I guess we could do that, but I have no clue how one would go about measuring the power draw of the app.

[andreimarinescu]

I'm really curious about this as well. I'd like to get around to setting up a test, if we do get some data we'll share it here.

[martinbonnin]

Just linking #677 here. It seems to be about the same issue and also mentions sharing the same SurfaceTexture between two TextureViews.