getCapabilities seems to leak hardware capabilities w/o a permission

[pes10k]

Apologies if I'm misreading the spec, but if I'm reading it correctly it looks like a site can learn about the visitors underlying hardware capabilities w/o a permission prompt or some other positive, affirmative action by the visitor.

Is my reading of the spec correct then, there is a FP vector exposed by the current text that would need to be mitigated (e.g. sites couldn't access it by default).

Otherwise, if this is addressed elsewhere, could you kindly point me to where, so I dont make the same mistake twice? :) Thanks!

[aboba]

WebRTC-SVC extends the existing RTCRtpCodecCapability dictionary from WebRTC to provide additional information relating to supported SVC scalability modes.

Since getCapabilities() is a static method, it only returns information related to supported codecs, and cannot vary its response based on the underlying hardware. So the application is unable to tell from getCapabilities() whether there is hardware encode or decode support. Since the application performance could vary considerably based (in particular) on whether encoding is done in software or hardware, we have had requests to enhance the capabilities (see: w3c/webrtc-nv-use-cases#53 ), but this is not part of the WebRTC-SVC specification.

Within the VP8, VP9 and AV1 codecs (the only WebRTC codecs that support SVC), a compliant decoder is required to decode anything than an encoder can send. So getCapabilities() only provides information about an endpoint's encoder. However, since SVC support in hardware is rare, this typically only relates to the scalability mode support within the software codec implementation.

[henbos]

This API does does indeed not tell you whether a codec is HW or SW. It would be very beneficial if it (or another API) did, but that is not the case today.

Since getCapabilities() is a static method, it only returns information related to supported codecs, and cannot vary its response based on the underlying hardware.

Conceivably, could there not be a codec or profile that was only implemented in hardware, and as such the existence of a particular codec/profile would indirectly tell you that?

[henbos]

WebRTC's getStats() currently leaks encoderImplementation and decoderImplementation strings though, which in some cases could tell you whether an encoder or decoder is HW or SW (depending on the UA) after-the-fact.

[aboba]

@henbos Indirect inference is not easy. All the codecs currently implemented within WebRTC can be supported in both hardware and software, so getCapabilities() by itself doesn't leak hardware information. For more demanding codecs such as AV1, stats relating to encoder performance could give an indication of exceptional performance (e.g. 4K encoding at 60 fps), but you'd need implementation strings to know exactly what was providing that performance.

[pes10k]

@aboba @henbos thanks for following up! I'm a little lost from the above notes though. Is it the case that the responses from getCapabilities() and or getStats() will be consistent for, say, every browser major version, arch pairing? Or will it be the case that the same browser release on different hardware will report different values (and so fingerprinters could use these end points to learn about underlying hardware).

Apologies for the possibly redundant questions, but the above comments seem to point in opposite directions, as best as I can understand

[aboba]

@snyderp getCapabilities can vary between browsers, although this is less common than it used to be. For example, at one point Chrome and Firefox supported VP8, but Edge and Safari did not. Now Chrome, Firefox and Edge beta return VP8 in `getCapabilities('video'), as well as H.264/AVC and VP9. Differences may emerge again once RTC support for cutting edge codecs like AV1 is added.

[pes10k]

@aboba Apologies, but I think I didn't do a good job asking the question. I'm trying to figure out whether getCapabilities() or getStats() is revealing things about the browser, or about the underlying hardware. E.g. would the same version of Chrome always give the same answers on the same OS, regardless of the device its running on? Can a site learn anything beyond whats in the existing UA from these APIs?

[aboba]

@snyderp I don't believe that getCapabilities as specified in WebRTC or ORTC reveals information about how codecs are implemented (software or hardware). This specification doesn't change the operation of getCapabilities.

At a given time, it is possible that the browsers will support different codecs. For example, at one point Chrome supported VP8, H.264 and VP9 and Edge only supported H.264. However, there are easier ways to tell browsers apart than by using getCapabilities.

[pes10k]

Understood! As long as the requires that the interface only describe capabilities shipped by the browser, and it doesn't reveal anything about the underlying hardware (hardware supported codecs, hardware vs software rendering, etc.) then my concern is addressed.

Put differently, if the answer to "can I use this API to learn anything other than browser major version + OS + arch?" is "no", then this is good to close.

[henbos]

The capabilities does not say anything about if a supported codec is implemented in HW or SW, only if it is supported. So if we're only looking at the return value then the answer is "no". A codec in the returned list could be implemented in HW or SW - you have no idea from this API alone.

In practice, a caller of this API may have more information than what this API proves you on a particular device. Such as, what does this API return on other devices?

For example, if you know that a particular browser version has SW support for codecs X and Y (which you would know if you used this API on a device that doesn't have HW codecs) and on a different device getCapabilities() gives you "X, Y and Z" then Z would have to be an additional codec that is only available in HW and thus tell you something about the device.

At least that's my understanding. And if I look at what codecs are negotiated on my macOS device which has HW H.264 support with my Linux desktop which doesn't have HW H.264 support I see a difference: On my macOS device there are 6 different H264 profiles, on my Linux device there are 4 different H264 profiles.

[pes10k]

Apologies but I'm really having a difficult time understanding this; it seems like there are two different answers being given (or im even more lost than I though).

I understand @aboba to be saying that getCapabilities() only will describe what codec's the browser knows about (e.g. same version of Chrome will return the same values, irregardless of hardware capabilities).

I understand @henbos to be saying that getCapabilities() will return the codec's the browser knows about + whatever the underlying hardware supports?

If the former is correct, then i dont see a privacy risk. If the latter is correct, then this API seems to leak hardware capabilities and becomes a fingerprinting vector, and so needs to some form of access control / permissioning / etc.

[aboba]

@snyderp I don't believe there is a contradiction.

getCapabilities() (which was defined in WebRTC-PC) returns the codecs (and profiles) that the browser supports. This extension only extends the capabilities to advertise support for SVC scalability modes. To date, hardware support for SVC has been poor because most codec acceleration hardware is built for streaming applications (e.g. decoding) not realtime communications (which requires encoding). So it is very rare for hardware to specifically support any SVC modes (even temporal scalability). This means that hardware-info leakage specifically relating to SVC (the topic of this specification) is not a concern.

That said, Henrik is correct that getCapabilities() returns information on supported codecs and profiles. There is no explicit indication whether those codecs/profiles are supported in hardware or software and even if there is support for hardware acceleration, it is possible that the hardware resources may be fully utilized so that the application may only be able to achieve the performance associated with a software implementation. However in some cases the performance associated with higher profiles may only be obtainable via hardware acceleration, so that devices without the ability to achieve that performance may choose not to advertise them.

As an example, a device might advertise the capability to decode AV1 (via RTCRtpReceiver.getCapabilities()) but not to encode it (via RTCRtpSender.getCapabilities()) because the CPU could not achieve sufficient AV1 encoder performance (e.g. 720p at 30fps). A more capable device with hardware support for AV1 encoding might be able to advertise support for AV1 encoding via RTCRtpSender.getCapabilities().

[henbos]

So even though this API does not say anything about if something is implemented in HW or SW, because the API may return different things on different devices, the API can in fact be used for fingerprinting, though the information is indirect and probably pretty weak compared to other fingerprinting methods.

[aboba]

Since this discussion relates to the getCapabilities API defined in WebRTC-PC, not specifically to WebRTC-SVC, I'm moving the issue to the WebRTC-PC repo.

[pes10k]

@aboba please do not do that. if this feature will push additional fingerprintable information through an existing API, that is a distinct problem that needs to be sorted out here.

Is there any permission related to calling getCapabilities? Anything that keeps it out of the common path?

[aboba]

Since SVC is not supported in hardware this specification doesn't add to the fingerprinting surface.

[pes10k]

Again, I do not consider this issue resolved. The broader feature in WebRTC-PC needs to be addressed, but that does not change that webrtc-svc exposes seems to expose additional information that makes the problem worse.

I really cannot follow the above conversation, @aboba and you @henbos, you seem to be saying opposite things.

Specifically, how to reconcile:

@henbos

the API can in fact be used for fingerprinting, though the information is indirect and probably pretty weak compared to other fingerprinting methods

and @aboba

Since SVC is not supported in hardware this specification doesn't add to the fingerprinting surface.

[juberti]

Basically in theory this could be used for a faint fingerprinting signal but in practice there will be no signal since all current implementations will return the same results.

I do tend to agree that this issue is larger than SVC support, there are other places where codec capabilities could end up exposed to the application, e.g., a=imageattr in SDP.

[henbos]

For capabilities defined by webrtc-pc, the issue I described is not just theoretical, different devices do return different things; the H264 profile results I quoted earlier have to do with non-SVC codec capabilities and is not new with this spec.

For additional capabilities defined here in webrtc-svc, the same issue could exist in theory, but at least for now, SVC-related capabilities are always the same on all devices.

If we solve the webrtc-pc privacy issue we would most likely solve any webrtc-svc privacy issues for free, since these are just extensions of what already exist in webrtc-pc. If not, we would have to re-open this issue when we know what to do with webrtc-pc.

[ShivanKaul]

IMO this issue should be re-opened - from the discussion here and in https:/w3c/webrtc-pc/issues/2460, it does seem like there is a (weak, for now) fingerprinting signal that is added by this spec as it adds more (potential) info to a known fingerprinting vector. If we decide to put a permission gate on getCapabilities() there then this would be mitigated, but until then/if not, this is an issue that needs to be resolved with this spec. So it is very much an open question here.

I'm happy to open up a new issue but would rather not muddy up the discussion, so it would be best if we could re-open this one.

[pes10k]

I second what @ShivanKaul says above and am reopening the issue. Whether or not a permission gate is the right solution here, there needs to be some solution to the fingerprinting surface the feature exposes.

Point taken that current implementations may not expose FP bits presently, but the more useful the API is (e.g. the more difference there is between platforms / implementations / devices) the more privacy-harming this FP surface will be. So, its important to get a solution in place before the horse is out of the barn :)

[aboba]

Related issue: https:/w3c/webrtc-pc/issues/2539

Note: the above issue asserts that getCapabilities() cannot be used to obtain hardware capabilities.

[samuelweiler]

Our security reviewer (thank you Artur!) pointed this out, also, saying "The main issue, already called out in the Privacy and Security considerations section, is the potential for this extension to reveal system-dependent codecs, which may vary between different users/platforms;"