CameraCaptureUI API Implementation

[Saharsh979]

Validation Note: A microsoft employee must use /azp run to validate using the pipelines below.

API Spec: #4721

Fixes : #1034

API Design Overview

The CameraCaptureUI API allows desktop applications to effortlessly integrate native camera capture features across various Windows platforms. This proposal aims to address the limitations of the existing OS CameraCaptureUI, primarily designed for UWP, which poses challenges for desktop environments. Key issues include its reliance on CoreWindow and lack of support for IInitializeWithWindow, necessitating cumbersome workarounds.

This new API streamlines the integration of camera capture capabilities while ensuring feature parity across all Windows platforms supported by WinAppSDK, thus simplifying developers' workflows.

The CameraCaptureUI class provides a comprehensive UI for capturing audio, video, and photos. It allows customization of capture settings, including trimming video and adjusting camera settings.

Constructor:

• CameraCaptureUI(windowId) - Initializes a new CameraCaptureUI object with the specified window ID.

Properties

• PhotoSettings: Manages photo capture settings.

• VideoSettings: Manages video capture settings.

Methods

• CaptureFileAsync: Launches the capture UI and returns the resulting file.

This implementation ensures that the CameraCaptureUI API aligns closely with existing UWP functionality, only modifying the constructor to accept a WindowId.

Example C++ Code

This example shows how to use the CameraCaptureUI Class to take a picture.

    // Get the WindowId for the window
    Microsoft::UI::WindowId windowId = this->AppWindow().Id();
 
    // Initialize CameraCaptureUI with a window handle
    winrt::Microsoft::Windows::Media::Capture::CameraCaptureUI cameraUI(windowId);
 
    // Configure Photo Settings
    cameraUI.PhotoSettings().Format(CameraCaptureUIPhotoFormat::Jpeg);
    cameraUI.PhotoSettings().AllowCropping(false);
 
    // Capture a photo asynchronously
    auto photo = co_await cameraUI.CaptureFileAsync(CameraCaptureUIMode::Photo);
 
    // Assert the result
    if (photo != nullptr)
    {
        // Log success if photo capture was successful
        Log::Comment(L"Photo capture was successful.");
    }
    else
    {
        // Log error if photo capture failed or was canceled
        Log::Error(L"Photo capture failed or was canceled.");
    }

Testing

• Successful local build
• Produced a experimental NuGet package leveraging the aggregator build pipeline, and tested it using a sample app by calling the CaptureFileAsync API, which functioned as intended.
• Executed unit tests with the mock framework package, and everything performed as expected.

Note: The Interactive Experiences package has been incorporated as a dependency in the Foundation repo to utilize Microsoft.UI API features.

Pending Action Items:

• Ensure that Maestro recognizes the dependency updates being made to the Foundation branch (the Interactive Experiences package has been integrated into the Foundation repo)
• Implement support for C# projections.
• Update unit tests accordingly.
• Publishing Public/Private symbols

[Saharsh979]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[dmachaj]

Its been long enough that I forgot some details, but is this factory supposed to be non-agile so it has thread affinity? Or is it fine to be agile and the API is multi-thread safe it just needs to be called from a UI thread?

[Saharsh979]

Yes, the factory can be designed to be agile, meaning it can operate across multiple threads, as long as it adheres to the requirement of being called from a UI thread.

Can you elaborate more on "thread affinity" in this context and your concerns regarding it.

[dmachaj]

Do these validate methods need STA thread type checking?

[Saharsh979]

The CameraCaptureUI API needs to be called from an STA thread. If it’s invoked from a non-STA thread, it will throw an exception: RPC_E_WRONG_THREAD. This occurs because accessing some UI elements or WinRT components which interact with UI is restricted to the thread that created it, which is typically the UI/STA thread.

I believe we don’t need an additional layer of checking or handling for this. The existing behavior sufficiently enforces the STA requirement.

[Hemantxk]

Why 2 and not just CameraCaptureUI.idl

[Saharsh979]

We can stick with "CameraCaptureUI.idl" for simplicity. However, if we expect more modifications or want to distinguish it from the original, adding this version number might help with clarity.

[Hemantxk]

You can create an Activity instead to mark the start and end of the capture operation. Consider adding mode of capture in these events.

[Hemantxk]

Should we check results.Status() before this?

[Hemantxk]

There should be a lock to prevent/hold parallel calls to CaptureFileAsync

[AjitSurana]

If two (or more) consecutive CaptureFileAsync async calls run well, no change required.

Interested to know if this is a real use case for the developers since existing UWP API had issues (with lock) itself but not aware if there is bug raised for the same. Great to know if such feedback comes from the preview program. Can be taken up once the preview feedback is received.

[Hemantxk]

The problem with current implementation without lock is that parallel calls to CaptureFileAsync API accidently overwrites the values of m_photoTokenFile and m_videoTokenFile member variables depending upon the mode of capture. That seems fundamentally wrong as the first call is not finished and tokens are already overwritten, and UWP API prevent that by lock (of-course that lock is not working as per expectation, needs to be fixed separately).
Either we can address this by modifying the API implementation and make it safe for parallel calls or write a lock and restrict it.

[AjitSurana]

If values are overridden, below piece of code will be executed and the result will be as expected.

file = co_await SharedStorageAccessManager::RedeemTokenForFileAsync(tokenValue);

Your concern is valid. However, as per the conversation with Saharsh, the API gives expected result so probably a follow up task or waiting to hear from preview feedback is ok.

If the below statement works well, why we need to have m_photoTokenFile and m_videoTokenFile member variables. Can we remove these two variable?

file = co_await SharedStorageAccessManager::RedeemTokenForFileAsync(tokenValue);

[Saharsh979]

I understand that the Camera app has certain limitations, and the team recognized this when deciding to implement a lock to ensure only one capture operation occurs at a time on the original API. Even if multiple capture operations are initiated, only the focused camera will render, while others will show a message stating "Camera is already in use by another app."
Also, for real world use cases, if people do actually want multiple camera capture instances up, they should go through a feature request process.
For now, I am going ahead with the Locking mechanism and restricting this behavior just as it's done for Original API.

[AjitSurana]

Another idea was discussed is can m_photoTokenFile and m_videoTokenFile be converted to local variable to avoid lock?

[Saharsh979]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Saharsh979]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[akanpatel2206]

Better to add the contract "CameraCaptureUIContract" in specs/WinRT/WinRTAPIContracts.md as well.

[AjitSurana]

Any reasoning why the "CCapture" name is used here? I assume CCapture stands for CameraCapture. Can we use the CameraCapture sort of name instead? the reasoning behind that is it is better human readable name.

Having clear file name would help the program to determine the source of the file as well.

[Saharsh979]

That makes sense. For better readability, I'll go ahead and use the full name instead.

[AjitSurana]

If you can get the file name from the tokenValue itself, why you need to manage m_videoTokenFile and m_photoTokenFile? It would simplify your implementation.

[AjitSurana]

In the pervious catch statement, throw ex is written. But in the current exception, hresult_error(E_FAIL) is written, curious to know the difference?

[AjitSurana]

[Minor] Shouldn't window name be windowId? or Id? for improving code readability.

[AjitSurana]

Same condition twice

[AjitSurana]

We should capture this information in the spec doc.

Don't specify both size and aspect ratio in the PhotoSettings, it will cause invalid argument exception

[AjitSurana]

This as well we should capture in the spec doc.

[AjitSurana]

Please add containment in the current PR or in a follow up PR.