Software Project Contribution and Onboarding - Vuu

[chrisjstevo]

Business Problem

User interfaces for financial applications often have differing requirements to generic UI Frameworks. In general these can be summarised as:

• The style of interface is often largely grid based
• it can often contain large amounts of information, tens of millions of rows is common
• the data often is dynamic, incorporating rapid updates and ticking data sources like market data
• the data is often sourced from multiple different technologies (slow moving data sources, like databases and fast moving data sources like order management systems and market data systems) and the user wants this data to be joined at runtime

Proposed Solution

Vuu is a framework for building front end applications for trading and risk use-cases. It is split into two components, the first is a Scala based server component which acts as the server side cache of data, this provides a table model for representing and joining live tables of data from various sources, it also provides a framework for sorting, filtering and treeing on the server based on client requests. It also provides viewports onto the trees or tables for multiple users. The server also provides an ability to tick data quickly through the system for rapid updates to individual rows.

The second component is a React based UI grid and layout framework that manages the communications with the server, that renders the visible viewports for the user (whether they be tree or flat table based) and that gives the impression to the user that all of the data that a user is viewing is seamlessly available on the client.

The glue that connects these pieces is a web-worker/websocket layer that handles realtime updates from the server based on updates on the underlying datasource or based on user actions on the front end, such as scrolling or sorting.

Tentative Roadmap

The initial phase would be working towards a v1.0 release of the framework, where we can support all of the necessary use cases for building trading front ends. Its felt we are between 80% and 90% of the way there with the existing code base.

A phase 2 release would incorporate additional functionality such as real time charting, additional user widgets for specialised forms of visual filtering and the ability to support big data streaming uses cases (10bn+ rows)

Current State

The project began in 2014, it was worked on as a PoC for several years before gaining more momentum towards a functional beta release in 2020/2021. The project has now beed adopted by UBS as part of the strategic front office UI toolkit.

Existing Materials

The software is already opened source on Github under an Apache 2.0 licence.

Code: https:/venuu-io/vuu/
Book: https://venuu-io.github.io/vuu/

Development Team

Maintainers

Chris Stevenson, UBS, [email protected], Founding Developer, Admin, chrisjstevo
Steve Heron, UBS, [email protected], Founding Developer, Admin,
Dimitar Dimitrov, UBS, [email protected], Developer
Michael Stonestreet, UBS, [email protected], Developer
Michael Banham, UBS, [email protected], Developer, Admin
Allan Jakobsen, UBS, [email protected], Product Owner
Marina Bambang, UBS, [email protected], Product Owner

Confirmed contributors

(as above)

Target Contributors

• Neil Slinger, JP Morgan + Team
• Others TBC

Infrastructure needs

Describe the FINOS infrastructure you will need for this project, in addition to a GitHub repository. The FINOS team will connect with you before setting up any of this infrastructure

• Mailing list

What's next?

Upon submission of this project proposal, the FINOS team will get in touch with you to discuss next steps.

---

Contribution process (v. 1.1, last updated on December 20, 2021)

Below is the list of tasks that FINOS Team and the contribution author go through in order to complete the FINOS contribution process.
Please do not edit these contents at contribution time!

Kick-off meeting

• Set up kick-off meeting with project leads
• Run kick-off meeting
  • FINOS overview (if necessary)
  • FINOS Maintainers cheatsheet
  • FINOS Project/Standards Governance
  • FINOS Project Lifecycle
  • Walk through the checklist, answer questsion and remove items that don't apply
  • Write and send contribution proposal announcement (optional - see below)

FINOS Contrib POC

• Identify and Assign FINOS Project Coordinator - @maoo
• Identify and Assign FINOS Strategic Advisor - @jgavronsky

Proposal (Lead Maintainer)

• Lead maintainer to send out announcement to [email protected] using this template:

  Dear FINOS Community, 
  
  We would like to propose a new FINOS project. Please review the proposal details at (_TODO: add link to the GitHub issue proposal_).
  
  If you're interested in participating, please :+1: the GitHub issue proposal and drop a comment with your name, org and email
  
  Thanks a lot,
  

Identify project meta (Lead: FINOS Contrib POC, Support: FINOS Marketing)

• Project main coordinates
  • Project Name - Vuu
  • Project Slug - vuu
  • Does the name have a registered trademark? - no
  • Request logo design (if needed) - UBS working on the logo already
• Category and sub-category (for FINOS Landscape) - Applications > UI Frameworks
• Is there existing code? If so, is it public? If not, can you grant read access to user github.com/finos-admin ? - https:/venuu-io/vuu
• Was the project ever released? If so, are releases public? And what's the latest released version? - see https:/venuu-io/vuu/tags
• Team composition: lead maintainer and other maintainers (see above)
• Are meetings currently held for the project? - no
• Are meeting minutes, agenda and attendance tracked? - no
• Is Continuous Integration used? If so, which system is used? - see https:/venuu-io/vuu/actions
• Is there an existing Documentation website? If not, would you like to have one? - see https://venuu-io.github.io/vuu/

Maintainers, contributors and CLAs (Lead: FINOS Contrib POC, Support: FINOS infra)

• For each maintainer identified in the previous step, collect: the following info:
  • Fullname
  • GitHub username (missing)
  • Corporate email address
• Identify other existing contributors (assuming there's a contribution history (eg Git history) - see https:/venuu-io/vuu/graphs/contributors (identify ararus and heswell)
• Check if maintainers and other contributors are all covered by FINOS CLA - not covered
• Reach out to contributors and employers to coordinate CLA signatures - ask Will Rothwell to allow UBS employees listed above from EasyCLA dashboard

Project Communication Channel(s)

• Ask maintainers which communications channels they'd like to use
• Asynchronous
  • GitHub Issues (public)
  • Mailing-list (groups.io) - not for now, maybe in the future
• Synchronous
  • FINOS Slack Channel (general public Slack / leadership private Slack) - not for now, maybe in the future
• Link communication channels linked front and center in the project README.md

Code validation (only if code is contributed) (Lead: FINOS Infra)

• The codebase doesn’t have HIGH or CRITICAL CVEs across direct and transitive libraries - see CVE and static security scanning vuu#242
• The codebase doesn’t have any unfriendly licenses across direct and transitive libraries - see CVE and static security scanning vuu#242
• The build process runs successfully
• finos-admin is Admin of the GitHub repository to transfer
• The codebase doesn’t include any patent or copyright that conflicts with FINOS Governance and bylaws (to be validated with FINOS Legal team)
• Apply project blueprint contents - see ODP docs - Apply FINOS blueprint vuu#256
  • Ensure that the proper project governance is in the CONTRIBUTING.md file
• All incubating criteria are checked and documented below

Approval (Lead: FINOS Infra)

• Prioritise this issue on the TSC Backlog
• FINOS TSC approves/rejects the contribution
• Update the contribution status to "Engaged" by sending another email to LF Legal Representative with the name of the project and its new status.

Code transfer (Lead: FINOS Infra)

• Backup (even with screenshot) GitHub permissions of the repository to transfer
• Check GitHub repository transfer requirements:
  • finos-admin has Admin to all repositories to transfer
  • finos-admin ia allowed to transfer repositories out of the org
  • if the repository is owned by a user (and not an org), the user must be able to transfer the repository to finos-admin
• Review FINOS project blueprint contents
  • Project title/description in README
  • Project badge in README
  • License in README
  • Contributing in README
  • CONTRIBUTING.md
  • LICENSE (look for {} placeholders)
• Check protection settings and disable after transfer if necessary
• Transfer all code assets as GitHub repositories under github.com/finos
• Invite GitHub usernames to GitHub FINOS Org
• Create <project-name>-maintainers GitHub team and invite users
• Configure finos-admins (Maintain role) and finos-staff (Triage role) team permissions

Infra setup (Lead: FINOS Infra)

• Enable EasyCLA
• Enable branch protection
• Enable Renovate Bot
• Add project to metadata
• Add project maintainers emails to [email protected] list
• Add project maintainers GitHub usernames to the project-maintainers Team- [x] Enable security vulnerabilities scanning
• Update the contribution status to "Active" by sending another email to LF Legal Representative with the name of the project and its new status. Issue created https://jira.linuxfoundation.org/browse/SS-1030
• Setup NPM_TOKEN as GitHub secret

Announcement (Lead: FINOS Contrib POC)

• Lead maintainer works with FINOS marketing to send out announcement to [email protected] , checkout announcement template at the Contribution page
• Notify FINOS Contrib POC and FINOS marketing (@grizzwolf + finos-marketing internal Slack channel)

[mindthegab]

This is amazing @chrisjstevo! Thanks for getting this going! Paging @neilslinger for visibility here!

@jgavronsky @maoo can you update the checklist to 1. reflect the TSC approval step and 2. I think the contribution looks good enough to be fast tracked for TSC approval (given it's OSS and has already identified contributors).

As a general suggestion, I think that now that we have the TSC, TSC approval should be much front loaded and code validations can happen in parallel or after.

Thanks!

[mindthegab]

@ColinEberhardt This contribution is ready for TSC review. I've put it in the backlog. If there's room, it might make sense to add to the agenda for Wednesday.

[ColinEberhardt]

HI @mindthegab - it is on our agenda for tomorrow finos/technical-oversight-committee#37, although this will just be a high-level introduction. If the FINOS team have specific priorities / deadlines regarding Vuu / ISDA / Desktop Agent Bridge, it would be good to understand this.

[chrisjstevo]

@ColinEberhardt @mindthegab we're very keen to get something incubating from a UBS perspective. This is our first major contribution and there is quite a lot of workflow we need to do internally when the repo and scaffolding have been created to get productive.

@ColinEberhardt if you'd like me to join the meeting tomorrow to discuss it am happy to.

[mindthegab]

@ColinEberhardt No specific deadline, but I did try to reflect priorities / readiness for TSC review as agreed by:

1. Adding it at the top of the "Prioritized Backlog" column of the TSC Kanban (currently not public yet)
2. Labeling this issue as "ready-for-review"
3. Assigning it to you as proxy for the TSC

Feel free to liaise directly with @chrisjstevo and assign back to me a comment / outcome once the TSC has had a chance to review.

[ColinEberhardt]

Hi @chrisjstevo - we didn't really have time to slot you in for the meeting yesterday, lots of other house-keeping!

However, the TSC are very keen to hear more about this project. Are you happy for us to arrange a demo to the group? If it helps, please contact me via email to arrange.

[ColinEberhardt]

The following is a summary of the TSC’s assessment of the Vuu contribution, evaluating this project with respect to our contribution principles.

General background information:

• The Vuu project started in 2015
• It comprises a server component (written in Scala), which aggregates data, and applied various transformations (sort, filter, group) and a client-side component (written in React)
• The main USP (unique selling point) of this contribution is the ability to render large volumes of data by applying a ‘windowing’ concept, i.e. the data sent to the browser depends on the current viewport.

The TSC advises that the FINOS Team and Governing Board accept this contribution.

Contributions must be aligned with the purpose of FINOS

The purpose of FINOS is to drive collaboration and innovation within Financial Services through open source software. The problem which this project tackles (rendering and navigating very large datasets) is quite common within FS organisations, furthermore, there is little competitive advantage to be had by building a proprietary solution in isolation. This is an area that is ideal for multi-party collaboration.

A proposed standard must not compete with other FINOS standards

This is a software project, not a standard.

An open source project that is a component of a commercial offering must provide tangible value in its own right

This is entirely open source, without any commercial components.

Contributions should generate interest within the member organisations

The display and navigation of large volumes of data is a ‘classic’ problem within financial services. Most of the TSC members have built or use similar technology. The alignment with FINOS is clear and obvious, a number of TSC members expressed a direct interest in this project.

A proposed contribution that competes with an existing FINOS project should consider potential routes to a merger

There are some clear parallels between Vuu and Perspective, both provide the ability to view and analyse streaming datasets. However, the two projects do have a different focus. Perspective is more aligned with the needs of analysts, allowing the creation of dashboards that compete with the likes of Tableau. Whereas Vuu is targeted more at the needs of developers, and would likely be incorporated into much larger platforms (e.g. trading platforms, risk systems). There may be some opportunities to exchange ideas and approach, however, it is unlikely that the two would (or should) merge.

Contributions should be long-term projects

This project has been under development for a number of years and UBS is keen to start using this technology within their own applications.We are comfortable with this informal commitment.

[TheJuanAndOnly99]

Project onboarding completed, please find announcement at https://groups.google.com/a/finos.org/g/announce/c/i7rzIhlj0e0/m/EygO4fgVBAAJ. Congratulations to @chrisjstevo and team!