CAFE - Test Automation Framework Contribution and Onboarding

[kharikishan]

Please note that only FINOS members can propose new projects. If you're interested in membership, see https://www.finos.org/membership-benefits#become-a-member.

Onboarding Process

Completing an onboarding of a project into FINOS requires following these 5 main steps:

1. Describing the Contribution led by contributor
2. Approval led by FINOS TOC
3. Preparing for Onboarding led by contributor
4. Onboarding completed by FINOS Infra
5. Announcement led by FINOS Marketing

1. Describing The Contribution

Business Problem

Describe the business problem the contribution solves

The business problem that this contribution aims to solve is the need for a versatile and adaptable test automation framework. This framework should be open-source and modular in nature, allowing organizations to efficiently automate their testing processes across different layers of their software applications. Additionally, it should provide a comprehensive set of utilities and tools to facilitate data validation use cases within ETL and data pipeline worlds, making it a valuable asset for quality assurance and software testing teams.

Proposed Solution

Describe the type of contribution (project or working group) and how it solves that business problem

The proposed solution addresses the business problem by offering an open-source test automation framework developed in Python. This framework is designed to be feature-rich and versatile, providing support for various testing layers such as Web, Mobile, API, Database, and Thick Client (Desktop). A distinctive feature of the solution is its exhaustive support for various databases, warehouses, analytical cubes, data validations and data conversions. One of the key advantages of this solution is its extensive library of reusable methods, making it easier for quality assurance teams to create and execute tests efficiently.

This contribution takes the form of a project, delivering a modular framework that allows users to install specific components as needed, rather than requiring the installation of the entire framework. This modular approach enhances flexibility and scalability, ensuring that organizations can tailor their test automation setup to their specific requirements.

Moreover, the solution is accompanied by comprehensive documentation and a user guide. This documentation not only assists in the seamless onboarding of new users but also serves as a valuable resource for ongoing support and troubleshooting. In summary, this project offers a robust and adaptable solution to the test automation needs of organizations, helping them improve the efficiency and effectiveness of their software testing processes across different layers of their applications.

Tentative Roadmap

Describe the short and medium term goals and phases of the project. What does success look like for this project?

In the short term, our project aims to achieve modularity by breaking down various components of the framework into independent and reusable modules. This will provide users with the flexibility to select and use specific modules that align with their testing needs, ensuring a more tailored and efficient experience.

In the medium term, we aspire to build a comprehensive ecosystem around the framework. This ecosystem will include the development of features such as advanced reporting capabilities, seamless integration with CI/CD pipelines, and the incorporation of additional test automation utilities. These enhancements will not only streamline the testing process but also enhance the overall efficiency and effectiveness of test automation efforts.

Success for this project will be defined by several key indicators. First and foremost, it will be measured by the adoption rate within the testing and quality assurance community. A growing user base and positive feedback from users would signify the project's success. Additionally, the ability of the framework to consistently support various testing layers and the maturity of the ecosystem built around it will be crucial factors in determining success. Ultimately, success will mean providing a valuable, adaptable, and widely embraced solution for test automation that addresses the diverse needs of organizations across different industries.

Current State

Summarize the history and current state of the project

The project has evolved significantly over its history, growing from its initial stages to become a robust and versatile test automation framework. It has benefited from ongoing development and enhancements, primarily driven by the internal Quality Assurance teams at S&P Global.

At its current state, the framework boasts an extensive collection of utilities and features that have been meticulously crafted and refined over time. These utilities are a testament to the framework's maturity and effectiveness. They cover a wide range of functionalities, including but not limited to:

Test Automation Coverage: The framework supports test automation across multiple layers, including Web, Mobile, API, Database, and Thick Client (Desktop). This broad coverage ensures that various aspects of software testing can be efficiently addressed.

Utility Functions: Beyond test automation, the framework includes a plethora of utility functions. These functions encompass file parsing, file transfer, date-time handling, integration with AWS services, Elasticsearch integration, and a comprehensive set of assertions and exception handlers. These utilities make it a versatile toolkit for handling various software testing challenges.

Data Validation Functions: Connectors to various database platforms such as MSSQL, MySQL, Oracle, Postgres, Cassandra etc., and to various engines like Databricks, Snowflake and Apache tools. Other utilities would include parsers, converters, comparators and not limited to these alone.

Existing Materials

If materials already exist, provide a link to them that Foundation staff can access - if it's in a private GitHub.com repositories, you should invite the finos-admin user with R/O permissions to those repositories

• GitHub / GitLab Repository (delete as appropriate)
  • URL for the repository (if it exists)
  • Project Name CAFE
  • @finos-admin has been given read-only permissions if private
• Is Continuous Integration used? yes, Jenkins
• Was the project ever released? yes
  • If so, are releases public? no)
  • And what's the latest released version?
• Existing Project Documentation ( URL / microsite / PDF etc detail here)
• Does the name have a registered trademark? no
• Is there a logo? no
• High-Level Presentation prepared for Technical Oversight Committee yes
• Are meetings currently held for the project? yes
• Are meeting minutes, agenda and attendance tracked? yes

Development Team

• Name of the org: S&P Global
• Team Size: 5
• Developers: To be furnished later

Maintainers

Who will be the project maintainer(s)? Provide full name, affiliation, work email address, and GitHub / GitLab username.

Name	Affiliation	Work Email Address	Github / GitLab username
Hari Kishan Kurapati	S&P Global	[email protected]	@kharikishan
Sameer Shaik	S&P Global	[email protected]	@SameerShaik248

Confirmed contributors

If applicable, list all of the individuals that have expressed interest in and/or are committed to contributing to this project, including full name, affiliation, work email address, and GitHub.com username

Name	Affiliation	Work Email Address	Github / GitLab username
Hari Kishan Kurapati	S&P Global	[email protected]	@kharikishan
Sameer Shaik	S&P Global	[email protected]	@SameerShaik248

• Additional contributors to be furnished later

Target Contributors

Describe the contributor profile (background, position, organization) you would like to get contributions from.

Python Developer
Skilled in developing robust and scalable applications using Python. Experienced in writing clean, efficient code and familiar with test automation frameworks. Passionate about problem-solving and continually improving code quality.

CI/CD Engineer
Expert in Continuous Integration and Continuous Deployment practices. Proficient in setting up and managing CI/CD pipelines using tools like Jenkins, and GitHub Actions. Dedicated to automating workflows to ensure seamless and reliable software delivery.

Project Communication Channel(s)

• Contributor to ask maintainers which communications channels they'd like to use:
• Asynchronous
  • GitHub Issues (public)
  • GitHub Discussions (public)
  • GitHub Team Discussions (consisting of the above described contributors)
    • Public
    • Private
  • Mailing-list (groups.io)
  • FINOS Slack Channel (consisting of the above described contributors)
    • General (public) (supply channel name)
    • Leadership (private) (supply channel name)
• Synchronous
  • Recurring meetings

Understanding FINOS Onboarding Requirements

As a project onboarding into FINOS, you will need to familiarize yourself and your contributor team with the following materials:

• FINOS overview (if necessary)
• FINOS Maintainers cheatsheet
• FINOS Project/Standards Governance
• FINOS Project Lifecycle

Record The Contribution (FINOS Infra)

• (optional) Identify and Assign FINOS Strategic Advisor
• Submit contribution to LF by opening a ticket via https://jira.linuxfoundation.org/browse/SS and marking contribution as "Exploratory"; attach a summary of the Business Problem and Proposed Solution (above) of the project.

2. Approval

The FINOS Technical Oversight Committee (TOC) is responsible for approving FINOS project contributions; feel free to check their contribution principles.

If needed, the TOC will request a follow up either via GitHub Issue comments or by inviting project leads to one of their recurrent meetings.

Tasks (for FINOS Infra/TOC)

• Prioritise this issue on the TOC Backlog
• Assign this issue to @ColinEberhardt
• Add the ready-for-tsc label
• TOC to invite contributors to present their project
• FINOS TOC approves/rejects the contribution
• Ask @jgavronsky to mark contribution as "Engaged" within LF systems
• (optional) If additional socialization is required, the Executive Director may bring projects to the FINOS Governing Board
• Update the contribution status to "Engaged" by sending another email to LF Legal Representative with the name of the project and its new status.

TOC Findings / Report

TOC to enter findings summary here.

3. Preparing For Onboarding

Before the FINOS team can onboard your project, there are a few housekeeping that need to be taken care of. These must be completed by the contributor, with help if required from the FINOS Infra.

Kick-off meeting

• Set up kick-off meeting with project leads
• Run kick-off meeting
  • Walk through the checklist in part 1, ensure all the questions are answered and remove items that don't apply
  • Write and send contribution proposal announcement (optional - see below)

Proposal (Lead Maintainer)

• Lead maintainer to send out announcement to [email protected] using this template:

  Dear FINOS Community, 
  
  We would like to propose a new FINOS project. Please review the proposal details at (_TODO: add link to the GitHub issue proposal_).
  
  If you're interested in participating, please :+1: the GitHub issue proposal and drop a comment with your name, org and email
  
  Thanks a lot,
  

Logo / Trademarks

• Sign the project contribution agreement to allow FINOS to act on behalf of the contributor for accounts related to the project (e.g., GitHub, domain names, social media) and to optionally manage trademark assignment
• Request logo design from [email protected] (if needed)
• The codebase doesn’t include any patent or copyright that conflicts with FINOS Governance and bylaws. (Infra team to validate with FINOS Legal team if anything important is raised)

FINOS Project Blueprint

• finos-admin is Admin of the GitHub repository to transfer
• Apply project blueprint contents - see our docs on collaboration infrastructure
  • README.md file contains the sections in the README Template
  • Project FINOS badge in README (incubating)
  • OpenSSF Passing Badge
  • License in README
  • Contributing in README
  • CONTRIBUTING.md
  • LICENSE (replace {} placeholders)
  • Ensure that the proper project governance is in the CONTRIBUTING.md file
• All incubating criteria are checked and documented below

Add documentation here

4. FINOS Onboarding

This is performed by FINOS Infra once the three previous stages are complete, with support from the contributor and the FINOS Infra team.

Maintainers, Contributors and CLAs

• Identify other existing contributors (assuming there's a contribution history (eg Git history)
• Check if maintainers and other contributors are all covered by FINOS CLA
• Engage with FINOS Legal team to figure out what’s needed to cover all maintainers and contributors with FINOS CLA
• Reach out to contributors and employers to coordinate CLA signatures
• Invite GitHub usernames to GitHub FINOS Org
• Create <project-name>-maintainers GitHub team and invite users

Validation (only if code is contributed)

• Backup (even with screenshot) GitHub permissions of the repository to transfer
• Check GitHub repository transfer requirements:
  • finos-admin has Admin to all repositories to transfer
  • finos-admin ia allowed to transfer repositories out of the org
  • if the repository is owned by a user (and not an org), the user must be able to transfer the repository to finos-admin
• The codebase doesn’t have HIGH or CRITICAL CVEs across direct and transitive libraries
• The codebase doesn’t have any unfriendly licenses across direct and transitive libraries
• (optional - if a build system is provided) The build process runs successfully
• The codebase doesn’t include any patent or copyright that conflicts with FINOS Governance and bylaws (to be validated with FINOS Legal team)
• All incubating criteria are met (review documentation provided above)

Code transfer

• Transfer all code assets as GitHub repositories under github.com/finos
• Rename main branch to main (instead of master)
• Configure finos-admins (Maintain role) and finos-staff (Triage role) team permissions

Project Communication Channel(s)

• Create the identified communication channels during infra set up
• Create mailing-list on lists.finos.org (if requested)
  • Enable Hubspot Sync for all project mailing lists created
  • Aggregate mailing lists to [email protected]
  • Update marketing lists (see Email List and Email filter fields), particularly Hubspot all community list
• Create Slack channels if requested.
• Link communication channels linked front-and-center in the project README.md

Repository setup

• Enable EasyCLA
• Add project to metadata
• Add identities, orgs and affiliations to metadata (deprecated by EasyCLA)
• Add logo to FINOS landscape
  • Create staging branch on finos/finos-landscape
  • Merge finos/metadata changes on master (will udpdate landscape.yml in finos/finos-landscape)
  • Create PR from staging branch on finos/finos-landscape
  • Review Netlify preview
  • Merge PR
• Add project maintainers emails to [email protected] list
• Add project maintainers GitHub usernames to the project-maintainers Team
• Ask @jgavronsky to mark contribution as "Active" within LF systems
• (best effort) Update release coordinates and code namespace to include finos
• Check that CVE (and preferably static code analysis, if applicable) scanning is in place
• Enable automated dependency update, using Renovate
• Enable branch protection (Require a pull request before merging)
• (optional) Enable meeting attendance tracking
• (optional) Onboard into legend.finos.org/studio

5. Announcement

(Lead: Project Lead and FINOS Infra team)

• Update the contribution status to "Active" by sending another email to LF Legal Representative with the name of the project and its new status.
• Lead maintainer works with FINOS marketing to send out announcement to [email protected] , checkout announcement template at the Contribution page
• Notify FINOS marketing (@grizzwolf + finos-marketing internal Slack channel)

[TheJuanAndOnly99]

@kharikishan thank you for the contribution proposal! As described in point number 2 of this issue the FINOS TOC will be taking a look at your proposal. Please allow us some time to get back to you.

[TheJuanAndOnly99]

Update: FINOS TOC has asked CAFE team to share some code in order to be able to properly asses the contribution. CAFE team is working on sharing a repo.