Open XDMoD 8.1.1

DO NOT USE, THERE IS A KNOWN ISSUE PLEASE USE 8.1.2

• Bug Fixes
  • General
    • Allow upgrade to finish if cloud realm schema has not been created (#882)
      DO NOT USE, THERE IS A KNOWN ISSUE PLEASE USE 8.1.2

Open XDMoD 8.1.0

DO NOT USE, THERE IS A KNOWN ISSUE PLEASE USE 8.1.2

• Documentation
  • ETL
    • Improve storage aggregation (PR #882)
    • Processor Buckets documentation update (PR #881)
  • General
    • Update broken url and change wording of supported software (PR #876)
    • Add Federated XDMoD to About page (PR #873)
    • Update with some of the options that are avalbile (PR #804)
    • Improve storage documentation (PR #771)
  • Cloud
    • Updated Cloud Documentation for 8.1 Release (PR #875)
• New Features
  • Cloud
    • Add cloud user and system account group by (PR #797)
    • Add support for cloud data to xdmod-shredder and xdmod-ingestor (PR #739)
• Enhancements
  • General
    • Add support for configurable email subject prefix (PR #872)
    • Add node_modules to the RPM. (PR #835)
    • Update config read order (PR #818)
    • Add initial summary charts for the cloud realm (PR #803)
    • Support simplesaml's internal session naming (PR #757)
    • Support asynchronous loading of Usage tab thumbnail charts (PR #750)
    • Update Job Viewer API to support multiple realms (PR #733)
    • Support non-numeric values for Usage chart filter parameters (PR #716)
    • Make "Show raw data" for multiple realms configurable (PR #706)
    • Update Sign On Panel to collapse local login if SSO is enabled (PR #701)
  • Cloud
    • Do not truncate aggregate tables on each ingest (PR #841)
    • Truncate staging tables after ingestion (PR #778)
    • Update events that are used to determine VM session starts and stops (PR #732)
  • ETL
    • Improve resiliency of ETLv2 manage tables (PR #807)
    • Add storage shredder/ingestor support (PR #786)
    • Support ETL '$include' directive (PR #785)
    • Update Configuration class to support merging objects in local config files (PR #782)
    • DirectoryScanner support for last modified time based on filename and/or directory (PR #780)
    • Move the job performance postprocessing SQL to the aggregation pipeline (PR #770)
    • Implement dynamic fact tables via ETLv2 for job performance ETL (PR #742)
    • Add exception code to logAndThrowException (PR #719)
    • Skip only records that fail verificaiton instead of rest of file (PR #714)
• Bug Fixes
  • Cloud
    • Fix roles file comparison to use object and not string (PR #878)
    • Update session_records timestamps to non-nullable for MySQL 5.7 support (PR #877)
    • Add cloud raw tables to cloud manage tables action (PR #874)
    • Change staging table to use 1 as unknown id instead of -1 (PR #866)
    • Update GroupBys In Cloud Aggregate Table (PR #863)
    • Change cloud person username fields to be not null (PR #860)
    • Change where staging action gets user id from for cloud data (PR #845)
    • Remove event_id from the event table primary key (PR #844)
    • Prevent null usernames being added when ingesting cloud data (PR #838)
    • Remove duplicate join statement that was causing 1066 Not unique table/alias error (PR #837)
    • Fix for generic cloud datetime, and openstack instance type datetime (PR #820)
    • Update cloud realm to not throw away event precision (PR #811)
    • Guarantees a deterministic order for events received by the event reconstructor (PR #805)
  • General
    • Add proper namespace in VerifyDatabase (PR #819)
    • Fix path to Exception in TimeAggregationUnit::factory (PR #810)
    • Explictly check for stdClass in VariableStore initializer (PR #802)
    • Make lastLogin time be floored to the second instead of microtime (PR #755)
    • Make logrotate.d file not override global settings (PR #749)
    • Fix security vulnerabities in job performance (PR #738)
    • Allow search panel scroll bars on small displays (PR #702)
  • ETL
    • Update primary key on resourcefact table to improve cloud ingestion (PR #795)
    • Throw Exception if lockfile could not be obtained (PR #793)
    • Improve debugging messages when executing SQL (PR #783)
    • Don't automatically rewind DirectoryScanner file handle (PR #768)
    • Explicitly cast potentiall nulls to array for array_merge() (PR #756)
    • Improve verification of resource codes (PR #720)
    • Fix uncaught ETL exception in PdoIngestor (PR #718)
  • Metric Explorer
    • Fix refesh button for Metric Explorer (PR #740)
• QA / Testing
  • General
    • Speedup integration tests (PR #850)
    • Clean up linker.php (PR #847)
    • Add environment variable to force regression test harness to generate expected results (PR #848)
    • Enforce javascript unit tests (PR #766)
    • Robustness improvements for integration tests (PR #726)
  • Cloud
    • Update cloud reference data (PR #827)
      DO NOT USE, THERE IS A KNOWN ISSUE PLEASE USE 8.1.2

Open XDMoD 8.1.0 Release Candidate 1

v8.1.0-rc.1

Improve storage aggregation (#882)

Open XDMoD 8.0.0

• Features
  • General
    • Added a beta version of the Cloud realm to provide metrics relevant to cloud computing resources.
    • Added a beta version of the Storage realm to provide metrics relevant to storage systems installed at a center.
    • Federated XDMoD has been released for production. Federated XDMoD allows individual, locally managed, XDMoD instances to report all or a subset of their accounting data to a central Hub which provides a global view of the federation.
    • All XDMoD user profiles are now associated with an organization. Previously, this was only required for Campus Champions.
    • Added support for automatically detecting / assigning a new SSO User's organization.
    • Added support for automatically detecting if a user's organization has changed and updating their accounts accordingly. This may include, but is not limited to, the removal of elevated privileges.
    • Hardened the login and password reset process as a result of a security audit by University of Cambridge.
    • Improved support for resource manager job arrays.
    • Many improvements to the documentation.
  • ETL
    • Reorganized several ETL pipelines.
    • Improved data sanitization for tighter checks present in MySQL 5.7.
    • Refactored Jobs realm ingestion to utilize ETLv2.
    • Standardize action names to follow the format module.pipeline.action. For example, xdmod.acls.manage-tables.
    • Added character set and coalition to table definitions.
    • Added support for foreign key constraints.
    • Added support for the definition of ETL variables on the command line using -d variable=value.
    • Add ingestion of node hostname data from SGE logs.
    • Various ETL performance improvements.
• Bug Fixes
  • User Interface
    • Deep linking when logged in using SSO has been restored.
    • Update the logrotate configuration to use the su and create options.
  • ETL
    • Add primary keys to select ETL source queries.
    • When modifying an existing table, preserve the order of the columns in the definition file.
    • Ensure that file handles are flushed before inserting the final chunk of data.
  • Misc
    • Fixed several exceptions that were outside of a namespace.
    • Fixed an issue where ACLs were not properly created on upgrade.
    • Several minor bugfixes

Open XDMoD 7.5.1

• Bug Fixes
  • Properly implement data access for non-feature ACLs (e.g., ACLs that provide access to data
    but not a feature such as a tab) that are not part of the hierarchy. For example, the Value
    Analytics ACL.

This RPM was missing the user manual, it has been attached a separate file that you can download and extract to /usr/share/xdmod/html/

Open XDMoD 7.5.0

• Features
  • General
    • Added support for Globus as a federated authentication provider
    • Improvements to the user login dialog and matching of users to institutions
    • Added the ability to use fine-grained ACLs for controlling access to features such as tabs and realms as well as data
    • Code pertaining to creating/sending emails has been moved to a central location.
    • Improved online documentation for installations and upgrades
    • Improve performance of Utilization statistic
    • Do not embed JavaScript in chart objects returned by the back end, instead
      include this code directly in the user interface code
    • General improvements to the performance of the REST stack
    • Updated Google Captcha to v2 (v1 is now discontinued)
    • Added numerous component and unit tests
    • Removed unused code paths
    • Update greenlion/PHP-SQL-Parser to newer release that does not autoload itself
  • Internal Admin Dashboard
    • Updated user management functionality in the administrative dashboard to support fine-grained ACLs for individual users
    • Many stability improvements to the administrator dashboard, especially for managing users
  • Storage Realm
    • Added alpha version of the Storage realm to track resource storage utilization
  • User Interface
    • Legend item edit box is now displayed next to the legend item that is being edited and is larger to accommodate longer strings
    • Added a "Select All" button to the filter dialog
    • Added PDF export capability, which greatly improves the quality of images included in LATEX documents
  • ETL
    • When ingesting Slurm data, return duplicate jobs such as those that were resubmitted due to node fail conditions
    • Improve performance of filter list generation
    • Added an ETL pipeline for ingesting log files generated by Eucalyptus clouds
    • Added support for PBS/Torque logs where the host of a job is not included in the job's ID string
    • Support references into complex source records such as JSON objects
  • Application Kernels
    • Several minor bug fixes and UI improvements
• Bug Fixes
  • General
    • Improved error reporting
    • Fixed several issues where JSON was not properly encoded
    • For end dates that fall on the current date or in the future, do not automatically adjust the date to the end of the aggregation period that it falls into.
    • Fix the Show Guidelines button in the Usage tab
    • Fixed an issue with data filtering when an unprivileged user tries to view timeseries data for a restricted realm or statistic
  • Report Generator
    • Make timeframes editable for individual charts
  • User Interface
    • Fix the "TypeError: element is undefined" error when plotting Pie chart in metric explorer
  • ETL
    • Ignore duplicate hosts found in LSF accounting log files
    • Don't verify data endpoints associated with disbaled actions
    • Always regenerate source data queries prior to execution to ensure that any modified ETL variables are properly applied
    • Update host list parser to ensure that empty host names are not returned
    • Remove PHP memory limit when running ETL pipelines

v7.5.0-rc.2: Disallow Updating of Federated User's Passwords (#444)

* Disallow editing passwords for federated users.

v7.5.0-rc1

Merge pull request #440 from smgallo/etl/flush-before-load

Flush data file before calling LOAD DATA INFILE

Open XDMoD 7.0.1

• Bug Fixes
  • General
    • Fixed compatibility with PHP 5.3.3 (#269)

Open XDMoD 7.0.0

• Features
  • General
    • Enhanced authorization framework (#97, #146, #206)
    • Improved login prompt to automatically appear when an unauthenticated user attempts to access any private tab (#110)
    • Improved design of Metric Explorer Load Chart menu (#144)
    • Re-enabled aggregate mode for "Wall Hours: Per Job" metric (#186)
    • Added quarterly report template for center directors (#199)
    • Improved support for third-party PHP libraries used by modules (#205)
  • ETLv2
    • Added support for RFC-6901 JSON References (#100, #166)
    • Added file directory data endpoint (#154)
    • Improved support for order_id columns in Open XDMoD database (#201)
    • Refactored various components to support new features (#138, #145, #151, #167, #174, #180, #196)
  • Job Viewer
    • Added Gantt chart view of job peers (#153, #164)
    • Added ability to link directly to jobs (#156)
    • Improved user feedback while job data is loading (#168)
• Bug Fixes
  • General
    • Fixed various compatibility issues with PHP 7 (#101, #183)
    • Fixed handling of 4-byte UTF-8 characters during XRAS ingestion (#122)
    • Improved handling of invalid start and end dates received by API (#160)
    • Fixed validation of length of names in contact forms (#175)
    • Improved handling of jobs with "0" start or end time (#197)
    • Fixed case where About tab would display a blank page when loaded using Chrome (#232)
  • Job Viewer
    • Fixed directions not always appearing when all jobs are closed (#155)
    • Fixed memory leak (#149)
    • Fixed duplicate search nodes being created when opening jobs from Metric Explorer charts with "#" in the title (#236)
  • Metric Explorer
    • Fixed handling of "%" characters in Metric Explorer options when using Firefox (#114)
    • Fixed XSS vulnerability involving chart names (#239)
  • Usage
    • Fixed error bars option being enabled when error bars are not available (#188)
    • Fixed metrics appearing to be available to users that do not have access (#189)
• Miscellaneous
  • Moved Node.js ETL framework to Open XDMoD repository (#106)
  • Fixed build script running out of memory allocated by PHP (#118)
  • Performed work in anticipation of federated instances (#148)
  • Improved development workflow (#124, #157, #195)
  • Improved quality assurance (#107, #116, #134, #143, #150, #163, #165, #169, #173, #184, #185, #187, #190, #193, #194, #198, #212, #235)
  • Cleaned up old and/or unused code (#104, #105, #109, #112, #117, #128, #158, #159, #182, #191, #213)
  • Improved documentation (#161, #202, #203, #229, #247)