Skip to content
This repository has been archived by the owner on Jan 21, 2022. It is now read-only.

Install Abacus

Jump to bottom
Adriana Dimitrova edited this page Mar 2, 2018 · 3 revisions

Abacus runs as a set of applications deployed to Cloud Foundry. Each application is configured to run in multiple instances for availability and performance. Service usage data is stored in CouchDB or MongoDB databases.

Creating a Billing epoch and Abacus CF bridges

Before starting Abacus for the first time you need to create your billing epoch. Abacus comes with a CF app bridge and services bridge that act as Resource providers for Cloud Foundry app and service runtime usage. They read Cloud Foundry usage events using Cloud Controller API and report usage to the Abacus usage collector. The Abacus CF bridges enables you to see runtime usage reports for the apps and services, running on your Cloud Foundry instance. In order to start Abacus with its CF bridges, you should execute the purge-events script for your system once. Make sure you have exported the right environment variables and that you have the right UAA clients created. To start the bridges, use the cf profile in the following way:

npm start -- cf

Local Development

Deploying Abacus

The following steps assume a local Cloud Foundry deployment created using Bosh-Lite, running on the default local IP assigned by the Bosh-Lite CF installation script.

cd cf-abacus

# Point CF CLI to your local Cloud Foundry deployment
# Create a CF security group for the Abacus apps
bin/cfsetup

# Run cf push on the Abacus apps to deploy them to Cloud Foundry
npm run cfpush

# Check the state of the Abacus apps
cf app

Running the Demo

The Abacus demo runs a simple test program that simulates the submission of usage by a Cloud service provider, then gets a daily report for the usage aggregated within a Cloud Foundry organization.

The demo data is stored in a small in-memory PouchDB test database so the demo is self-contained and you don't need to set up a real CouchDB database in order to run it.

Once the Abacus apps are running on your Cloud Foundry deployment, proceed as follows:

cd cf-abacus

# Run the demo script
npm run demo -- \
  --collector https://abacus-usage-collector.both-lite.com \
  --reporting https://abacus-usage-reporting.both-lite.com

# You should see usage being submitted and a usage report for the demo organization

Cloud Foundry

Application Manifests

All application manifests should be customized to reflect the setup of the Cloud Foundry landscape you are using.

Environment variables

The environment variables that should be set can be divided into the following groups:

CF-specific

These variables can be obtained from your deployment manifest or the CF CLI target.

API: https://api.<domain>:443
AUTH_SERVER: https://api.<domain>:443
JWTALGO: RS256
JWTKEY: -----BEGIN PUBLIC KEY-----
<  UAA public key >
-----END PUBLIC KEY-----
Abacus landscape

These include installation-specific variables: UAA client IDs, secrets, and application names.

CF_CLIENT_ID: abacus-cf-bridge
CF_CLIENT_SECRET: <secret>
CLIENT_ID: abacus-linux-container
CLIENT_SECRET: <secret>
COLLECTOR: abacus-usage-collector
EUREKA: abacus-eureka-plugin
EUREKA_USER: abacus-hystrix
EUREKA_PASSWORD: <secret>
SECURED: true
Abacus-specific

These include variables related to scaling, debugging, and plan handling.

EVAL_VMTYPE: vm
DEBUG: e-abacus-*
NODE_MODULES_CACHE: false
NODE_OPTS: --max_old_space_size=256
BATCH_SIZE: 50
THROTTLE: 100
DB

These allow you to specify what DB should be used (service or external URL), type of database, and finally, specify app-level sharding if needed.

DBALIAS: db-common
DBCLIENT: abacus-mongoclient

Example manifests

Applications bridge 
### CF installation ###
API: https://api.<domain>:443
AUTH_SERVER: https://api.<domain>:443
JWTALGO: RS256
JWTKEY: -----BEGIN PUBLIC KEY-----
<  UAA public key >
-----END PUBLIC KEY-----

PRIMARY_DOMAIN: <domain>

### Abacus landscape ###
CF_CLIENT_ID: abacus-cf-bridge
CF_CLIENT_SECRET: <secret>
CLIENT_ID: abacus-linux-container
CLIENT_SECRET: <secret>
COLLECTOR: abacus-usage-collector
EUREKA: abacus-eureka-plugin
EUREKA_USER: abacus-hystrix
EUREKA_PASSWORD: <secret>
SECURED: true

### Abacus specific ###
EVAL_VMTYPE: vm
DEBUG: e-abacus-*
NODE_MODULES_CACHE: false
NODE_OPTS: --max_old_space_size=256
BATCH_SIZE: 50
THROTTLE: 100

### DB ###
DBALIAS: db-common
DBCLIENT: abacus-mongoclient
Accumulator

The accumulators use the following set of variables:

ACCOUNT: abacus-account-plugin
AGGREGATOR: abacus-usage-aggregator
API: https://api.<domain>:443
AUTH_SERVER: https://api.<domain>:443
BATCH_SIZE: 50
CLIENT_ID: abacus
CLIENT_SECRET: <secret>
CONF: large
DBALIAS: db-common
DBCLIENT: abacus-mongoclient
DEBUG: e-abacus-*
EUREKA: abacus-eureka-plugin
EUREKA_USER: abacus-hystrix
EUREKA_PASSWORD: <secret>
EVAL_TIMEOUT: 800
EVAL_VMTYPE: vm
JWTALGO: RS256
JWTKEY: -----BEGIN PUBLIC KEY-----
<UAA public key>
-----END PUBLIC KEY-----

NODE_MODULES_CACHE: false
NODE_OPTS: --max_old_space_size=256
PRIMARY_DOMAIN: <domain>
PROVISIONING: abacus-provisioning-plugin
REPLAY: 1800000
SAMPLING: 1h
SECURED: true
SLACK: 5D
THROTTLE: 100

Note that we have SAMPLING above, that means accumulator will create a new accumulated data document every hour. This reduces the amount of data in the DB, but also means we can lose 1h of data in case of crash, corrupted database, and so forth.

Also we do a replay of our input DB to guarantee no data is stuck in our input DBs by supplying REPLAY with a value of 30 minutes. In this way we allow our applications to be down for updates for up to 30 minutes.

UAA Clients

There are several UAA clients that Abacus needs to function correctly in secured mode. In case you use the application and service bridges, you need to create UAA clients to fetch usage events from Cloud Controller. All of these are created by our Concourse pipeline.

Additionally each Resource Provider needs to authenticate via token. We automate this via Abacus broker.

All UAA client IDs and secrets can be changed, but they should contain the right scopes that Abacus uses for authentication/authorization of Resource Providers.

Database

Abacus requires CouchDB or MongoDB databases. Abacus Integrators can:

  • use Cloud Foundry service broker providing a database instance
  • use user-provided services
  • specify DB URL directly with DB environment variable in application manifests

To use the last two approaches you should set create-db-service: false in our Concourse pipeline, as you don't need a DB service instance to be created in this case.

<< Abacus Integrator Tutorial Set up Concourse >>

Related Links:

ABOUT | RESOURCE PROVIDER | ABACUS INTEGRATOR

*Abacus icon made by Freepik from www.flaticon.com

Logo

HOME > ABOUT

HOME > RESOURCE PROVIDER

HOME > ABACUS INTEGRATOR

Clone this wiki locally