Database tables for Tuna integration

[pcf000]

Introduction

The goal of #910 is to move our currently weekly tuning runs into the Tuna workflow. For a first stage, we want to be able to push at least a single config through the Tuna flow by hand. In other words, set up only for convolutions (so we can borrow from miopen support), take one of our existing configs and stuff it into the tables, and call a script that starts up a Tuna session and job, makes a tuning run, and returns results into the database.

Subsequent stages will develop support for gemm and fusion, automation through Jenkins, and transmission of the Tuna results to the MLIR performance runs, and verify that we're doing the same tuning with the same results as the old way.

Four database tables are needed for a minimal Tuna setup: config, session, job, and results.

Config table

The purpose of the config table is to record enough information to reproduce the command run in a tuning session. For a first cut, we'll use tuningRunner.py with the --config option to run the tuning loop on a single config. We may choose to trim the command down later, but tuningRunner.py is known to work.

The fields in the table will be

id              Integer, primary_key=True
data_type	String(60)
fil_layout      String(8)
in_layout       String(8)
out_layout      String(8) 
direction	String(8)
spatial_dim	Integer
in_channels	Integer
in_h            Integer
in_w            Integer
fil_h           Integer
fil_w           Integer
out_channels	Integer
batchsize	Integer
pad_h           Integer
pad_w           Integer
conv_stride_h	Integer
conv_stride_w	Integer
dilation_h	Integer
dilation_w	Integer
kernel_repeats  Integer
group_size 	Integer

This schema is adapted from the miopen convolution config, but is not as general because we don't need that. We can revisit later if necessary.

Integer fields default to 0. String fields default to the empty string.

Session table

A session is described as "capturing anything that might affect tuning results." For us, that means arch (gfx908, etc), num_cu, rocm version, and mlir version. The mlir version is likely to be a git hash instead of a simple version. It also records some info about the context of the tuning run; at the moment those fields are reason, ticket, and docker.

The fields will be

id         Integer, primary_key=True
arch       String(20)
num_cu     Integer
rocm_v     String(64)
mlir_v     String(64)
reason     String(60)
ticket     String(64)
docker     String(64)

Job table

The job table holds information necessary for running and monitoring a single tuning job. #976 mentions session through result in the list below. MIOpen's job table is more complex, and I've borrowed some fields from it that appear useful.

The fields will be

id              Integer, primary_key=True
session         Integer, foreign key
config          Integer, foreign key
reason          String(60)
state           JobEnum
retries         Integer
result          Text
compile_start   DateTime
compile_end     DateTime
gpu_id          Integer
machine_name    String(60)

We'll have to double-check exactly what the job returns for the result and make sure we have the right type for that field. The result of a job and the exact format of the results table must be designed together.

Results table

The results table holds the tuning results in an arrangement that is amenable to looking up for performance runs. It has fields for session, config, and perf-config. In practice, we could look up the latest session with the desired arch, and use that session plus the config to look up the perf-config. Or we could convert the results table into a .tsv file, which is just arch, config, and perf-config.

It's likely to be useful to record all the tuning runs, to measure performance trends even in cases that don't win. For that, we could either make an auxiliary results table that saves everything, or we could put it all in a single table but have a "winner" flag. The former separates concerns better.

The fields of the results table will be

id              Integer, primary_key=True
session         Integer, foreign key
config          Integer, foreign key
perf_config     Text
kernel_time     Float

[krzysz00]

Question 1: What about GEMMs?
That is, do you want to put convolutions and gemms in the same config table and then just not set the fields that aren't relevant?
Or will be there two types of config: MLIR conv and MLIR gemm?

Re results table, SQL does have the ability to GROUP BY and take a minimum to find the winner.
Or, if we don't like it, I'd be good with the winners table: session, config, and a key into the results table.

[pcf000]

1. Just conv for this stage. There will likely be a separate config table for gemm.
2. Could be debated endlessly. Writing a .tsv of the winners keeps it local to performance run but depends on a string match; group-by adds some extra overhead to queries for performance run; winners-table still has the round-trip to the sql server.

[yiqian1]

It would be useful to have input_datatype, filter_datatype, and output_datatype in the Config table to support mixed types. Alternatively the data_type field specifies mixed types using the syntax type_type_type.

[pcf000]

The miopen support goes all the way to having a "tensor table" whose entries define dimensions, layout, and data type, and then input and weight refer to that table. I don't think I want to go that far, but separate datatype fields are easy. We'd need to update rocmlir-gen to take advantage of them, though.

[krzysz00]

rocmlir-gen has already been updated for that: I wrote that PR

[krzysz00]

(Or at least I'm pretty sure I did, I know I got Conv2DGenerator)

[pcf000]

Ah, the underscore type_type thing. I was looking for multiple options and missed that.