Protocol Documentation

Table of Contents

api.proto

Top

Katib GRPC API v1beta1

AlgorithmSetting

HP or NAS algorithm settings.

FieldTypeLabelDescription
name string

value string

AlgorithmSpec

HP or NAS algorithm specification.

FieldTypeLabelDescription
algorithm_name string

algorithm_settings AlgorithmSetting repeated

DeleteObservationLogReply

DeleteObservationLogRequest

FieldTypeLabelDescription
trial_name string

EarlyStoppingRule

EarlyStoppingRule represents single early stopping rule.

FieldTypeLabelDescription
name string

Name of the rule. Usually, metric name.

value string

Value of the metric.

comparison ComparisonType

Correlation between name and value, one of equal, less or greater

start_step int32

Defines quantity of intermediate results that should be received before applying the rule. If start step is empty, rule is applied from the first recorded metric.

EarlyStoppingSetting

Early stopping algorithm settings.

FieldTypeLabelDescription
name string

value string

EarlyStoppingSpec

Early stopping algorithm specification.

FieldTypeLabelDescription
algorithm_name string

algorithm_settings EarlyStoppingSetting repeated

Experiment

Structure for a single Experiment.

FieldTypeLabelDescription
name string

Name for the Experiment.

spec ExperimentSpec

Experiment specification.

ExperimentSpec

Specification of an Experiment. Experiment represents a single optimization run over a feasible space.

Each Experiment contains a configuration describing the feasible space, as well as a set of Trials.

It is assumed that objective function f(x) does not change in the course of an Experiment.

FieldTypeLabelDescription
parameter_specs ExperimentSpec.ParameterSpecs

objective ObjectiveSpec

Objective specification for the Experiment.

algorithm AlgorithmSpec

HP or NAS algorithm specification for the Experiment.

early_stopping EarlyStoppingSpec

Early stopping specification for the Experiment.

parallel_trial_count int32

How many Trials can be processed in parallel.

max_trial_count int32

Max completed Trials to mark Experiment as succeeded.

nas_config NasConfig

NAS configuration for the Experiment.

ExperimentSpec.ParameterSpecs

List of ParameterSpec.

FieldTypeLabelDescription
parameters ParameterSpec repeated

FeasibleSpace

Feasible space for optimization.

Int and Double type use Max/Min.

Discrete and Categorical type use List.

FieldTypeLabelDescription
max string

Max Value

min string

Minimum Value

list string repeated

List of Values.

step string

Step for double or int parameter

GetEarlyStoppingRulesReply

FieldTypeLabelDescription
early_stopping_rules EarlyStoppingRule repeated

GetEarlyStoppingRulesRequest

FieldTypeLabelDescription
experiment Experiment

trials Trial repeated

db_manager_address string

GetObservationLogReply

FieldTypeLabelDescription
observation_log ObservationLog

GetObservationLogRequest

FieldTypeLabelDescription
trial_name string

metric_name string

start_time string

The start of the time range. RFC3339 format

end_time string

The end of the time range. RFC3339 format

GetSuggestionsReply

FieldTypeLabelDescription
parameter_assignments GetSuggestionsReply.ParameterAssignments repeated

algorithm AlgorithmSpec

early_stopping_rules EarlyStoppingRule repeated

GetSuggestionsReply.ParameterAssignments

FieldTypeLabelDescription
assignments ParameterAssignment repeated

GetSuggestionsRequest

FieldTypeLabelDescription
experiment Experiment

trials Trial repeated

All completed trials owned by the experiment.

request_number int32

Deprecated. Use current_request_number instead. This API will be removed in Katib 0.14. The number of Suggestions requested at one time.

current_request_number int32

The number of Suggestions requested at one time. When you set 3 to current_request_number, you get three Suggestions at one time.

total_request_number int32

The number of Suggestions requested till now.

Fields with deprecated option

Name Option
request_number

true

GraphConfig

GraphConfig contains a config of DAG

FieldTypeLabelDescription
num_layers int32

Number of layers

input_sizes int32 repeated

Dimensions of input size

output_sizes int32 repeated

Dimensions of output size

Metric

FieldTypeLabelDescription
name string

value string

MetricLog

FieldTypeLabelDescription
time_stamp string

RFC3339 format

metric Metric

NasConfig

NasConfig contains a config of NAS job

FieldTypeLabelDescription
graph_config GraphConfig

Config of DAG

operations NasConfig.Operations

List of Operation

NasConfig.Operations

FieldTypeLabelDescription
operation Operation repeated

ObjectiveSpec

Objective specification.

FieldTypeLabelDescription
type ObjectiveType

Type of optimization.

goal double

Goal of optimization, can be empty.

objective_metric_name string

Primary metric name for the optimization.

additional_metric_names string repeated

List of additional metrics to record from Trial. This can be empty if we only care about the objective metric.

Observation

FieldTypeLabelDescription
metrics Metric repeated

ObservationLog

FieldTypeLabelDescription
metric_logs MetricLog repeated

Operation

Config for operations in DAG

FieldTypeLabelDescription
operation_type string

Type of operation in DAG

parameter_specs Operation.ParameterSpecs

Operation.ParameterSpecs

List of ParameterSpec

FieldTypeLabelDescription
parameters ParameterSpec repeated

ParameterAssignment

FieldTypeLabelDescription
name string

value string

ParameterSpec

Config for a hyperparameter.

Katib will create each Hyper parameter from this config.

FieldTypeLabelDescription
name string

Name of the parameter.

parameter_type ParameterType

Type of the parameter.

feasible_space FeasibleSpace

FeasibleSpace for the parameter.

ReportObservationLogReply

ReportObservationLogRequest

FieldTypeLabelDescription
trial_name string

observation_log ObservationLog

SetTrialStatusReply

SetTrialStatusRequest

FieldTypeLabelDescription
trial_name string

Trial

Structure for a single Trial.

FieldTypeLabelDescription
name string

Name for the Trial.

spec TrialSpec

Trial specification.

status TrialStatus

Trial status.

TrialSpec

Specification of a Trial. It represents Trial's parameter assignments and objective.

FieldTypeLabelDescription
objective ObjectiveSpec

Objective specification for the Trial.

parameter_assignments TrialSpec.ParameterAssignments

List of assignments generated for the Trial.

TrialSpec.ParameterAssignments

List of ParameterAssignment

FieldTypeLabelDescription
assignments ParameterAssignment repeated

TrialStatus

Current Trial status. It contains Trial's latest condition, start time, completion time, observation.

FieldTypeLabelDescription
start_time string

Trial start time in RFC3339 format

completion_time string

Trial completion time in RFC3339 format

condition TrialStatus.TrialConditionType

Trial current condition. It is equal to the latest Trial CR condition.

observation Observation

The best Trial observation in logs.

ValidateAlgorithmSettingsReply

Return INVALID_ARGUMENT Error if Algorithm Settings are not Valid

ValidateAlgorithmSettingsRequest

FieldTypeLabelDescription
experiment Experiment

ValidateEarlyStoppingSettingsReply

Return INVALID_ARGUMENT Error if Early Stopping Settings are not Valid

ValidateEarlyStoppingSettingsRequest

FieldTypeLabelDescription
early_stopping EarlyStoppingSpec

ComparisonType

NameNumberDescription
UNKNOWN_COMPARISON 0

Unknown comparison, not used
EQUAL 1

Equal comparison, e.g. accuracy = 0.7
LESS 2

Less comparison, e.g. accuracy < 0.7
GREATER 3

Greater comparison, e.g. accuracy > 0.7

ObjectiveType

Direction of optimization. Minimize or Maximize.

NameNumberDescription
UNKNOWN 0

Undefined type and not used.
MINIMIZE 1

Minimize
MAXIMIZE 2

Maximize

ParameterType

Types of value for HyperParameter.

NameNumberDescription
UNKNOWN_TYPE 0

Undefined type and not used.
DOUBLE 1

Double float type. Use "Max/Min".
INT 2

Int type. Use "Max/Min".
DISCRETE 3

Discrete number type. Use "List" as float.
CATEGORICAL 4

Categorical type. Use "List" as string.

TrialStatus.TrialConditionType

Trial can be in one of 6 conditions.

TODO (andreyvelich): Remove unused conditions.

NameNumberDescription
CREATED 0

RUNNING 1

SUCCEEDED 2

KILLED 3

FAILED 4

EARLYSTOPPED 5

UNKNOWN 6

DBManager

DBManager service defines APIs to manage Katib database.

Method NameRequest TypeResponse TypeDescription
ReportObservationLog ReportObservationLogRequest ReportObservationLogReply

Report a log of Observations for a Trial. The log consists of timestamp and value of metric. Katib store every log of metrics. You can see accuracy curve or other metric logs on UI.
GetObservationLog GetObservationLogRequest GetObservationLogReply

Get all log of Observations for a Trial.
DeleteObservationLog DeleteObservationLogRequest DeleteObservationLogReply

Delete all log of Observations for a Trial.

EarlyStopping

EarlyStopping service defines APIs to manage Katib Early Stopping algorithms

Method NameRequest TypeResponse TypeDescription
GetEarlyStoppingRules GetEarlyStoppingRulesRequest GetEarlyStoppingRulesReply

SetTrialStatus SetTrialStatusRequest SetTrialStatusReply

ValidateEarlyStoppingSettings ValidateEarlyStoppingSettingsRequest ValidateEarlyStoppingSettingsReply

Suggestion

Suggestion service defines APIs to manage Katib Suggestion from HP or NAS algorithms

Method NameRequest TypeResponse TypeDescription
GetSuggestions GetSuggestionsRequest GetSuggestionsReply

ValidateAlgorithmSettings ValidateAlgorithmSettingsRequest ValidateAlgorithmSettingsReply

Scalar Value Types

.proto TypeNotesC++JavaPythonGoC#PHPRuby
double double double float float64 double float Float
float float float float float32 float float Float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int int32 int integer Bignum or Fixnum (as required)
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long int64 long integer/string Bignum
uint32 Uses variable-length encoding. uint32 int int/long uint32 uint integer Bignum or Fixnum (as required)
uint64 Uses variable-length encoding. uint64 long int/long uint64 ulong integer/string Bignum or Fixnum (as required)
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int int32 int integer Bignum or Fixnum (as required)
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long int64 long integer/string Bignum
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int uint32 uint integer Bignum or Fixnum (as required)
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long uint64 ulong integer/string Bignum
sfixed32 Always four bytes. int32 int int int32 int integer Bignum or Fixnum (as required)
sfixed64 Always eight bytes. int64 long int/long int64 long integer/string Bignum
bool bool boolean boolean bool bool boolean TrueClass/FalseClass
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode string string string String (UTF-8)
bytes May contain any arbitrary sequence of bytes. string ByteString str []byte ByteString string String (ASCII-8BIT)