Katib GRPC API v1beta1
HP or NAS algorithm settings.
| Field | Type | Label | Description |
| name | string |
|
|
| value | string |
|
HP or NAS algorithm specification.
| Field | Type | Label | Description |
| algorithm_name | string |
|
|
| algorithm_settings | AlgorithmSetting | repeated |
|
| Field | Type | Label | Description |
| trial_name | string |
|
EarlyStoppingRule represents single early stopping rule.
| Field | Type | Label | Description |
| name | string | Name of the rule. Usually, metric name. |
|
| value | string | Value of the metric. |
|
| comparison | ComparisonType | Correlation between name and value, one of equal, less or greater |
|
| start_step | int32 | Defines quantity of intermediate results that should be received before applying the rule. If start step is empty, rule is applied from the first recorded metric. |
Early stopping algorithm settings.
| Field | Type | Label | Description |
| name | string |
|
|
| value | string |
|
Early stopping algorithm specification.
| Field | Type | Label | Description |
| algorithm_name | string |
|
|
| algorithm_settings | EarlyStoppingSetting | repeated |
|
Structure for a single Experiment.
| Field | Type | Label | Description |
| name | string | Name for the Experiment. |
|
| spec | ExperimentSpec | Experiment specification. |
Specification of an Experiment. Experiment represents a single optimization run over a feasible space.
Each Experiment contains a configuration describing the feasible space, as well as a set of Trials.
It is assumed that objective function f(x) does not change in the course of an Experiment.
| Field | Type | Label | Description |
| parameter_specs | ExperimentSpec.ParameterSpecs |
|
|
| objective | ObjectiveSpec | Objective specification for the Experiment. |
|
| algorithm | AlgorithmSpec | HP or NAS algorithm specification for the Experiment. |
|
| early_stopping | EarlyStoppingSpec | Early stopping specification for the Experiment. |
|
| parallel_trial_count | int32 | How many Trials can be processed in parallel. |
|
| max_trial_count | int32 | Max completed Trials to mark Experiment as succeeded. |
|
| nas_config | NasConfig | NAS configuration for the Experiment. |
List of ParameterSpec.
| Field | Type | Label | Description |
| parameters | ParameterSpec | repeated |
|
Feasible space for optimization.
Int and Double type use Max/Min.
Discrete and Categorical type use List.
| Field | Type | Label | Description |
| max | string | Max Value |
|
| min | string | Minimum Value |
|
| list | string | repeated | List of Values. |
| step | string | Step for double or int parameter |
| Field | Type | Label | Description |
| early_stopping_rules | EarlyStoppingRule | repeated |
|
| Field | Type | Label | Description |
| experiment | Experiment |
|
|
| trials | Trial | repeated |
|
| db_manager_address | string |
|
| Field | Type | Label | Description |
| observation_log | ObservationLog |
|
| Field | Type | Label | Description |
| trial_name | string |
|
|
| metric_name | string |
|
|
| start_time | string | The start of the time range. RFC3339 format |
|
| end_time | string | The end of the time range. RFC3339 format |
| Field | Type | Label | Description |
| parameter_assignments | GetSuggestionsReply.ParameterAssignments | repeated |
|
| algorithm | AlgorithmSpec |
|
|
| early_stopping_rules | EarlyStoppingRule | repeated |
|
| Field | Type | Label | Description |
| assignments | ParameterAssignment | repeated |
|
| trial_name | string | Optional field to override the trial name |
|
| labels | GetSuggestionsReply.ParameterAssignments.LabelsEntry | repeated | Optional field to add labels to the generated Trials |
| Field | Type | Label | Description |
| key | string |
|
|
| value | string |
|
| Field | Type | Label | Description |
| experiment | Experiment |
|
|
| trials | Trial | repeated | All completed trials owned by the experiment. |
| current_request_number | int32 | The number of Suggestions requested at one time. When you set 3 to current_request_number, you get three Suggestions at one time. |
|
| total_request_number | int32 | The number of Suggestions requested till now. |
GraphConfig contains a config of DAG
| Field | Type | Label | Description |
| num_layers | int32 | Number of layers |
|
| input_sizes | int32 | repeated | Dimensions of input size |
| output_sizes | int32 | repeated | Dimensions of output size |
| Field | Type | Label | Description |
| name | string |
|
|
| value | string |
|
| Field | Type | Label | Description |
| time_stamp | string | RFC3339 format |
|
| metric | Metric |
|
NasConfig contains a config of NAS job
| Field | Type | Label | Description |
| graph_config | GraphConfig | Config of DAG |
|
| operations | NasConfig.Operations | List of Operation |
| Field | Type | Label | Description |
| operation | Operation | repeated |
|
Objective specification.
| Field | Type | Label | Description |
| type | ObjectiveType | Type of optimization. |
|
| goal | double | Goal of optimization, can be empty. |
|
| objective_metric_name | string | Primary metric name for the optimization. |
|
| additional_metric_names | string | repeated | List of additional metrics to record from Trial. This can be empty if we only care about the objective metric. |
| Field | Type | Label | Description |
| metrics | Metric | repeated |
|
| Field | Type | Label | Description |
| metric_logs | MetricLog | repeated |
|
Config for operations in DAG
| Field | Type | Label | Description |
| operation_type | string | Type of operation in DAG |
|
| parameter_specs | Operation.ParameterSpecs |
|
List of ParameterSpec
| Field | Type | Label | Description |
| parameters | ParameterSpec | repeated |
|
| Field | Type | Label | Description |
| name | string |
|
|
| value | string |
|
Config for a hyperparameter.
Katib will create each Hyper parameter from this config.
| Field | Type | Label | Description |
| name | string | Name of the parameter. |
|
| parameter_type | ParameterType | Type of the parameter. |
|
| feasible_space | FeasibleSpace | FeasibleSpace for the parameter. |
| Field | Type | Label | Description |
| trial_name | string |
|
|
| observation_log | ObservationLog |
|
| Field | Type | Label | Description |
| trial_name | string |
|
Structure for a single Trial.
| Field | Type | Label | Description |
| name | string | Name for the Trial. |
|
| spec | TrialSpec | Trial specification. |
|
| status | TrialStatus | Trial status. |
Specification of a Trial. It represents Trial's parameter assignments and objective.
| Field | Type | Label | Description |
| objective | ObjectiveSpec | Objective specification for the Trial. |
|
| parameter_assignments | TrialSpec.ParameterAssignments | List of assignments generated for the Trial. |
|
| labels | TrialSpec.LabelsEntry | repeated | Map of labels assigned to the Trial |
| Field | Type | Label | Description |
| key | string |
|
|
| value | string |
|
List of ParameterAssignment
| Field | Type | Label | Description |
| assignments | ParameterAssignment | repeated |
|
Current Trial status. It contains Trial's latest condition, start time, completion time, observation.
| Field | Type | Label | Description |
| start_time | string | Trial start time in RFC3339 format |
|
| completion_time | string | Trial completion time in RFC3339 format |
|
| condition | TrialStatus.TrialConditionType | Trial current condition. It is equal to the latest Trial CR condition. |
|
| observation | Observation | The best Trial observation in logs. |
Return INVALID_ARGUMENT Error if Algorithm Settings are not Valid
| Field | Type | Label | Description |
| experiment | Experiment |
|
Return INVALID_ARGUMENT Error if Early Stopping Settings are not Valid
| Field | Type | Label | Description |
| early_stopping | EarlyStoppingSpec |
|
| Name | Number | Description |
| UNKNOWN_COMPARISON | 0 | Unknown comparison, not used |
| EQUAL | 1 | Equal comparison, e.g. accuracy = 0.7 |
| LESS | 2 | Less comparison, e.g. accuracy < 0.7 |
| GREATER | 3 | Greater comparison, e.g. accuracy > 0.7 |
Direction of optimization. Minimize or Maximize.
| Name | Number | Description |
| UNKNOWN | 0 | Undefined type and not used. |
| MINIMIZE | 1 | Minimize |
| MAXIMIZE | 2 | Maximize |
Types of value for HyperParameter.
| Name | Number | Description |
| UNKNOWN_TYPE | 0 | Undefined type and not used. |
| DOUBLE | 1 | Double float type. Use "Max/Min". |
| INT | 2 | Int type. Use "Max/Min". |
| DISCRETE | 3 | Discrete number type. Use "List" as float. |
| CATEGORICAL | 4 | Categorical type. Use "List" as string. |
Trial can be in one of 8 conditions.
TODO (andreyvelich): Remove unused conditions.
| Name | Number | Description |
| CREATED | 0 | |
| RUNNING | 1 | |
| SUCCEEDED | 2 | |
| KILLED | 3 | |
| FAILED | 4 | |
| METRICSUNAVAILABLE | 5 | |
| EARLYSTOPPED | 6 | |
| UNKNOWN | 7 |
DBManager service defines APIs to manage Katib database.
| Method Name | Request Type | Response Type | Description |
| ReportObservationLog | ReportObservationLogRequest | ReportObservationLogReply | Report a log of Observations for a Trial. The log consists of timestamp and value of metric. Katib store every log of metrics. You can see accuracy curve or other metric logs on UI. |
| GetObservationLog | GetObservationLogRequest | GetObservationLogReply | Get all log of Observations for a Trial. |
| DeleteObservationLog | DeleteObservationLogRequest | DeleteObservationLogReply | Delete all log of Observations for a Trial. |
EarlyStopping service defines APIs to manage Katib Early Stopping algorithms
| Method Name | Request Type | Response Type | Description |
| GetEarlyStoppingRules | GetEarlyStoppingRulesRequest | GetEarlyStoppingRulesReply | |
| SetTrialStatus | SetTrialStatusRequest | SetTrialStatusReply | |
| ValidateEarlyStoppingSettings | ValidateEarlyStoppingSettingsRequest | ValidateEarlyStoppingSettingsReply |
Suggestion service defines APIs to manage Katib Suggestion from HP or NAS algorithms
| Method Name | Request Type | Response Type | Description |
| GetSuggestions | GetSuggestionsRequest | GetSuggestionsReply | |
| ValidateAlgorithmSettings | ValidateAlgorithmSettingsRequest | ValidateAlgorithmSettingsReply |
| .proto Type | Notes | C++ | Java | Python | Go | C# | PHP | Ruby |
| double | double | double | float | float64 | double | float | Float | |
| float | float | float | float | float32 | float | float | Float | |
| int32 | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required) |
| int64 | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. | int64 | long | int/long | int64 | long | integer/string | Bignum |
| uint32 | Uses variable-length encoding. | uint32 | int | int/long | uint32 | uint | integer | Bignum or Fixnum (as required) |
| uint64 | Uses variable-length encoding. | uint64 | long | int/long | uint64 | ulong | integer/string | Bignum or Fixnum (as required) |
| sint32 | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required) |
| sint64 | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. | int64 | long | int/long | int64 | long | integer/string | Bignum |
| fixed32 | Always four bytes. More efficient than uint32 if values are often greater than 2^28. | uint32 | int | int | uint32 | uint | integer | Bignum or Fixnum (as required) |
| fixed64 | Always eight bytes. More efficient than uint64 if values are often greater than 2^56. | uint64 | long | int/long | uint64 | ulong | integer/string | Bignum |
| sfixed32 | Always four bytes. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required) |
| sfixed64 | Always eight bytes. | int64 | long | int/long | int64 | long | integer/string | Bignum |
| bool | bool | boolean | boolean | bool | bool | boolean | TrueClass/FalseClass | |
| string | A string must always contain UTF-8 encoded or 7-bit ASCII text. | string | String | str/unicode | string | string | string | String (UTF-8) |
| bytes | May contain any arbitrary sequence of bytes. | string | ByteString | str | []byte | ByteString | string | String (ASCII-8BIT) |