Validation

Related links

 

Validation

Component description

A validation corresponds to the validation of a model on a test dataset. The results are stored in another dataset. Parameters with default values are optional.

REST operations


 

Description Method URI Parameters Result Status codes
Get all validations GET / [subjectid] List of validation URIs 200,404
Retrieves a validation representation GET /{id} [subjectid] Validation representation in one of the supported MIME types 200,404
Validates a model on a test dataset POST /test_set_validation [subjectid]
model_uri
test_dataset_uri
test_target_dataset_uri (default = test_dataset_uri)
prediction_feature (default = dependent variable of model)
Validation URI or Task URI 200,400,404,500
Builds a model on a training dataset and validates it on a test dataset POST /training_test_valdiation algorithm_uri
prediction_feature
algorithm_params (string, default="")
training_dataset_uri
test_dataset_uri
test_target_dataset_uri (default = test_dataset_uri)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]
Validation URI or Task URI 200,400,404,500
Splits a dataset into training and test dataset according to a certain ratio, and performs a validation POST /training_test_split algorithm_uri
prediction_feature
algorithm_params (string, default="")
dataset_uri
split_ratio (float, default=0.66)
random_seed (integer, default=1)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]
validation URI or Task URI 200,400,404,500
Performs a bootstrap validation POST /bootstrapping algorithm_uri
prediction_feature
dataset_params (string, default="")
dataset_uri
bootstrap_percentage (float, default=0.66)
random_seed (integer, default=1)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]
Validation URI or Task URI 200,400,404,500
Directly perform a validation by specifying test- and prediction dataset  POST  /validate_datasets prediction_feature
test_dataset_uri
test_target_dataset_uri (default = test_dataset_uri)
prediction_dataset_uri
predicted_feature (.i.e feature in prediction dataset)
[subjectid]
Validation URI or Task URI 200,400,404,500
Deletes a validation. DELETE /{id} [subjectid] - 200,404

Validation representation

  • Requested MIME type should be set in the requests "Accept" header, e.g. curl -X GET -H "Accept:application/xml" http://{server}/validation/{id}
  • RDF representation defined in opentox.owl
  • subjectid (optional) parameter that contains the OpenSSO A&A token needed to access protected services.

Supported MIME types:

Mandatory:

•application/rdf+xml (default)

Optional:

•application/xml, see XML schema definition for validation object

HTTP status codes

Interpretation Nr Name
Success 200 OK
Validation not found 404 Not Found
Illegal model/algorithm/dataset/algorithm params 400 Bad request
Validation/prediction error 500 Internal Server Error

Cross-Validation

Component description

Performs a k-fold cross-validation, which results in k validations. Parameters with default values are optional.

REST operations


 

Description Method URI Parameters Result Status codes
Get all cross-validations GET /crossvalidation [subjectid] List of crossvalidation URIs 200,404
Retrieves a cross-validation representation GET /crossvalidation/{id} [subjectid] Cross-Validation in one of the supported MIME types 200,404
Returns all (k) validations that belong to a crossvalidation GET /crossvalidation/{id}/validations [subjectid] List of validation URIs 200,404
Performs a k-fold cross-validation. POST /crossvalidation algorithm_uri
prediction_feature
algorithm_params (string, default="")
num_folds (integer, default=10)
random_seed (integer, default=1)
stratified (boolean, default=true)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]
Cross-Validation URI or Task URI 200,400,404,500
Performs a leave-one-out cross-validation. POST /crossvalidation/loo algorithm_uri
prediction_feature
algorithm_params (string, default="")
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]
Cross-Validation URI or Task URI 200,400,404,500
Deletes a cross-validation. DELETE /crossvalidation/{id} [subjectid] - 200,404

Cross-validation representation

  • Requested MIME type should be set in the requests "Accept" header, e.g. curl -X GET -H "Accept:application/xml" http://{server}/crossvalidation/{id}

Supported MIME types:

Mandatory:

  • application/rdf+xml (default)

Optional:

 

HTTP status codes

Interpretation Nr Name
Success 200 OK
Cross validation not found 404 Not Found
Illegal model/algorithm/dataset/algorithm params 400 Bad request
Validation/prediction error 500 Internal Server Error

Document Actions

Validation - Report

Component description

A report visualizes the (prediction) results of algorithms.

REST operations


 

Description Method URI Parameters Result Status codes
Get all report types GET /report [subjectid] List of available report types 200,404
Get all reports for the particular report type GET /report/{report-type} [subjectid] List of available reports as URI 200,404
Retrieves a report, format according to accept-header GET /report/{report-type}/{id} [subjectid] According to accept-header:
text/x-yaml, application/rdf+xml
-> report object with meta data (see rdf example below)

text/xml, text/html, application/pdf
-> report document in specified format

200,404
Creates a report POST /report/{report-type} various params, see below
[subjectid]
Report URI or Task URI 200,400,404,500
Deletes a report. DELETE /report/{report-type}/{id} [subjectid] - 200,404
Available (validation-)report types          
Create single validation report
(one model, one test dataset)
POST /report/validation validation_uris (comma separated list of validation URIs)
[subjectid]
Report URI or Task URI 200,400,404,500
Create cross-validation report
(crossvalidation with one algorithm and one dataset)
POST /report/crossvalidation validation_uris (comma separated list of crossvalidation URIs)
[subjectid]
Report URI or Task URI 200,400,404,500
Create report for comparing different prediction algorithms
(crossvalidations/validations with multiple algorithms and datasets)
POST /report/algorithm_comparison validation_uris (comma separated list of crossvalidation URIs)
identifier (comma seperated list of string-ids, one for each crossvalidation-uri)
[subjectid]
Report URI or Task URI 200,400,404,500

 

Output (file) format

Validation reports are returned in 2 different ways according to the accept-header

  • Accept header is text/xml, text/html, application/pdf: the actual report document is returned (the core document is an XML format according to the DocBook standard, which allows conversion to the other formats).
  • Accept header is application/rdf+xml or text/x-yaml: the report object is returned, containing meta-data. Example (in N3) below
@prefix : <http://www.opentox.org/api/1.1#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
    
<http://opentox.informatik.uni-freiburg.de/validation/report/validation/8>     
a :ValidationReport;
	:date "Wed Jun 09 13:23:28 +0200 2010"^^<http://www.w3.org/2001/XMLSchema#dateTime>;
        :reportAlgorithm <http://ambit.uni-plovdiv.bg:8080/ambit2/algorithm/LR>;
        :reportModel <http://ambit.uni-plovdiv.bg:8080/ambit2/model/359152>;
        :reportType "validation"^^<http://www.w3.org/2001/XMLSchema#string>;
        :reportValidation <http://opentox.informatik.uni-freiburg.de/validation/20> .

HTTP status codes

Interpretation Nr Name
Success 200 OK
Report type / report not found 404 Not Found
Illegal params 400 Bad request
Error creating the report 500 Internal Server Error