RESTful API for MyTardis¶
The data and metadata stored in MyTardis instances is made accessible through a RESTful API.
Not all functionality has been exposed via the API at this time. This documentation reflects what is available and tested.
The API version v1
is built on the Tastypie framework.
The RESTful API can also be explored via the automatically generated Swagger documentation at http://mytardis-example.com/api/v1/swagger/.
See swagger.io for details on the Swagger standard.
API accessible models¶
- Experiment
- ExperimentParameterSet
- ExperimentParameter
- Dataset
- DatasetParameterSet
- DatasetParameter
- DataFile
- DatafileParameterSet
- DatafileParameter
- StorageBox
- StorageBoxOption
- StorageBoxAttribute
- Schema
- ParameterName
- User
- Group
- Facility
- Instrument
- ObjectACL
Authentication¶
Currently implemented are Basic Auth, to be used via HTTPS only, and SessionAuth which queries Django sessions.
Due to our desire to provide information to users without any login, eg. for public data, the Basic Auth mechanism is slightly non-standard.
The standard sends an anonymous request, awaits a WWW-Authenticate header, then sends authentication credentials. Instead, this API sends public data for anonymous requests.
Using curl
or the requests
library this poses no problem. However,
using urllib2
or web browser without a Django session is not going to work
out of the box.
Here is a snippet (found here: http://stackoverflow.com/questions/4628610/does-urllib2-support-preemptive-authentication-authentication) that makes urllib2 work, should you want to use this library:
class PreemptiveBasicAuthHandler(urllib2.BaseHandler):
def __init__(self, password_mgr=None):
if password_mgr is None:
password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
self.passwd = password_mgr
self.add_password = self.passwd.add_password
def http_request(self, req):
uri = req.get_full_url()
user, pw = self.passwd.find_user_password(None, uri)
if pw is None:
return req
raw = "%s:%s" % (user, pw)
auth = 'Basic %s' % base64.b64encode(raw).strip()
req.add_unredirected_header('Authorization', auth)
return req
auth_handler = PreemptiveBasicAuthHandler()
auth_handler.add_password(realm=None,
uri=url,
user='mytardis',
passwd='mytardis')
opener = urllib2.build_opener(auth_handler)
# ...and install it globally so it can be used with urlopen.
urllib2.install_opener(opener)
Querying the database (GET)¶
All endpoints support querying lists and individual records via GET requests. Some support more complex queries via GET parameters as well.
Creating objects, adding files (POST)¶
The creation of Experiments, Datasets and Dataset_Files via POSTs with the option to include metadata/parametersets has been implemented and tested.
The following examples demonstrate how to go about it.
In all except the file attachment case the POST data should be a JSON string,
the Content-Type
header needs to be set to application/json
and the
Accept
header as well. Other response formats may be made available in the
future.
In all cases the URI of the created object is returned in the Location
header of the response.
Experiments¶
Example JSON input
{
"title": "API-created Experiment #1",
"description": "Wow, all automatic!",
"institution_name": "Monash University",
"parameter_sets": [
{
"schema": "http://institution.com/my/schema",
"parameters": [
{
"name": "important_parameter_1",
"value": "Test16"
},
{
"name": "important_parameter_3",
"value": "57.136"
}
]
},
{
"schema": "http://company.com/some/other/schema",
"parameters": [
{
"name": "meaningful_name",
"value": "Test17"
},
{
"name": "meaningless_name",
"value": "1234"
}
]
}
]
}
This creates an experiment with two parametersets with two parameters each.
Alternative to Schema namespaces and Parameter names, you can also specify the URIs to each. Until the querying of Schemas and Parameters is documented this is discouraged.
Datasets¶
Example JSON input:
{
"description": "API-created Dataset",
"experiments": [
"/api/v1/experiment/1/",
"/api/v1/experiment/2/"
],
"immutable": false,
"parameter_sets": [
{
"parameters": [
{
"name": "obscure-instrument-setting-52",
"value": "awesome dataset api POST"
},
{
"name": "temperature",
"value": "301"
}
],
"schema": "http://datasets.com/need/schemas/too"
},
{
"parameters": [
{
"name": "someotherparameter",
"value": "some other value"
}
],
"schema": "http://better-datasets.com/offers/better/schemas"
}
]
}
DataFiles¶
There are three ways to add a file to MyTardis via the API.
Via multipart form POST¶
This works for single files at the moment.
The key is to send a multipart-form instead of ‘application/json’. This can be accomplished with the requests library as shown in the following example.
To use requests you need to install it first, eg. pip install requests
.
Also, for this to work, the POST data needs to be sent with the JSON string
called 'json_data'
and the file called 'attached_file'
.
Example JSON input:
{
"dataset": "/api/v1/dataset/1/",
"filename": "mytestfile.txt",
"md5sum": "c858d6319609d6db3c091b09783c479c",
"size": "12",
"mimetype": "text/plain",
"parameter_sets": [{
"schema": "http://datafileshop.com/fileinfo/v1",
"parameters": [{
"name": "fileparameter1",
"value": "123"
},
{
"name": "fileparameter2",
"value": "1234"
}]
}]
}
Example requests script:
import requests
from requests.auth import HTTPBasicAuth
url = "http://localhost:8000/api/v1/dataset_file/"
headers = {'Accept': 'application/json'}
response = requests.post(url, data={"json_data": data}, headers=headers,
files={'attached_file': open(filename, 'rb')},
auth=HTTPBasicAuth(username, password)
)
Via staging location¶
Another way to add a file is to create the database entry first without providing a storage location. This will return back a location on the server that you are assumed to have access to. Once the file appears there, for example when you copy it there, it will be moved to its permanent storage location managed by MyTardis.
The full file path that you should copy/move the file to is returned as the content of the response.
Example JSON input:
{
"dataset": "/api/v1/dataset/1/",
"filename": "mytestfile.txt",
"md5sum": "c858d6319609d6db3c091b09783c479c",
"size": "12",
"mimetype": "text/plain",
"parameter_sets": [{
"schema": "http://datafileshop.com/fileinfo/v1",
"parameters": [{
"name": "fileparameter1",
"value": "123"
},
{
"name": "fileparameter2",
"value": "1234"
}]
}]
}
urllib2 POST example script¶
Replace MODEL
with
one of the available model names in lower case. data
is the JSON as a
string.
import urllib2
url = "http://localhost:8000/api/v1/MODEL/"
headers = {'Accept': 'application/json',
'Content-Type': 'application/json'}
auth_handler = urllib2.HTTPBasicAuthHandler()
auth_handler.add_password(realm="django-tastypie",
uri=url,
user=username,
passwd=password)
opener = urllib2.build_opener(auth_handler)
urllib2.install_opener(opener)
myrequest = urllib2.Request(url=url, data=data,
headers=headers)
myrequest.get_method = lambda: 'POST'
output = "error"
output = urllib2.urlopen(myrequest)
print output.headers["Location"]