EYRA Tools

EYRA Tools is a Python package that helps you generate Docker containers containing a submission or evaluation for the EYRA Benchmark Platform.

Benchmark participants should implement a submission container, while benchmark organizers should implement an evaluation container. The Docker containers can be generated by typing:

eyra-generate [submission|evaluation] <container_name> [-d <docker hub account>]

Links

• EYRA Benchmark Platform
• EYRA Tools on Github
• EYRA Tools documentation

Warning

EYRA Tools and the documentation are works in progress. Not everything already works as advertised. If you are interested in organizing a Benchmark on the EYRA Benchmark Platform, please contact us via info@eyrabenchmark.net.

Contents

• Installation
  • Requirements
  • Prepare virtual environment
• User Manual
  • Quickstart
  • Complete example
• What are all the files?
  • Implementation
  • Local data handling
  • Scripts
  • Miscellaneous
• Scripts
  • test.sh
  • build.sh
  • push.sh
  • export.sh
• Best Practices
  • Put your code under version control
  • Put the complicated parts of your algorithm in a separate Python package
  • Use semantic versioning for your Docker tags
  • Add an Open Source license to your code
• More examples
  • Heimdall
  • Amber
  • frb-evaluation
• Demo benchmark: the Iris data
  • For participants: create a model
  • For participants: create a submission
  • For organizers: create a benchmark
  • For organizers: create an evaluation
• Building these docs