Summary of the paper
| Title | LoonyBin: Keeping Language Technologists Sane through Automated Management of Experimental (Hyper)Workflows |
|---|---|
| Authors | Jonathan H. Clark and Alon Lavie |
| Abstract | Many contemporary language technology systems are characterized by longpipelines of tools with complex dependencies. Too often, these workflows areimplemented by ad hoc scripts; or, worse, tools are run manually, makingexperiments difficult to reproduce. These practices are difficult to maintainin the face of rapidly evolving workflows while they also fail to expose andrecord important details about intermediate data. Further complicating thesesystems are hyperparameters, which often cannot be directly optimized byconventional methods, requiring users to determine which combination of valuesis best via trial and error. We describe LoonyBin, an open-source tool thataddresses these issues by providing: 1) a visual interface for the user tocreate and modify workflows; 2) a well-defined mechanism for tracking metadataand provenance; 3) a script generator that compiles visual workflows into shellscripts; and 4) a new workflow representation we call a HyperWorkflow, whichintuitively and succinctly encodes small experimental variations within alarger workflow. |
| Language | |
| Topics | Tools, systems, applications |
Full paper  |
LoonyBin: Keeping Language Technologists Sane through Automated Management of Experimental (Hyper)Workflows |
| Bibtex | @InProceedings{CLARK10.570, author = {Jonathan H. Clark and Alon Lavie}, title = {LoonyBin: Keeping Language Technologists Sane through Automated Management of Experimental (Hyper)Workflows}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |