Competitions

• One way to compare different ways of solving a problem is a competition
• This is relatively common in AI
• For example, the Message Understanding Competitions (MUCs) were open to any participants and the task was Text Extraction.
• Text extraction systems read texts (usually newspaper articles), and generate a database entry based on the text.
• It's a domain specific task (e.g. terrorist events in Central America, or rocket launches.)
• It enables different systems to share resources.
• It also enables a relatively fair comparison.
• In MUCs case, it lead the field of NLP from toy systems to commercial applications (they coined the term Text Extraction) in about 15 years.
• There are also frequently machine learning competitions.
• Another example is the Loebner prize (Turing test).