Provides an insightful and practical introduction to crowdsourcing as a means of rapidly processing speech data Intended for those who want to get started in the domain and learn how to set up a task, what interfaces are available, how to assess the work, etc. as well as for those who already have used crowdsourcing and want to create better tasks and obtain better assessments of the work of the crowd. It will include screenshots to show examples of good and poor interfaces; examples of case studies in speech processing tasks, going through the task creation process, reviewing options in the interface, in the choice of medium (MTurk or other) and explaining choices, etc. Provides an insightful and practical introduction to crowdsourcing as a means of rapidly processing speech data. Addresses important aspects of this new technique that should be mastered before attempting a crowdsourcing application. Offers speech researchers the hope that they can spend much less time dealing with the data gathering/annotation bottleneck, leaving them to focus on the scientific issues. Readers will directly benefit from the book s successful examples of how crowd- sourcing was implemented for speech processing, discussions of interface and processing choices that worked and choices that didn t, and guidelines on how to play and record speech over the internet, how to design tasks, and how to assess workers. Essential reading for researchers and practitioners in speech research groups involved in speech processing

Provides an insightful and practical introduction to crowdsourcing as a means of rapidly processing speech dataIntended for those who want to get started in the domain and learn how to set up a task, what interfaces are available, how to assess the work, etc. as well as for those who already have used crowdsourcing and want to create better tasks and obtain better assessments of the work of the crowd. It will include screenshots to show examples of good and poor interfaces; examples of case studies in speech processing tasks, going through the task creation process, reviewing options in the interface, in the choice of medium (MTurk or other) and explaining choices, etc.* Provides an insightful and practical introduction to crowdsourcing as a means of rapidly processing speech data.* Addresses important aspects of this new technique that should be mastered before attempting a crowdsourcing application.* Offers speech researchers the hope that they can spend much less time dealing with the data gathering/annotation bottleneck, leaving them to focus on the scientific issues.* Readers will directly benefit from the book's successful examples of how crowd- sourcing was implemented for speech processing, discussions of interface and processing choices that worked and choices that didn't, and guidelines on how to play and record speech over the internet, how to design tasks, and how to assess workers.Essential reading for researchers and practitioners in speech research groups involved in speech processing

Contributors viiPreface ix1 An Overview1.1 Growing Needs for Speech Data1.1.1 Origins of Crowdsourcing1.1.2 Operational Definition of Crowdsourcing1.1.3 Functional Definition of Crowdsourcing1.2 Some Issues1.3 Some Terminology1.4 AcknowledgementsReferences2 The Basics2.1 An Overview of the Literature on Crowdsourcing for Speech Processing2.1.1 Evolution of the Use of Crowdsourcing for Speech2.1.2 Geographic Locations of Crowdsourcing for Speech2.1.3 Specific Areas of Research2.2 Alternate Solutions2.3 Some Ready-Made Platforms for Crowdsourcing2.4 Making Task Creation Easier2.5 Getting Down to Brass Tacks2.5.1 Hearing and Being Heard Over the Web2.5.2 Prequalification2.5.3 Native Language of the Workers2.5.4 Payment2.5.5 Choice of Platform in the Literature2.5.6 The Complexity of the Task2.6 Quality Control2.6.1 Was that Worker a Bot?2.6.2 Quality Control in the Literature2.7 Judging the Quality of the Literature2.8 Some Quick TipsReferences13 Collecting Speech from Crowds13.1 A Short History of Speech Collection13.1.1 Speech Corpora13.1.2 Spoken Language Systems13.1.3 User-Configured Recording Environments13.2 Technology for Web-based Audio Collection13.2.1 Silverlight13.2.2 Java13.2.3 Flash13.2.4 HTML and JavaScript13.3 Example:WAMI Recorder13.3.1 The JavaScript API13.3.2 Audio Formats13.4 Example: The WAMI Server13.4.1 PHP Script13.4.2 Google App Engine13.4.3 Server Configuration Details13.5 Example: Speech Collection on Amazon Mechanical Turk13.5.1 Server Setup13.5.2 Deploying to Amazon Mechanical Turk13.5.3 The Command Line Interface13.6 Using the Platform Purely for Payment13.7 Advanced Methods of Crowdsourced Audio Collection13.7.1 Collecting Dialogue Interactions13.7.2 Human Computation13.8 Summary13.9 AcknowledgementsReferencesIndex