More Yahoo Pipes details
The tool consists of two major components: an interface, called an editor, where a Pipe is put together; and an execution engine that runs the Pipe instructions. Once a project is saved in the editor, the instructions are saved as a special kind of document on the engine. To run the Pipe, the engine reads the document and then accesses anywhere from dozens to hundreds of Web services--from feeds supplied by Craigslist to geography data on Yahoo Maps. To optimize the response time, says Sadri, the engine parallelizes as much of the execution as possible, breaking up the instructions into chunks that run simultaneously.