Bill Schilit, Google Research
Scanning books, magazines, and newspapers is widespread because people believe a great deal of the world's information still resides off-line. In general, after works are scanned they are indexed for search and processed to add links. In this talk I will describe a new approach to automatically add links by mining repeated passages. This technique connects elements that are semantically rich, so strong relations are made. Moreover, link targets point within rather than to the entire work, facilitating navigation. Our system has been run on a digital library of over 1 million books (Google Book Search), has been used by thousands of people, and has generated the world's largest collection of quotations. I will also present a follow-on project based on the theory that authors copy passages from book to book because these quotations capture an idea particularly well: Jefferson on liberty; Stanton on women’s rights; and Gibson on cyberpunk. These projects suggest that mining quotations for links and ideas are an important mechanism for understanding the knowledge contained in books.
video archived at: http://www.parc.com/cms/get_article.php?id=772
------------
The automatic creation of the idea map is very interesting. These algorithms may be used effectively on the collected papers of luminaries as well as books. Papers leading up to ideas expressed in books may be linked into the idea map and easily found. The luminary themselves could add additional links and meaning.
