{"id":281,"date":"2009-01-30T21:57:58","date_gmt":"2009-01-30T21:57:58","guid":{"rendered":"http:\/\/opendna.com\/notebook\/?p=240"},"modified":"2022-11-08T11:08:47","modified_gmt":"2022-11-08T11:08:47","slug":"classic-shell-scripting-commands-that-unlock-the-power-of-unix","status":"publish","type":"post","link":"https:\/\/opendna.com\/blog\/2009\/01\/30\/classic-shell-scripting-commands-that-unlock-the-power-of-unix\/","title":{"rendered":"Classic Shell Scripting: the history of wf.sh"},"content":{"rendered":"<p>wf.sh is a frequently used in introductory CompSci courses to get students to rethink how they imagine data. The classic puzzle is expressed as:<\/p>\n<blockquote><p>&#8220;Given a text file and an integer K, you are to print the K most common words in the file (and the number of their occurrences) in decreasing frequency&#8221; (Bentley &amp; Knuth, May 1986).<\/p><\/blockquote>\n<p>McIlroy&#8217;s solution (June 1986) forms the core of my <a href=\"http:\/\/opendna.com\/blog\/?tag=catastrophic-frequency\">Catastrophic Frequency<\/a> script, which can process an infinitely large corpus at a rate of approximately 100 megabytes per minute (on a single 2Ghz processor). Below the fold: the sources.<\/p>\n<ul>\n<li><a href=\"http:\/\/books.google.ca\/books?id=_mbgnxL-QvoC&amp;dq=%22word+frequency%22+bash+ACM&amp;source=gbs_summary_s&amp;cad=0\">Classic Shell Scripting: Hidden Commands that Unlock the Power of Unix<\/a>, by Arnold Robbins, Nelson H. F. Beebe &amp; Nelson H. F. Beebe. Published by O&#8217;Reilly, 2005. p. 102<br \/>\n<em>Wherein Robins &amp; Beebe tell the history of wf.sh.<\/em><\/li>\n<li><a href=\"http:\/\/portal.acm.org.proxy.lib.sfu.ca\/citation.cfm?id=315644\">Programming pearls: literate programming<\/a>, by Jon Bentley &amp; Don Knuth. Communications of the ACM archive, Volume 29, Issue 5 (May 1986)\u00a0 p.384 &#8211; 369<em><br \/>\nWherein Bentley presents his word counting challenge to Knuth.<\/em><\/li>\n<li><a href=\"http:\/\/portal.acm.org\/citation.cfm?id=5948.315654\">Communications: Volume 29, Issue 6 , Programming pearls: a literate &#8230;<\/a>, by Jon Bentley, Don Knuth &amp; Doug McIlroy. Communications of the ACM archive. Volume 29, Issue 6 (June 1986). p.471 &#8211; 483<br \/>\n<em>Wherein Doug McIlroy presents wf.sh in response to a challenge by Bentley to Knuth.<\/em><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>A short bibliography of the famous wf.sh from 1986: <br \/><em>&#8220;Given a text file and an integer K, you are to print the K most common words in the file (and the number of their occurrences) in decreasing frequency.&#8221; <\/em><\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"webmentions_disabled_pings":false,"webmentions_disabled":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":3,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[19],"tags":[43,42,44,45,102],"class_list":["post-281","post","type-post","status-publish","format-standard","hentry","category-under-construction","tag-programming","tag-scripting","tag-software","tag-unix","tag-word-frequency"],"_links":{"self":[{"href":"https:\/\/opendna.com\/blog\/wp-json\/wp\/v2\/posts\/281","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/opendna.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/opendna.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/opendna.com\/blog\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/opendna.com\/blog\/wp-json\/wp\/v2\/comments?post=281"}],"version-history":[{"count":1,"href":"https:\/\/opendna.com\/blog\/wp-json\/wp\/v2\/posts\/281\/revisions"}],"predecessor-version":[{"id":1842,"href":"https:\/\/opendna.com\/blog\/wp-json\/wp\/v2\/posts\/281\/revisions\/1842"}],"wp:attachment":[{"href":"https:\/\/opendna.com\/blog\/wp-json\/wp\/v2\/media?parent=281"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/opendna.com\/blog\/wp-json\/wp\/v2\/categories?post=281"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/opendna.com\/blog\/wp-json\/wp\/v2\/tags?post=281"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}