{"id":8905,"date":"2006-03-27T06:34:30","date_gmt":"2006-03-27T13:34:30","guid":{"rendered":"http:\/\/hill-kleerup.org\/blog\/wp\/2006\/03\/27\/wist-makeover-thoughts.html"},"modified":"2006-03-27T06:34:30","modified_gmt":"2006-03-27T13:34:30","slug":"wist_makeover_t","status":"publish","type":"post","link":"https:\/\/hill-kleerup.org\/blog\/2006\/03\/27\/wist_makeover_t.html","title":{"rendered":"WIST makeover thoughts"},"content":{"rendered":"<p>Had a eureka moment this morning, and I&#8217;ve done some noodling on the white board and I want to write this down before I forget it (and so that I can let it percolate in the backgrouind and get on with my real work).<\/p>\n<p>I have a quotations page, <a href=\"http:\/\/wist.info\" target=\"_blank\">WIST<\/a> that suffers from a fair amount of neglect.  The problem is, I&#8217;ve wanted to keep things on a database basis, and it&#8217;s been a bitch to move from the Access DB on my PC to the output pages.  I&#8217;ve simplified it a fair amount, but it still is an irksome task (run an export by a letter of the alphabet, do some text manipulation, export it into the appropriate WIST page, view it, clean it up, etc., repeat 20-odd times).  Part of that is the difficulty in doing an HTML-based report out of Access (which really wants you to generate a Windows server DB out of it if you&#8217;re going to do something with it), part of it is my own technical limitations, and part of it is my perfectionism as to the output.<\/p>\n<p>Regardless, right now it&#8217;s so irksomely difficult that I just don&#8217;t do exports often enough.  As a result, the data out there hasn&#8217;t been fixed in a couple of years.  And when people point out where I&#8217;ve made a goof, I either have to manually fix it online or make a big effort to change it in my DB and online or simply leave it be until the next export.  Which is greatly irritating, whichever direction I go.<\/p>\n<p>Okay, epiphany time:  I think I&#8217;ve sort of figured out how to do it via Movable Type.  I might even be able to do it all online, which would be fabulous.<\/p>\n<p><!--more--><\/p>\n<p>Overall design goals:<\/p>\n<ol type=\"A\">\n<li>Easy to maintain.  No double-entry.  No clean-up needed (if I export from another source regularly).\n<li>As an extension of the above, no hacks to MT.  Plug-ins\/extensions, maybe.  But no direct hacking of the MT code.  And no hacking to the database &#8212; I don&#8217;t want to add extra fields to tables.  <\/p>\n<li>Handles structured data that has lots of gaps in it.<\/ol>\n<p>Okay.  The data is basically a 2-3 level hierarchy.  The top level is <strong>author<\/strong>.  Authors have a bunch of individual characteristics &#8212; life dates, mini-biographies, pseudonyms &#8212; that bear some structure, but that structure is sort of optional (more to keep things straight than anything else).<\/p>\n<p>The bottom level is a <strong>quotation <\/strong>itself.  Roughly speaking, a quote has an author, the text itself, and a citation.  It might, optionally, have some keywords for searching purposes.  <\/p>\n<p>The middle level is the citation.  In theory, all quotes have a citation (the name of a book, article, whatever); in practice, many do not (or I can&#8217;t find it).  I worry about citations a lot, and a lot of my work on WIST over the past few years (when I work on it) has been establishing citations for quotes.  Frankly, I&#8217;ve probably done more work on that than anyone else I know.<\/p>\n<p>At any rate, there are enough gaps in the data that actually creating that middle level is not really worth the effort.  I like to group\/sort the quotes by citation, but too many citations would be &#8220;unknown\/attributed&#8221; to be worthwhile.<\/p>\n<p>So we have a two-level hierarchy.  And, coincidentally, that matches the basic two-level hierarchy of MT:  <strong>categories <\/strong>and <strong>entries<\/strong>.<\/p>\n<p>So, consider something like this:<\/p>\n<ol>\n<li>Set up categories for authors.\n<ol type=\"a\">\n<li>One category for each author.\n<li>Use the <em>category description <\/em>for the biographical information.\n<li>Issue:  I currently have over 1,700 authors.  Is there a functional limit on categories?\n<li>Issue: How are names to be sorted?  I have an internal sort order used for the database that is (very roughly) lastname, firstname, but that&#8217;s not necessarily how I might want to present it.  So I might include the desired &#8220;presentation&#8221; name in the category description.  But that implies embedding some basic character formatting in the description block, which would need to be maintained.  I&#8217;m essentially going from twelve fields in my Access DB to one.  Some of those twelve are to handle different cases (born, died, flourished, contemporary), but it&#8217;s still a slightly discomforting flattening of the data.  That argues to maintaining the overall DB separately and populating the blog periodically through the import function.  That&#8217;s sub-optimal process-wise, but may be best functionally.\n<li>Issue:  Searching for biographical info.  Is that necessary?\n<ol type=\"i\">\n<li>Well, it might be for pseudonyms (e.g., if you were looking for Clemens and I wanted you to see that was Mark Twain, that might not be immediately visible in a normal MT-based search, since i would be in the category description.\n<li>Alternately, would I want to have the pseudonyms generated as their own categories (with the description pointing back to the &#8220;real&#8221; name)?  Can I do that programmatically?  And I note that the category will come up empty (no category archive generated) if there&#8217;s not something for that person (i.e., if it&#8217;s just a pointer).  <\/p>\n<p>Unless &#8230; we duplicate the categories for an entry, i.e., each Twain quote appears with categories &#8220;Mark Twain&#8221; and &#8220;Samuel Clemens.&#8221;<\/p>\n<li>Would I want the pseudonyms\/akas to come out as subcategories?  Probably not.<\/p>\n<li>There&#8217;s some verbiage in this field (&#8220;a\/k\/a&#8221;  and &#8220;pseud. for&#8221;) that I would need to clean up if I used the field for extra categories.<\/p>\n<li>Alternately still I could use Google as the search engine, rather than MT.  I&#8217;d rather not, but it would solve some of the issues above.  To be considered.<\/ol>\n<p>\n<\/ol>\n<\/p>\n<li>Set up entries for quotes.<\/p>\n<ol type=\"a\">\n<li>One entry for each quote.\n<li>The entry&#8217;s <em>category <\/em>is the author.  (See above for pseudonym\/a.k.a. issues).\n<li>The <em>entry body <\/em>is the quotation itself.\n<li><em>Keywords <\/em>aren&#8217;t used (currently), but could be.\n<li>The citation info for the quote is stored in the &#8230; hmmmm.\n<ol type=\"i\">\n<li>Do it by sub-category?  That&#8217;s just crazy talk, given the length of the citation info and the number of places it&#8217;s missing.  Plus subcategory handling in MT is not as clean as I&#8217;d like it to be; too many places where it flattens out to all the categories.\n<li>Well, it could be formatted and stored in the <em>extended entry<\/em>.  That&#8217;s easy. But that wouldn&#8217;t allow me to sort the quotes by the citation data.<\/p>\n<li>I could use the <em>entry title <\/em>field for the citation.  <\/p>\n<p>Can you sort a category by entry title?  I think so.  You can sort entries by &#8220;title,&#8221; &#8220;status,&#8221; &#8220;modified_on,&#8221; &#8220;author_id,&#8221; and &#8220;excerpt.&#8221;<\/p>\n<p>Is there a limit on title size?  Need to look that up.<\/p>\n<p>As citation info is refined, the titles for posts would change.  Is that a problem?  <\/p>\n<p>Well, if I want persistent links, I might want to go to an <em>entry ID <\/em>nomenclature for the files (rather than a dirified title name).  (If you have a dirified title filename and you change the title, does it update the post filename?  Or does that happen only on the original build?)<\/p>\n<p>Can title be blank?  No.  And many quotes don&#8217;t have a title.  That may be a showstopper for this.<\/p>\n<p><li>I could use the <em>excerpt <\/em>field for the citation.  That would let me sort as well, without worrying about the title field oddities (but begging the question of what should go in the <em>title <\/em>field).  I think this is the way to go.\n<\/ol>\n<\/p>\n<li>What should go in the <em>title <\/em>field?  Something has to be there.  It doesn&#8217;t have to display.  <\/p>\n<p>I could use some sort of serial number\/quote ID.  But I&#8217;d need to be able to generate that on the fly, and there isn&#8217;t a built-in function for doing that.<\/p>\n<p>If I&#8217;m not displaying it, every entry could have the same (or a made-up\/random) title.  I just hate to waste the field like that.<\/p>\n<p>Pondering.<\/p>\n<li>Fields not being used (maybe):  extended entry, title?, secondary categories?, keywords, basename.<\/ol>\n<li>Special authors:<\/p>\n<ol type=\"a\">\n<li>The &#8220;~sigs&#8221; category is a bunch of one-liners\/slogans\/etc.  My thought would be to have them all in a single entry under the &#8220;~sigs&#8221; author, and not worry about breaking them into individual entries.\n<p><li>The &#8220;~other&#8221; category is for non-author entries, mostly, e.g., &#8220;Spanish proverb&#8221; &#8212; or where the author is so trivial it didn&#8217;t seem worth tracking separately.  I&#8217;d keep these as individual entries under that same category.\n<\/ol>\n<\/p>\n<li>Pages:<\/p>\n<ol type=\"a\">\n<li>Main page (index):  Flat text, explaining the site?  Or the most recent added quotes?  Or the most recent updated quote (using the last updated plug-in)?\n<li>Individual archives:  A single quote.  (Notes above on filenames &#8212; probalby want to use the <em>entry id <\/em>format rather than dirified titles).<\/p>\n<li>Categoriy archives:  Quotes for a given author.<\/p>\n<li>Archive index:  List of authors (categories).  Don&#8217;t really need to maintain date archives?  Can&#8217;t imagine why.<\/ol>\n<li>Comments and trackbacks?  Why not?<\/p>\n<li>QotD:  There are MT plug-ins that will choose a random entry, I believe.  I could use something like that to a quote-of-the-day, which I could then use for my WIST mailing list. Maybe.<\/p>\n<li>Looking at the above, the big question is, could this all be single-source in MT?  I.e., once I have the data into MT, would I need to maintain it separately?<\/p>\n<p>The biggest issue is where I&#8217;m conflating different fields from my WIST DB into a single MT field.  That&#8217;s happening in two cases:<\/p>\n<ol type=\"a\">\n<li>In the author table, I track (besides the author_key internal field):\n<ul>\n<li>author_key (database key field)\n<li>title (Lord, King, etc., stuff that should display sometimes but not be sorted by)\n<li>lastname (usual sort key 1)\n<li>suffix (Jr., III, &#8220;of Hippo&#8221;)\n<li>firstname (usual sort key 2)\n<li>displayname (if the above fields don&#8217;t produce a normal name, e.g., &#8220;Robert Cecil, Lord Salisbury&#8221;)\n<li>aka (pseudonyms and a\/k\/a&#8217;s.  Pen names, alternate translations, maiden names, birth names, pre-title days, etc. Messy RL stuff)\n<li>other-aka (as above, but stuff I don&#8217;t usually display.  Usually pedantic inclusions of middle names, etc.)<\/ul>\n<ul>\n<li>born (not necessarily numeric; may include &#8220;c.&#8221; or &#8220;?&#8221;)\n<li>died (ditto)\n<li>contemporary (T\/F, in case I don&#8217;t know the actual birth\/death dates)\n<li>flourished (again, in case I don&#8217;t know the actual birth\/death dates)<\/ul>\n<ul>\n<li>notes (free-form text, usually &#8220;nationality profession&#8221;)<\/ul>\n<p>Squeezing all of that into a single field is troublesome to me, because not only is some of it actually meant to be hidden, but I see a value in some of the breakouts as they stand.<\/p>\n<li>In the quotation table I track (besides the author_key internal field):<\/p>\n<ul>\n<li>title (italicized title of the book, movie, etc.)\n<li>subtitle (non-italicized chapter, verse, story name, &#8220;speech at location,&#8221; &#8220;also attributed to,&#8221; etc.)\n<li>source (where I found the quote, not usually displayed, only occasionally populated)\n<li>citedate (date, usually just year, of the quote, if known)<\/ul>\n<p>Again, these would all be conflated into a single field, most likely.  That&#8217;s less problematic, though I would lose some formatting standards.<\/ol>\n<\/p>\n<p>As I look at the above, I&#8217;m inclined to keep the DB in Access and simply maintain a &#8220;clear out all blog entries, re-export, re-import&#8221; cycle.  That&#8217;s extra work, which means it&#8217;s less likely to be done.  On the other hand, it keeps some of those data fields separate, and that might be a good thing.<\/p>\n<p>Or is it?  I just feel like I&#8217;m losing data structure (and, thus, information) irrecovably, but short of tweaking the MT table fields (which I will not do), I don&#8217;t see an alternative.<\/p>\n<p>Pondering.\n<\/ol>\n<\/p>\n<p>So, there it is.  Feel free to comment if you are in the least bit interested.  I welcome the input.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Had a eureka moment this morning, and I&#8217;ve done some noodling on the white board and I want to write this down before I forget it (and so that I&#8230;<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_seopress_titles_title":"","_seopress_titles_desc":"","_seopress_robots_index":"","_seopress_robots_follow":"","_seopress_robots_imageindex":"","_seopress_robots_snippet":"","_seopress_robots_primary_cat":"","_seopress_robots_breadcrumbs":"","_seopress_robots_freeze_modified_date":"","_seopress_robots_custom_modified_date":"","_seopress_robots_canonical":"","_seopress_social_fb_title":"","_seopress_social_fb_desc":"","_seopress_social_fb_img":"","_seopress_social_fb_img_attachment_id":0,"_seopress_social_fb_img_width":0,"_seopress_social_fb_img_height":0,"_seopress_social_twitter_title":"","_seopress_social_twitter_desc":"","_seopress_social_twitter_img":"","_seopress_social_twitter_img_attachment_id":0,"_seopress_social_twitter_img_width":0,"_seopress_social_twitter_img_height":0,"_seopress_redirections_value":"","_seopress_redirections_enabled":"","_seopress_redirections_enabled_regex":"","_seopress_redirections_logged_status":"","_seopress_redirections_param":"","_seopress_redirections_type":0,"_seopress_analysis_target_kw":"","_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[33],"tags":[],"class_list":["post-8905","post","type-post","status-publish","format-standard","hentry","category-wist"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":2429,"url":"https:\/\/hill-kleerup.org\/blog\/2002\/06\/26\/rfq.html","url_meta":{"origin":8905,"position":0},"title":"RFQ","author":"***Dave","date":"Wed 26-Jun-02 9:06am","format":false,"excerpt":"I have a quotations database which I call WIST (Wish I'd Said That). I keep it online here. It's a pride and a joy and a pain in the ass...","rel":"","context":"In &quot;Quotations (WIST)&quot;","block_context":{"text":"Quotations (WIST)","link":"https:\/\/hill-kleerup.org\/blog\/category\/blogging\/wist"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4496,"url":"https:\/\/hill-kleerup.org\/blog\/2003\/08\/20\/whats_old_is_ne.html","url_meta":{"origin":8905,"position":1},"title":"What&#8217;s old is new again","author":"***Dave","date":"Wed 20-Aug-03 4:24pm","format":false,"excerpt":"Whew. I so rarely get to actually check off something from my Endless List of Big Projects I'm Working On ... but now I have. Introducing ... the new, improved,...","rel":"","context":"In &quot;Quotations (WIST)&quot;","block_context":{"text":"Quotations (WIST)","link":"https:\/\/hill-kleerup.org\/blog\/category\/blogging\/wist"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":26557,"url":"https:\/\/hill-kleerup.org\/blog\/2012\/02\/21\/quotations-damn-quotations-and-statistics.html","url_meta":{"origin":8905,"position":2},"title":"Quotations, Damn Quotations, and Statistics","author":"***Dave","date":"Tue 21-Feb-12 3:00pm","format":false,"excerpt":"My annual checkpoint for my \"WIST\" (Wish I'd Said That) quotations site (http:\/\/wist.info). It's ... a hobby. #ddtb Embedded Link ~~Admin - Doing the Numbers: 2\/2012 | WIST Quotations Another good year of WIST since I last ran these (back last March). Let's look at the historical count:Another good, steady\u2026","rel":"","context":"In &quot;~PlusPosts&quot;","block_context":{"text":"~PlusPosts","link":"https:\/\/hill-kleerup.org\/blog\/category\/blogging\/plusposts"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":11431,"url":"https:\/\/hill-kleerup.org\/blog\/2007\/08\/29\/wist_report.html","url_meta":{"origin":8905,"position":3},"title":"WIST report","author":"***Dave","date":"Wed 29-Aug-07 8:57pm","format":false,"excerpt":"It's been a bit over a month since I rebooted the WIST (my quotations database) site, and it's going well -- visits are up decently, and I've been getting...","rel":"","context":"In &quot;Quotations (WIST)&quot;","block_context":{"text":"Quotations (WIST)","link":"https:\/\/hill-kleerup.org\/blog\/category\/blogging\/wist"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":20516,"url":"https:\/\/hill-kleerup.org\/blog\/2011\/03\/22\/annualish-wist-tabulation.html","url_meta":{"origin":8905,"position":4},"title":"Annualish WIST tabulation","author":"***Dave","date":"Tue 22-Mar-11 2:20pm","format":false,"excerpt":"So in my copious free time, I run a website for collecting quotations. Or, rather, a website where I indulge that hobby of mine -- WIST (Wish I'd Said That).\u00a0 I just did my more-or-less annual review of numbers and what quotes are most popular ... and it's remarkable to\u2026","rel":"","context":"In &quot;Quotations (WIST)&quot;","block_context":{"text":"Quotations (WIST)","link":"https:\/\/hill-kleerup.org\/blog\/category\/blogging\/wist"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":13393,"url":"https:\/\/hill-kleerup.org\/blog\/2009\/01\/01\/quotes_by_the_numbers.html","url_meta":{"origin":8905,"position":5},"title":"Quotes by the numbers","author":"***Dave","date":"Thu 1-Jan-09 11:20pm","format":false,"excerpt":"I went and ran the numbers on my quotations database, WIST, yesterday, the first time since February 2007.\u00a0 I'm currently at 6600 quotations, vs. the 5100 I was at back...","rel":"","context":"In &quot;Quotations (WIST)&quot;","block_context":{"text":"Quotations (WIST)","link":"https:\/\/hill-kleerup.org\/blog\/category\/blogging\/wist"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/hill-kleerup.org\/blog\/wp-json\/wp\/v2\/posts\/8905","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hill-kleerup.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hill-kleerup.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hill-kleerup.org\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/hill-kleerup.org\/blog\/wp-json\/wp\/v2\/comments?post=8905"}],"version-history":[{"count":0,"href":"https:\/\/hill-kleerup.org\/blog\/wp-json\/wp\/v2\/posts\/8905\/revisions"}],"wp:attachment":[{"href":"https:\/\/hill-kleerup.org\/blog\/wp-json\/wp\/v2\/media?parent=8905"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hill-kleerup.org\/blog\/wp-json\/wp\/v2\/categories?post=8905"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hill-kleerup.org\/blog\/wp-json\/wp\/v2\/tags?post=8905"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}