Category: Blogging – Technical

Well, that little digression turned into a big time sink.

A few weeks back I converted my WIST quotations site into a static published site, rather than dynamic. That took a chunk of disk space, but the performance for calling up individual pages (and having them index) should save time.

So, today it I was in the mt-config.cgi file (to turn off the autosave function), when I realized it had been three weeks since I renamed my comment and trackback scripts. I’ve found that’s a moderately helpful way of foiling certain spammers.

I was about to do that, when it suddenly occurred to me that, unlike before, not all of my individual entry pages in my various blogs were still dynamic. WIST’s pages are static (actual HTML files generated at creation, vs. dynamic pages generated from the database on the fly). If I changed the names of the comment and trackback CGI files in the configuration, I would have to rebuild all my pages. Which, last time I did it, was a multi-hour task (that may have been in part because it was the first time I’d done it; I haven’t timed it again lately).

Ugh.

So instead, I needed a way for the script names to be dynamic when a given page is called, but the rest of the page to be static the rest of the time. Here’s what I did.

1. Create two new index templates, one for each script name. 

I figured I could use Server Side Includes (SSI) as the dynamic source of the CGI script names. You can create SSIs from MT without too much trouble (as the output file associated with a template), but to have them “built” with the value they need, so they need to be done as Index Templates (vs Template Modules), flagged to rebuild with each rebuild.

So I created a “dynamic comment script” index module that creates dynamic_comment_script.inc, and has as a single line:

<$MTCommentScript$>

That tag will return the of the comment script.  Then I did the same trackbacks.(“<$MTTrackbackScript$>” etc.)

So now whenever a rebuild happens, those two .inc files will have the name of the script (e.g., “xyztrackback.cgi”) in them. And when I change the name of the two scripts in my mt-config.cgi file, instead of rebuilding all the individual entries in WIST, I just have to do an index rebuild (which takes just a minute). And if I forget, it will still update the next time I add a new entry.

Note that SSI is not available on all hosts. It is on mine, though.

2. Change the comment form. 

Now to change the reference to the comment script. In the comment code, there’s a form call that starts something like:

<form method=”post” action=”<$MTCGIPath$><$MTCommentScript$>”

That’s the part that needs to be fixed. And, fortunately, it’s simple.

<form method=”post” action=”<$MTCGIPath$><!–#include virtual=”/dynamic_comment_script.inc”–>”

That’s an SSI call there at the end. It’s calling the contents of that .inc file I created in step 1, literally sucking it in at the time the page is loaded. Thus, the page is static (on file), but that particular piece gets pulled in dynamically. And recall that .inc file contains the name of the comment script, as most recently generated (even if that’s after when the entry’s static file was generated).

3. Change the Trackback text. 

I still have the default trackback address text at the bottom of the individual archive page, in case someone’s doing a manual ping that isn’t doing an auto-discover on the file. That line usually looks like:

TrackBack URL for this entry: <$MTEntryTrackbackLink$>

Instead, I do the same trick as above:

TrackBack URL for this entry: <$MTCGIPath$><!–#include virtual=”/dynamic_trackback_script.inc”–>/<$MTEntryTrackbackID$>

MT has all sorts of tags for this sort of stuff, so it was easy enough to (by looking at what was actually generated) find the surrounding pieces and substitute in the SSI of the TB script in the middle.

4. Change the Trackback autodiscovery code. 

A lot of blogging software can autodiscover trackback addresses for a file through special RDF tags embedded in it. So MT has a simple tag to generate the RDF tags:

<$MTEntryTrackbackData$>

That actually creates a 14-line set of tags and info for the trackback discovery process. Unfortunately, part of that info is the location of the trackback script (so that the autodiscovering system can generate a trackback entry).

Fortunately, though long, the format of those tags is pretty straightforward and the content is reproducable. So in my Individual Archive template, in lieu of the one line above, I now have:

<rdf:RDF xmlns:rdf=”http://www.w3.org/1999/02/22-rdf-syntax-ns#”
        xmlns:trackback=”http://madskills.com/public/xml/rss/module/trackback/”
        xmlns:dc=”http://purl.org/dc/elements/1.1/”>
<rdf:Description
   rdf:about=”<$MTEntryLink$>”
   trackback:ping=”<$MTCGIPath$><!–#include virtual=”/dynamic_trackback_script.inc”–>/<$MTEntryTrackbackID$>”
   dc:title=”<$MTEntryTitle$>”
   dc:identifier=”<$MTEntryLink$>”
   dc:subject=”<$MTEntryCategory$>”
   dc:description=”<$MTEntryExcerpt$>”
   dc:creator=”<$MTEntryAuthor$>”
   dc:date=”<$MTEntryDate format=”%Y-%m-%dT%H:%M:%S-07:00″ />
</rdf:RDF>

All the stuff in the first block is literal info as generated currently by MT. The next stuff is all use of MT tags — including, note, the SSI to get the current trackback script info into place.

The only thing I didn’t bother to look up or figure out how to do is on the last line. The “-07:00” is the GMT time zone offset, in this case Mountain Time. I don’t know if MT has a tag to do it, and I really didn’t feel the need to look it up (since I don’t plan to permanently move out of my time zone any time soon).

5. Kick back and relax. 

And that’s it. With those steps, I now can change my comment and trackback script names, do a simple index rebuild on WIST (or even — given that the trackback and comment traffic isn’t all that heavy — let it rebuild itself when I add more quotes each weekday), and the changed CGI script will be present in all of my entries without having to do a full rebuild.

Of course, you might say, I could simplify things by eliminating trackbacks — but I’m stubborn about this, as I think the TB concept is delightful, and use it a lot for internal cross-references if nothing else. I hate to let the spammers “win” on that one. Ditto for comments — the WIST site would not lose a lot by losing comments — but I’d be irked and saddened. So, to me, it’s worth the effort to have done this.

And to have shared the wealth with anyone else who’s looking for something similar.

An extension of my post from this morning. As I read various objections to the nofollow tag, aside from the “It won’t actually instantly end all comment spamming” (which nobody has claimed it would), the biggest cavil is, “It hurts commenters.”

This infliction of harm seems to be of two types.

1. It treats all commenters as criminals.

   I suppose that’s true, although another way of looking at it is that it distinguishes between the original poster/site owner and others visiting (whether dear, bosom friends or evil spammers). I suspect that there will be some way of whitelisting or allowing registered users to comment without having nofollow in their links, but …

   But, even if so, so what? When we put moderation onto comments, we do the same tarring with a broad brush. When we put registration onto them, ditto. When we make folks type in Captchas and other such hoops, likewise. When we keep our MT password a secret, so that folks can only post comments, not actual blog entries, we’re treating others as untrusted.

   Deal with it.

2. It robs commenters of something of value.

   The idea here is that people who make comments should get some sort of distinct reward, and the coin of the realm in this case is PageRank for their link targets.

   This is wrong, or beside the point, in so many ways.

   First off, if folks are commenting for PR, then screw ’em. Or, rather, I really don’t care whether they get that PR or not, so it’s not something I’m going to worry about.

   If they are conmenting to make a contribution to the discussion, then nothing has been lost by anyone. They’ve made their contribution, their content (and name and URL) are indexed, even if not PRed or spidered, and on display in front of God and everyone. All’s right with the world.

   I mean, already in MT (and this was widely applauded when it happens), the “Comment Author URL” field is redirected to keep spammers from using that field (and thus keep PR from accruing). If that’s not an evil thing, why is the nofollow tag (which effectively does the same thing) so bad?

   I may have a warped view of this, in that probably three quarters of my commenters don’t actually have a web site, and only a tiny fraction of my commenters include links. And of those links, only a small fraction as well are links back to their own pages. So this isn’t something that’s going to “hurt” my commenters.

   Trackback is slightly different, perhaps. Folks who trackback to articles here are, in theory, being similarly hurt because they didn’t get any PR for doing so. But, again, it’s hard to say that’s a distinct harm, unless that was your point in tracking back. The link is still there. I’m still likely to go out and look at the post in question. The poster has gotten something of value — my writing to refer to or be inspired by (hence the trackback). That their page should somehow be enrichened by their pointing to me seems a bit goofy.

   I’ll confess that I’ve a hankering for PageRank for myself. When I see that my front page is PR:5, I go, “Ooooooh, nifty.” I feel the same way about hit counts, or my rank in the TLB Ecosystem. But that’s such an indirect and passionless bit of whuffie that it’s insignificant next to folks actually caring to comment here, or trackback to my posts, or things like that. Those sorts of strokes aren’t dependent on PageRank, and so neither is my ego. Really.

   In other words, any “penalizing” I accrue by my comments on other pages not getting PageRank for my own blog is so trivial to my mind as to be non-existent. I find it hard, then, to empathize with the position that others find it intolerable.

Of course, none of this is either automatic nor unavoidable. If you’ve licked the comment spam problem on your site, implementing nofollow doesn’t really do anything for you, and thus can be avoided. But if stuff still gets through now and then, or you have a life beyond slapping down spammers as they hit, then I think it’s a fine long-term investment.

UPDATE: Anil Dash discusses how design decisions can have implications that could never be imagined — taking, as an example PageRank.

PageRank, when created, didn’t assume that content on a web page, especially links, would be generated by someone other than the publisher of that page. PageRank was not based on the assumption that the rankings would have monetary value. And PageRank is based on the assumption that site editors choose their content, particularly their links, based primarily on merit.

Because those assumptions no longer pertain in a blogging/wiki world, we end up with effects that were not predicted. How this applies to nofollow is discussed in further detail. A good read.

A bit of a backlash this morning on the announcement of the new rel=”nofollow” tag that Google et al. are going to be implementing. Most of the critiques are referenced in this Register article.

While acknowledging that the new nofollow tag will not do anything — immediately — to stop comment spam, I think most of the criticisms are off-base in one way or another.

1. The Internet is becoming balkanized, where people only accept content from people they know.

   This may come as a shock to some, but the Net is not a big wiki. Most web content is blocked against content from folks other than the creator. That blogs — and wikis — have the capacity to allow people other than a site owner/administrator to contribute is fairly amazing, and putting some restrictions on that is neither novel nor alarming.

   The “democratic nature of the web” doesn’t mean that everyone gets to post whatever they want on my site. It means that anyone can open their own site, and the web will “vote” on it by who links to it or visits it (and Google will index it regardless).

   I’ll say it again: comments are not the keys to blog content. Blog entries are. Of the blogs I visit, very few comments have links in them to go elsewhere — and, frankly, since those comments will still work, the damage to the “web” seems minimal.

   Now, if someone put the nofollow tag in their blog entries (which (a) nobody is proposing, and (b) the MT plugin doesn’t do), that would be a problem. But why would anyone actually do that, since the blog entry content is already protected? Again, this is not “invalidating” all links, just the Googling of those put into comments. Huge difference.

2. The nofollow tag is “effectively declaring PageRank dead for weblogs.”

   Huh? I don’t know about most blogs, but for mine, there are rarely links in comments (except from spammers) and rarely is it key to the actual post content. Comments in a blog are not really the same as discussions on a discussion board, in a variety of ways. If links coming out of comments are no longer considered in PageRanks or somehow indexed in Google, I don’t think that’s a huge problem.

   And if there’s a really key link that I think, from a comment, should be highlighted and PageRanked and all that, I can (as the original post owner) put it up in the actual post itself as an “UPDATE.” (Conversely, if I want to reference a site that I think is particularly despicable, I can now do so as a link, manually, without worrying that I’m boosting their Google ranking. That’s not necessarily a bad thing.)

   Now, what this does mean is that if I’ve written about subject X, and I comment on someone’s blog (that’s implemented this solution) and say, “I’ve written in more detail about this at [link back to my blog entry],” that cross-link won’t get indexed, my blog page won’t get a PageRank boost, and a search in Google for pages that mention my pages won’t see it. Boo-hoo. Folks can still click on the link. And since I wasn’t sobbing that such a comment didn’t generate a Trackback (in MT, at least), I don’t see why I should be sobbing that it doesn’t boost my PageRank — unless PageRank is what I’m looking for.

3. This does nothing to stop comment spam, but other things like Captcha does.

   There are two ways to disincent behavior. One is to make the behavior more difficult or painful or costly to perform, and the other is to make the payoff for the behavior less attractive. Captchas and blacklists and the like are part of the former, creating armor and fortifications against folks being able to willy-nilly post content. The nofollow tag takes the latter approach, making the reward for actually getting comment spam posted that much less. The two approaches are complimentary, not swappable.

   I’ve toyed with using Captchas here before, but have refrained for two reasons. The first (and, admittedly, minor) one is that Captchas are unfriendly to those with visual handicaps. The second is that, for determined comment spammers, there are ways to get around them, to essentially recruit humans to solve them for the spammers (either through pittance payments to third-worlders or by tying them into pr0n sites where someone resolves an intercepted Captcha to get in to see something, which then lets the site owner get in to where the Captcha was originally formed). I don’t know how common this bypassing is, but it’s at least conceivably possible, and as long as there is a reward for going to the effort, the effort will be gone to. Hence the long-term potential for the nofollow tag.

4. Spammers will just go to blogs that aren’t using the nofollow tag.

   Ultimately, yes, though it assumes that spammers are really looking at the sites they’re going to (and my experience is that they don’t, much). But if they do, that’s still a victory for me, because it means they’ll leave my site alone.

   The same, though, can be said for any defense, whether it’s blacklisting or moderation or ID verification or whatever. Burglars will hit the obvious targets on a street. Virus writers depend on the folks who have no AV on their system and never download security patches. That’s a separate problem (how to encourage updates, what to do about abandoned sites), but it doesn’t invalidate what the nofollow tag can do in this case.

The Register article, aside from taking a snarky attitude toward blogs in general, is missing the point. This approach seems to me to be a good, long-term way to disincent comment spamming, with minimal effect on the blogs or the Net itself. I’m not sure where the problem is with that.