Journal
Deflecting Digg
2 February 2006 › 45 comments
Update: Due to the arguments presented in the comments of this article, most importantly the case for accessibility made by Nathan Logan, I have decided to remove my embargo against Digg. My opinion of their community of users as a whole still stands, but I have been persuaded that it is unjust to shut out a majority of people because of misbehavior by a few bad apples.
My Reason
It is with a heavy heart that I have decided to do this. After being on the front page of Digg.com twice in the last two months (one, two), I have learned that these are not the type of visitors I want on my site. Let me prefix this explanation by saying I do not have anything against the creators of Digg.
In fact, I think that the whole idea of social bookmarking is pretty ingenious. Rather, it is the sum total of asinine comments that are left both on their site and mine that has made me decide to deflect incoming referrers back to the Digg front page. I have also removed the 1-click Digg link from this site. In fact, I just got rid of bookmarking links altogether. I still have great respect for the Del.icio.us service, because they started the whole social linking and bookmarking craze. Likewise, I enjoy Newsvine because it fosters a more civil environment, comprised of a much more discerning readership.
I am not alone in this. Other professional web designers / developers have gone through similar experiences. The trouble is, we write articles that appeal to a certain audience, of those who appreciate the nuances of what we are talking about: CSS, Design, JavaScript etc. These articles get linked to, and then become “mainstream,” which is not the intended target demographic.
Dustin Diaz recently spoke about this problem with his article on dealing with negative comments. The flaming he got came as a result of a rather clever article he wrote on how to create an Ajax Style Switcher. He was also heavily linked to for compiling what he considers the Top 10 JavaScript Functions.
Andy Rutledge posted an article recently that gave a light-hearted facelift to the popular search giant Google. I and many others in the design community appreciated the notion, and linked to his article. Again, the boom of visitors attracted so much attention that it had a lot of negative feedback. Due to this type of behavior, he left Stylegala and has no comments enabled on his site.
It seems that Digg is the domain of angry teenagers with gobs of time on their hands, basking in the glow of utter anonymity, all entered in some sort of contest to see who can use the “F” word most often and in the most creative manner. – Andy Rutledge
My Method
Since I’m using Textpattern, I will explain how to go about modifying the default .htaccess file, to include a new rewrite rule tailor fitted to Digg.com. This could of course be applied to any problematic domain name you want. When you open up the file, you will see something that looks like this…
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^(.+) - [PT,L]
RewriteRule ^(.*) index.php
</IfModule>
Before the closing </IfModule> bracket, you will need to add this code…
#Blocking Digg...
RewriteCond %{HTTP_REFERER} digg\.com [NC]
RewriteRule .* - [F]
ErrorDocument 403 '<meta http-equiv=refresh content="0; url=http://digg.com/" />'
Note that the #Blocking Digg part isn’t necessary for this to function, but it helps me remember what lines to add back in when I upgrade Textpattern. So, your final .htaccess file should look like this between the conditionals:
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^(.+) - [PT,L]
RewriteRule ^(.*) index.php
#Blocking Digg...
RewriteCond %{HTTP_REFERER} digg\.com [NC]
RewriteRule .* - [F]
ErrorDocument 403 "<meta http-equiv=refresh content='0; url=http://digg.com/'>"
</IfModule>
That will set up an “infinite” loop, so to speak, redirecting incoming links that come from Digg back to its front page. Hopefully, this will exhaust the short attention span of the types of people who like to leave useless comments. It’s like the old joke about the business card that says on each side: “To find out how to keep an idiot busy for hours, flip this card over.” – Happy deflecting!
Discussion + Dissension
Comments closed after 2 weeks.
#1 Dennis Bullock
I agree Nathan, in fact I wrote a piece about this topic the other day on my site.
#2 Chris Harrison
Can’t say that I blame you. Thankfully, I’ve never had to deal with a Digg-induced stampede on my site… But I’ve seen how (as a Digg user trying to access digg-linked sites) Digg.com traffic can bring a site down. Nice tutorial, btw…
#3 Nathan Logan
Another option would be to disable the ability to comment for those coming from Digg. That way, you still get the readership and aren’t seen as “hostile”, but still avoid the negative/mainstream comments.
There are several ways to do this, and I’m sure you’re clever enough to pick a good one. =)
#4 Nathan Smith
Nate Logan: Hey bro, thanks for the advice. I’d considered the possible implications of blocking out such a huge userbase, but in the end, I think it’s worth it. The kind of people who have meaningful things to say typically use other services in addition to Digg, and will probably find their way to my site regardless. This way, I’m just cutting off trigger-happy link clickers, because these are the types who are primed to leave spam, regardless of topic.
#5 Elliot Swan
Funny you mention this. I recently wrote an article on social bookmarking and “the new search,” then submitted it to digg. I got a fairly nasty comment from one guy…checked up his profile and it turns out he had never dugg a link before, but had left tons of comments. I’m assuming they were all negative.
What do they think they’re achieving?
#6 Nathan Smith
Elliot: That’s pretty hillarious. That guy sounds like a self-appointed critic of all articles, refusing to digg any but commenting on many. As far as what he’s achieving, I’m not sure. People like that just waste perfectly good oxygen.
#7 Dustin Diaz
Wow, sending diggers back to digg. That’s just awesome. I went for the simple solution of just closing entry comments where appropriate.
Nevertheless, I’m glad you’re taking a stance on this. Negative comments can be worse than spam at times.
The digg community just seems childish and it’s better off to shout and bark to look cool than to actually write something positive and be humble about the fact that you might have learned something. These are, in fact, the people that will not get very far in life if they keep up that attitude.
#8 Dustin Diaz
Just a thought…you know, for fun. You should send digg users to del.icio.us :D
#9 Nick Glaser
I’m sorry you feel this way. I was quite interested in your article and found it very interesting. The only way I would have found such an item is via digg or similar site. Obviously it is unfortunate when a negative element influences this kind of situation but speaking for the silent majority. We’re not all jerks (I hope).
#10 Nathan Smith
Dustin: That’s a good idea, and I’d considered sending them on to Delicious, but I don’t want to pollute that userbase. Also, by sending them back to where they came from, it makes it look like something is wrong with Digg. :)
Nick: I’m glad for most of the people that read my articles. Clearly there were more positive +digg’s than there were negative comments. I don’t think you’re a jerk, but I don’t have enough hours in the day to deal with all the people who are.
#11 Andrew Ingram
Assuming getting dugg doesn’t kill your bandwidth allowance maybe a better approach would be a mod for your cms that flags entries from various digg/digg-like sites for easier filtering/moderation.
#12 Nathan Smith
Andrew: Actually, the “digg effect” did knock my site offline for awhile yesterday. To me, I’d rather be able to actually get to my site via CMS / FTP, than be super-popular. Plus, I don’t want to mess with TXP just for that. Personally, I think it’s worth the trade-off to simply block Digg altogether.
#13 Andrew Ingram
There is a reason I said “assuming getting dugg doesn’t kill your bandwidth allowance”, ie if it does my suggestion isn’t relevant.
Personally i’d be hesitant to block out a potentially useful referrer based on the behaviour of a few people. I’d imagine that most people who use digg just read it and don’t comment – these are the people who i’d want to allow access.
Obviously it does depend on your target audience but outright blocking seems like overkill to me, i’m not say my idea would work that well either but this is certainly an area that could do with more discussion
#14 Nathan Smith
Andrew: I see your point, but what it boils down to is that I don’t mind categorically filtering out Digg referrals. I know that seems foolish / crazy to some people, but I’d rather do that than spend all day pruning comments. Technically savvy users will still manage to find their way here regardless.
#15 Collin
I don’t know how tech savvy I am, but it wasn’t that hard to find a way around your misdirection to reach your site. There’s nothing I could think of that should cause me to bounce back to digg, except if the post was linked incorrectly. That lead me to look at the link and, seeing it wasn’t digg, investigate other options. All I had to do was pay attention to the link address – in this case “http://sonspring.com/journal/drawing-a-good-logo” – then drop off the specific page being linked – ”/drawing-a-good-logo” – and enter it on a blank tab. In hindsight I could have probably used the entire link in the blank tab and I still would have reached what I was looking for.
Rather than sending the traffic back to digg, you might want to consider sending them to a ‘404’ page set up off of your main page. Then the visitors are likely to conclude that your site has been knocked out by high traffic rather than something fishy at your end and wander off to do other things.
Anyhow, I’m not one of the trolls that you are seeking to cut off. I really liked what you wrote about logo design and passed the link along to the other graphic designers in my company. Thanks for writing it.
#16 Nathan Smith
Collin: Thanks. The only problem with doing like that is it keeps using up my bandwidth, and allows those visitors from Digg to still reach my site. Plus, if I caused that URL to redirect to a 404 page, other legitimate visitors would not be able to read the article. I still want it to be available, just not via Digg.
#17 Collin
I meant redirect the traffic solely from Digg to a prepared ‘404’ page, not all traffic. Essentially the same way you are now bouncing them back to Digg. And really, how much bandwidth can a text ‘404’ page use up? Or, even better, a plain, image-free page that reads “This client’s monthly bandwidth has been exceeded.”
Or, even better! Use this link for the kickback:
http://www.sonsspring.com/journal/
Nothing of yours is used then, and it won’t inspire anyone from Digg to look any harder.
The way you are handling it now is just going to draw more trolls if the word of what you are doing gets out. Like throwing rocks at a hornet nest.
”... and allows those visitors from Digg to still reach my site.”
Um. That’s how I got here. It wasn’t hard.
#18 Nathan Smith
Ah, I get what you mean. I still prefer this way, sending them back to Digg, where they will quickly be distracted by a slew of other interesting stories. “That’s how I got here. It wasn’t hard.” – Yeah, and at least you’re civil!
#19 Thame
I actually enjoy having my articles on digg and del.icio.us popular lists because it means that my articles are actually helpful.
Most of the negative comments digg users make are directly on the digg page, and I simply ignore those that do make the effort to leave a negative comment on my page.
The fact is that each person that makes a negative comment is matched by two or more positive/constructive comments, and there should be an even bigger gap in terms of actual pos/neg visits.
Also, I don’t really have to worry about bandwidth with Dreamhost. I purposely included large images and downloadable files in my most recent front-page digg to see just how bad it would be and it “only” consumed 8GB out of my 1TB limit :D
#20 Justin Perkins
Nice writeup Nathan, it’s really a bummer digg brings such fools to your site (all sites, I suppose). Guess I’m glad I haven’t written anything considered digg worthy :)
Got some questions on your rewrite rules…why are they at the bottom of the ruleset instead of above your textpattern rules?
Secondly, why the meta refresh? Couldn’t you do a redirect rule back to digg in conjunction with the -F flag to show a forbidden request?
#21 Nathan Smith
Justin: It really doesn’t matter where the lines of code go for the 403, so long as they follow after the rewrite rule is activated. As far as the forced meta refresh, that’s the way to do it for referrers. A redirect would be used if a file had been moved to a new directory, to send people to its new location.
#22 Justin Perkins
After playing around with those rules for little bit, I realize you cannot simultaneously send a 403 response along with a redirect URL. Hence everything you did makes sense :)
> It really doesn’t matter where the lines of code go for the 403
All your other rewrite rules are being run through though, it seemed wasteful to me to run through all your rewrite rules only at the end to check the referrer and send them off. That was until I found that the ErrorDocument definition is at the end for a reason. If it was defined at the top of your .htaccess, then all 403s (not just digg referrers) would use the document…not good.
> A redirect would be used if a file had been moved to a new directory
That’s not the only reason why you’d use a redirect, but yeah I guess it’s better from a technical standpoint to give them the 403 than a 302.
#23 Nathan Logan
I don’t know. Even after reading your responses here, I just can’t come around to agreeing with you. While I understand the problem (somewhat), the solution you implemented just seems excessive. A shotgun works to kill a fly, but so does a fly swatter.
To penalize someone because they choose to use Digg over another social networking software just seems inflamatory and reactionary. My bet is that many people use Digg who don’t use other similar apps (and many who use the other apps don’t use Digg). That being said, I’m 100% sure that there are thoughtful, tech-savvy users who would be coming from Digg who are now essentially prevented from doing so. And these are people who may really want/need access to your articles. They may be your potential future employers or just some up-and-coming high school student who will be the next Dunstan.
And what about accessibility? On a screen reader, I’d be happy to endure a few layout tables if it was either that or being redirected right back to where I came from (and without warning, may I add). Talk about confusing…
Anyway, I hope I wasn’t too harsh here. That wasn’t my goal. It’s just that I think this solution is about the furthest thing from Web/user-friendly, and judging by some of the other comments, I’m not alone there. While it’s a solution, it reminds me more of atomic warfare than sniper shooting.
Then again, it is your blog. And last time I checked, most of us run a sitetatorship (myself included). =)
#24 Nathan Smith
Dang Nate, you dropped the bomb (pun intended). I realize what you’re saying, and perhaps I’ll reconsider my rash decision. I just wanted to “close the barn doors,” so to speak and wait out the storm. This saves both my bandwidth and my sanity, having to prune comments from all the kiddies.
#25 GregD
Very interesting post Nathan. I used to run a website called diggfan.com. If you go to that site now, you’ll see a text only page of my experience being a former “digger”. I no longer visit digg all that much and as far as I’m concerned, it’s losing the quality people that had been attracted to it in the beginning. I’m not sure what they could do to digg to filter out the riff raff, but I’m very tired of it. Thanks for speaking your mind on the subject…
#26 cynric
Hola. I too just came over from digg and am glad to see that it was possible. Great articles and site. I enjoy reading quality code and have picked up some interesting ideas. Thanks.
#27 Alaster
As a tactic to reduce the number of uncivil posters, Nathan’s method seems like it might be good. The article is not really blocked, but digg user would have to do more than just click on a link to get to it. Adding the bit of work to copy and paste the url could weed out some of the people who are just click happy, but doesn’t really prevent someone who wants to read the article from reading it.
#28 David Knighton
“My bet is that many people use Digg who don’t use other similar apps (and many who use the other apps don’t use Digg).”
Judging by how incestuously links get passed between digg, del.ico.us, slashdot, engadget, et al, I would disagree with you there. Other than the community, one of the biggest problems with digg is that half the stuff that makes the front page I already saw at one of the other sites I visit daily.
There’s certainly a lot of cross-pollenation going on.
And on the original subject, I think you’ve come up with a very creative solution, Nathan, and I can’t blame you. Hopefully the digg community will evolve and you will be able to accept traffic from them again, eventually.
#29 Nathan Smith
Thanks for the feedback. David brings up a good point – there are so many other sites that pick up on the same link trends, and even use RSS to aggregate huge lists. So, I’m not worried about alientating incoming visitors. It’s the equivalent of sitting children a different table for dinner. Until they grow up, they are given their own area, so they don’t disrupt the adults.
#30 tom
Why not discreetly hide any comments sections from visitors referred by digg? I’d guess that most of the people who leave wierd comments wouldn’t stick around for long but genuinly interested people would benefit. (I think I fall in to the second category there…)
#31 Nathan Smith
Tom: Because that would require configuring my content management system, and as I’ve said before, it’s well worth it to me to just block all Digg users. I don’t see why people care about this so much. It’s just one website.
#32 Dan
How very ironic that this article has been posted on digg. The poster has evaded your coding by using a shortURL site: (http://makeashorterlink.com/?D1172559C).
I do agree wholeheartedly with Andy Rutledge’s quote, but find that sometimes digg will cover items not yet found elsewhere. I use digg the way I use TV news: never swear by any one source.
#33 Nathan Smith
I tend to think of Digg in the same way as the Enquirer. Sure, it is sort of amusing to skim the headlines while waiting around at the grocery store, but aside from a good laugh, there is no good reason to take them seriously.
#34 Glen C.
This has always been my problem with digg. Some good links, but an awful community.
#35 Nathan Logan
David, while it’s certainly true that some people use multiple community link sites like Digg (hence the cross-posting of links), it’s also certainly true that some people only use Digg (whatever their reasoning may be).
That said, those people who solely use Digg, even if they are well-intentioned, are essentially prevented from coming here. Maybe they’ll get here another way, but there’s a good chance that they won’t.
Honestly, I think the biggest surprise for me in the whole thing was the accessibility issue being disregarded by Nathan, who is someone I usually consider to be exceptionally thorough with that topic.
But all-in-all, not a huge deal. There are just some negatives to going this route, that’s all.
#36 Bryan
When LifeHut was put online and got linked by Lifehacker.com, I got 10,000 visitors in 1 day, the most ever for any of my sites.
however, along with that came people who were VERY opinionated. Really bad!
Many of my articles, though written in good nature and overall had good advice, were completely flamed and were considered awful.
The “bright” visitors saw through some of my article flaws and understood at least what I was getting at…whereas others felt what I wrote was trying to preach gospel and that all those who oppose go to hell.
On one hand, I would love Digg traffic, on another, Andy is right that it’s probably just a bunch of techie teenagers who have nothing other to do then play XBox and spam websites with offensive comments.
Good article.
#37 Bryan
I agree, Digg does have an awful community, and that is how I feel about Metafilter.com as well.
The majority of the comments on their site are negative and it’s pretty depressing reading much of it.
#38 keng
well, maybe it will… however, if they’re really blood-in-the-mouth types they’ll just copy/paste like I did (of course you alluded to the work around in your post) and you back to sq1.
sorry, you had such a bad experience. have you tried blocking IP ranges or requiring registering to post comments?
#39 Nathan Smith
Keng: I considered that, except that blocking by IP range doesn’t do much for dialup users, and it locks out people who happen to be with the same ISP as those that have been blocked. In the end, blocking just a referrer allows everyone through, as long as they know how to get past the redirect.
#40 bbqribs
I’ve found that to be a problem with the internet in general – the ‘younger generation’ that’s getting Internet access. To them, it’s nothing more than a big appliance that lets them chat, have myspace ‘web pages,’ and to download mp3s from p2p apps. They don’t care about anything else on the internet but what’s ‘hip’ and ‘fashionable.’
To put it bluntly, there are just too many stupid people on the internet.
#41 David Martinez
I understand how you have come to this decision, but it leaves me wondering whether establishing this as a widespread practice won’t create a division that becomes too large to cross over. What I mean is that these people obviously need to learn how to behave in society, and locking them out of good targeted content because of the comments they leave doesn’t really teach them anything.
I’m not saying I have an answer for this. On my own site I use the tried and true method of manual comment moderation. Comments never appear on the site until I read them and manually approve them. The times that I have gotten offensive messages I don’t approve the comment and just send a polite response along the lines of “why exactly are you doing this if you know you offend people when you do?” (I do this from a throwaway account that just bounces any mail sent to it), and just never approve their comment or respond any further. A thick skin helps.
I know this method won’t scale, but Comment #11 has a point – maybe a better idea would be to trigger moderation required for people coming from digg. Then you can decide what to do, which would usually be to delete all unless you find something worth approving.
Awesome site by the way. I’m a coder and don’t know the first thing about proper design. I wish I could do half what you have accomplished here. Your site has made it into my must-read list.
God bless.
#42 Nathan Smith
David: I see your point, and have been thinking about what Nate Logan said this raising questions of accessibility. I am considering “reinstating” Digg. Thanks for your kind words about my site. Likewise, I’m in awe of coders!
#43 Chris Harrison
I still support the original decision to deflect digg users, but I digress.
I believe in a free and open internet, but when you have a plethora of moronic users coming from one specific source, it only makes sense that you block traffic from that source. I say this because ‘The Digg Effect’ brings about a lot of problems. With Nathan’s site, it brought it to a crawl… it ate up his site’s bandwidth. I know I’ve clicked on links to several diggs while on digg.com, and the sites wouldn’t pull up – because the increase in traffic knocked em offline.
#44 karl prigge
haven’t you guys tried reddit? very similar to del.icio.us in appearance except that it’s democratic news (like digg) without the 13-yr olds (like digg).
#45 Nathan Smith
Karl: I’ve noticed a few incoming link in my referral logs from Reddit, but haven’t tried it myself. It seems these days, there are almost too many bookmarking services to keep up with! Reddit seems pretty cool though.