The Internet - Intro Notes - Back to Index



This entry is not about Usenet, it is about Google Groups. The two things are somewhat related because you can access Usenet through the Google Groups interface. Anyway, I just read a post by John Resig in which he explains why he has finally given up on Google Groups, see Google Groups is Dead.


A posting on comp.sys.mac.system quoted from a Google Groups page about comp.sys.mac.system which includes two top ten lists; top posters this month and to posters all time (no, I wasn't on either list). The page also includes a monthly tally of the number of articles posted and an indication that it is a high activity group, with 80 recent authors. I had seen some of that before but don't remember seeing the top ten lists - is it a new thing?

In the past I have compiled a top ten list of posters by manually downloading the overview data from my news server and processing that with grep and sort and other command line wizzardry, so I would have thought that I would have checked that it wasn't already available. Actually, I also compiled some stats on which other groups things were crossposted to, which is not part of the Google Groups "about" page.


Last year I found the alt.atheism TwitList on the EAC NUDIS website. Checking the link just now I found that the site is down, and the domain has apparently expired. Searching for another copy of the TwitList I came across what seems to be the "official" website of alt.atheism, which is apparently being rewritten, and does not (at least currently) include a copy of the TwitList.


When I first set up an slrn scorefile I was killing some articles before checking whether they were followups to my own articles. At some point I modified it to unconditionally score up such followups because I always want to see them. Today I made another slight modification to my slrn setup.

On a whim I decided to change my posting_host from to and updated my my scorefile appropriately. For the record, here is my current slrn scorefile.


One of my favourite quotes about Usenet is attributed to Gene Spafford:

Usenet is like a herd of performing elephants with diarrhea -- massive, difficult to redirect, awe-inspiring, entertaining, and a source of mind-boggling amounts of excrement when you least expect it


I have just been looking at the XHDR, XOVER and LIST OVERVIEW.FMT commands which are all common NNTP extensions. As of 2003 there was limited implementation of LIST OVERVIEW.FMT in Usenet clients.


I only read a few Usenet groups, all of which are unmoderated. One in particular (comp.sys.mac.system) suffers from rampant trolling and would be a lot more useful if there was a good moderator in charge. However, that is unlikely to happen, and who would volunteer anyway? Boing Boing has an interesting moderation policy.


EAC NUDIS (The Evil Atheist Conspiracy - NNTP / Usenet Distributed Intelligence System) is is describes as a "kind of deja-news / Google usenet archive, but with a much more flexible search options than Google provides". I found it while searching for the Alt.Atheism TwitList.


Another interesting Usenet "personality" is Mike Corley - so interesting that he is the subject of a modern opera called The Corley Conspiracy which was staged in September in the Purcell Room in London's Southbank Centre. From the press release:

Under threat from those in power, Corley is the victim of "interactive watching": his every move is being monitored and "they" terrorise him through radio and television broadcasts.


Although I have "killfiled" anything posted to Usenet from Google Groups I do actually use Google Groups myself - though not for posting to Usenet of course! The thing is, Google "owns" the Usenet archive and access is via the Google Groups interface. The searching interface is not actually all that bad.

In a recent entry on this page I explained why I reconfigured slrn to generate Message-ID rather than letting Individual.NET do it and that I changed my From at the same time. Previously I was using "Ian Gregory <foo@bar.invalid>" where the email address was deliberately chosen such that it would be clearly recognised as invalid to both humans and software, as per RFC 2606. Although there is some confusion over the issue, this is allowed by Individual.NET. Digressing slightly, the reason I am a bit paranoid about using any real email address in my From header on Usenet is that while working as a sysadmin for the University of Hertfordshire I got hit hard by the SWEN virus. I was one of very few people at the Uni who were active on Usenet and one day I arrived at work to find that the email system was on its knees. This was a system handling email for some 30,000 users and it collapsed under the weight of virus-laden emails being sent to a single account - mine! The virus had harvested my address from my Usenet postings (SWEN is NEWS spelled backwards). Anyway, since then I have always used an invalid email address. I know this breaks the "reply" functionality in news clients but if someone really wants to send a personal message to me they need only go to my website address (given in my .sig) and use my contact form.

Where was I? Right, so until recently I was using foo@bar.invalid but I was aware that other people may well be using the same email address (see metasyntactic variable if you don't understand why). I knew it was unlikely that anyone else using the real name "Ian Gregory" would be using foo@bar.invalid but I thought it might prove useful to be able to search for my posts based only on email address, so I changed it to foo@STRING.invalid where STRING is a constant string of 16 randomly chosen lower case characters. It is exceptionally unlikely that anyone else would choose the same email address by chance (less than 1 in 26^16) but of course someone could deliberately use it if they wanted to.

This brings me back to Google Groups. It recently came to my attention that it groups posts by author based solely on the email address extracted from the From header. Each "author" gets their own profile page at Google Groups, so here for example is the profile of foo@bar.invalid. Note that foo@bar.invalid has (currently) posted 6,974 messages in 400 groups, with 30 ratings and an average rating of three stars (out of a possible five). However, no more than about 1,070 of those messages were actually posted by me.

On September 1st I changed my slrn configuration and since then all my posts have been grouped together by Google under my new profile. Barring deliberate spoofing this profile will only ever list messages that were actually posted by me.


Here is the Official FAQ for the alt.usenet.kooks kook awards and offices. I like the answer to question 24. "To whom should I complain if I think I have been treated unfairly?":

<Darth Vader>
You think you have been treated unfairly?
</Darth Vader>


Usenet has sucked up a lot of my time since my last entry here. Not so much reading and posting (though I have been doing plenty of that in comp.sys.mac.system) but in learning some more features of slrn and delving into the Net News Transfer Protocol (NNTP).

So, to start with slrn. I have specified a scorefile in my .slrnrc and started populating it with some scoring rules. I have a few rules which give posts a score of -9999, which effectively kills them; I don't see them when I go to a group and so don't waste my time on them. Firstly I kill anything with the string "googlegroups" in Message-ID. This may seem a bit drastic but is in line with UIP advice. Next I kill anything that is cross-posted, which I do by checking for the presence of two or more colons in Xref. Again this might seem a bit drastic and it probably is - I intend to relax this rule to allow posts theat are cross-posted to a maximum of four groups - any more than that indicates that the poster is either ignorant of netiquette or is simply a troll or butthead. Finally I kill anything where Subject starts with "Re:" (case insensitive) but there are no References - this is just broken and I don't want to see such posts.

Then I have some rules which don't set an absolute score but just increase or decrease the score by a certain amount. I increase the score by 500 if the post is either by me or is a direct followup to a post by me. I do this by looking for a particular string at the end of Message-ID or References. To achieve this I had to switch to generating my own Message-ID rather than allowing Individual.NET to do it for me. When Individual.NET was generating my Message-ID it was of the form, and given such a Message-ID it was not possible to determine whether it was posted by me as opposed to any other Individual.NET user. I fixed this by adding "posting_host" and "set generate_message_id 1" to my .slrnrc file. I could have used my own domain as the posting_host but since I post everything through Individual.NET I decided to use that. The numerical ID is the one which identifies my own Individual.NET account, so the posting_host uniquely identifies me. A big added benefit of this is that I can now see my Message-ID in the copies of my own posts that are saved by slrn in my saved_posts folder. While I was at it I decided to change my From as well - more on this later.

So that deals with scoring up my own posts. I am also scoring down posts (by 100) based on certain strings in From and Subject, but I won't go into that. So what is the benefit of all this? Well apart from not having to waste time looking at (or being tempted to reply to) the posts that get killed, there is a big advantage gained by scoring up followups to my own posts - they get tagged as important by slrn and appear at the top of the header display, so I no longer miss (or waste time searching for) them. And this works with threaded header display - threads are flagged as important if they contain important messages - even if the first unread message in the thread is not flagged as important. I can also see at a glance in the header display which messages have been marked down. Finally, while reading a message I can hit "v" to see which (if any) scoring rules were satisfied.

One more thing about slrn before I move onto NNTP. I just learned about hitting "U" while reading a message to get a list of URLs in the message. I can then select any URL to open it in a browser. This relies on setting a value for non-Xbrowser in my .slrnrc file, and I have set it simply to "open", which on my system opens the URL in a new tab in Safari (and starts Safari if it is not already running).

OK, so now for NNTP. I was playing around with this partly because I wanted to know what is included in the overview data on the Individual.NET server. The reason is that when you are constructing a scorefile it is expensive to base rules on headers that are not in the overview data (because slrn then has to download the message to workout the score). I realised that I can connect to the server using "telnet 119". After authenticating ("authinfo simple myusername mypassword") I can then just issue the "list overview.fmt" command to see the order of fields in the overview database, which for Individual.NET is:


I can also use the "group" command to go to a particular group and see the smallest and largest article numbers held on the server, and then use the "xover" command to obtain the overview data for all the articles in the group. I did this for comp.sys.mac.system and did some simple analysis on the data to find out things like the number of articles that were crossposted to other groups, and which groups they were crossposted to. I posted the results of the analysis in


"The Usenet Improvement Project is an attempt to make Usenet participation a better experience for those who are clued as to what the Usenet medium is and how to use it". The main recommendation of the UIP is to block all posts made from Google Groups. I use Google Groups to search the Usenet archive and to read and post to a few Google Groups. But I always use a proper news client (slrn) to read and post to Usenet (using an account at Individual.NET). I still haven't tried setting up a kill file with slrn, but when I do I may well following the UIP recommendation and use it to block posts from Google Groups.

The UIP originated on Blinkynet, which also has lots more stuff about Usenet in the Computing/Internet section.


The NetNews team at Individual.NET have disabled unchecked execution of cancels and supercedes on their server and have implemented something called "Cancel Lock" which looks like what is described in an old draft IETF document.


If you you have spent any appreciable ammount of time on Usenet you are likely to have come across people who seem to exhibit quite remarkable degrees of paranoia. People will often respond with references to a protective device known as a "tinfoil hat". In these situations it might be appropriate to point out one of the wonderful eclectech Flash presentations, The Tinfoil Hat Song.


Another interesting character I have come across on Usenet who has the distinction of having at least one extensive FAQ published about him is a certain Steve Winter.


I found some good advice on How to Deal with Trolls which was posted last year by David Emme, who apparently found it on an mailing list.


For reference, here is Richard Kettlewell's guide to Quoting Style.


I just learned about the excellent News Propagation Search Engine which has a very informative FAQ. I decided to test the engine by searching for a group called alt.timothy.sutter to see how many servers carry it. Since it is clearly a vanity group I assumed that it would not be widely propagated. but of the fifteen servers searched by the engine only one ( returned "not carried"!

The results page also gives a link to download the archive of control message history for the group (if it exists) by anonymous ftp. I did this for alt.timothy.sutter and found that he "created" the group in early 1997 with a sequence of three newgroup control messages sent from a server. Someone promptly sent an rmgroup citing "No discussion in alt.config" and "Vanity group". Undeterred, Timothy sent another newgroup the following day, but that was closely followed by five more rmgroups, and at that point he appeared to give up. All the messages in the archive were sent over a period of seven days. So basically Timothy failed to create his group and yet, nearly ten years later it seems to be fairly widely propagated. I have yet to fully understand quite why this should be the case.


I consider myself to be a comp.sys.mac.system regular, and as such am familiar with the off topic posts of "John Wolf". Another c.s.m.s regular, Mike Rosenberg, has compiled a list of the many identities of John Wolf.


I have just had a perfectly rational discussion with "Alan Connor" on comp.mail.mutt, which is surprising because I am lead to believe that he is the Alan Connor referred to by The FGA on Alan Connor.


I recently tried to respond to an article in alt.discordia and ran into problems. It took me a while to figure out what was going on but I got there eventually. First of all, there is an slrn variable called netiquette_warnings which is non-zero in my setup. This results in warnings when crossposting to more than four groups or without setting a "Followup-To:" header. Now this is quite reasonable, slrn is just trying to comply with The Good Net-Keeping Seal of Approval GNKSA and it is easy to ignore the warnings and force the post.

So I forced the post and it failed anyway! I soon realised it was being rejected by News.Individual.NET (NIN) and by running slrn with the debug option I discovered that it was due to a "441 437 crosspost violation". Now the article I was trying to respond to had been crossposted to the following eclectic set of groups:

When I tried again with "Followup-To: alt.discordia" it worked, but people could not understand why that had been necessary. A little more research and I found that NIN are using the Cleanfeed spam filter for news servers - but without access to their cleanfeed.conf file there is no way to know what policy they are enforcing. Was there a limit on the number of groups which I had exceeded, and if so what was the limit? Or was it something about the particular groups? Well to cut a long story short it seems like NIN do not allow crossposting to alt.alien.vampire.flonk.flonk.flonk - possibly because it is part of The Empire of Meow.


Effective 2005-04-01 News.Individual.NET will be charging 10 EUR per year for their service. Since it seems like an excellent service I have just converted my NetNews account to a paid one, which required setting up a FIRSTGATE account.


When I came back after Christmas break I found that I had lost access to the Easynet news server. This was not unexpected - Black Cat Networks stopped using Easynet as a transit provider some time ago and continued access to their news server was not guaranteed.

Google Groups is great for searching but I wouldn't want to post via Google so I had to come up with an alternative. I asked in the Black Cat discussion list and a couple of people recommended News.Individual.NET so I applied for an account and got access the following day. No problems so far.


Whatever the merits of (cryptographically) signing emails, the signing of Usenet posts is a controversial issue, with some people vehemently opposed to the practice. The one area where it is generally acceptable is with Usenet control messages, where administrators often want to have their systems automatically honour requests from some particular people to add or remove newsgroups - see Authentication of Usenet Group Changes for details.


A week ago I wrote "All your bases are belong to us." in a Usenet post. I was remiss in my research though and stupidly misquoted this famous phrase (I had not got the Engrish quite wrong enough). It should of course have read "All your base are belong to us.", or simply AYBABTU. Ian Gregory 2010