mail archives slow
gerard at linuxfromscratch.org
Mon Dec 4 12:47:33 PST 2000
> That's not what I want. Usually I just want the individual messages. I
> read them and delete them immediately after reading, but sometimes, when I
> want to look something up, I'd like an up-to-date archive. It's also very
> helpful if your mailfeed has problems for a while. The HTML version is no
> good for that. It's not uptodate and even if it was, it's a pain if you
> just want to read messages in sequence. You have to do it online and you
> have to click around for navigating all the time. A simple text file I
> could download for offline reading would be best in those cases. The HTML
> version is only nice for long-term archival.
The second best thing I can offer you is you using the downloadable mail
archives. They are bzip2'ed mbox archives which contain every single email
from the day the list started. I don't want to give smaller text files. the
question then comes up: how much should that file contain. A week? A month?
More? Less? So that's why you can download the entire list archive (for all
lists). lfsdiscuss.bz2 is a big file (3.6MB at the moment) but well worth it.
No, i can't offer you real-time archives. That is just not possible. That
means you want the server to rebuild the archive every time an email arives.
It takes about 7 minutes at least to rebuild the lfsdiscuss HTML archive. So
real-time is impossible at the moment. The mbox files can't be updated in
real-time either. I don't want to give the webserver read access to the
/var/mail/lfsdiscuss or /var/mail/<enter list name> files. So those files
need to be copied to the FTP archive and bzip2'ed periodically. bzip2 takes a
few minutes (the mbox file for lfsdiscuss is about 25MB so it takes a little
A last possible option is the digest. Every time an email arives to the list,
the digest for that day is updated (and sent out at midnight with the sum of
the previous day's messages for the list). I could perhaps give apache
read-access to those digest files. I don't think this is a major security
risc since there is no secret stuff in listar's digest folder. That way you
can get today's digest when you want it and it'll be up-to-day (well it only
contains the current's day's messages). And I could have listar save digests
so they can be archived.
If I can have postifx store the messages for the lfsdiscuss user's mbox in a
seperate folder (using procmail I could transfer the mails to lfsdiscuss'
homedir which I can give read-access without security risc) and give apache
read-access to that directory and file. That way you can get the big mbox
file and it'll be always up-to-date. Once a day you can get a bzip2'ed
version from the FTP archive.
As you see there are quit a few options. Give me some time please. I do need
time to implement this but I'll try my best to accomodate your and other
-*- If Linux doesn't have the solution, you have the wrong problem -*-
Unsubscribe: send email to lfs-discuss-request at linuxfromscratch.org
and put unsubscribe in the subject header of the message
More information about the lfs-dev