Hello,

[English is not my mother tongue, please let me apologize in advance,
for all the mistakes that I'll write]

<my use of postix>
I'm new to your mailing-list and I've been using Postfix for about 7
months now (switched from qmail after 5 years of use).

I'm using it with the mysql support to host virtual-domains and it
does work without a single glitch (maybe a performance issue during
massive lookups that I'll soon resolve using a memory cache for the
MySQL results when I'll switch it to production).
</>

SPAM and Viruses are not a problem anymore, since I'm using some nice
filtering techniques and selective greylisting.

The last source of junk mail, pretty annoying, is backscatter mail
(unsollicited failure notices from viruses/spam).

I've read the techniques on the Postfix website to filter backscatter...
"catch-all" accounts were already closed by default on my system and
unknown accounts were rejected.
Of course it helped, but still...

On my personal domains, I have a few accounts, with public exposure,
who are totally filled with backscatter, and I cannot filter out ALL
failure notices, since it would be a big breach in the mail protocol.

The last techniques seem to use the HELO, returned in most failure
notices, and some regexps.
I understand the idea behind and it's great....

But I'm sadly not able to use that kind of thing, since I may be
hosting thousands of virtual domains when in production step, and I
don't think a thousand lines regexp would be suitable for that
situation.

So I'm a little bit stuck, and I don't know what to do....

If you have any idea, or are in the same situation...
Any help would do ! 

Thanks a lot,

Ugo PARSI

PS :
Is the "IP filtering" idea on the postfix website, the only solution
to my problem ? (all users will use my outgoing SMTP servers on the
same /24)
Anyone tried it ?

[ Post a follow-up to this message ]

Ugo PARSI wrote:

> I'm using it with the mysql support to host virtual-domains and it
> does work without a single glitch (maybe a performance issue during
> massive lookups that I'll soon resolve using a memory cache for the
> mysql results when I'll switch it to production).

Supply the output of 'postconf -n'.

Cami

[ Post a follow-up to this message ]

On 19/12/04 11:14 +0100, Ugo PARSI wrote:
<snip>
> The last source of junk mail, pretty annoying, is backscatter mail
> (unsollicited failure notices from viruses/spam).
>
> I've read the techniques on the Postfix website to filter backscatter...
> "catch-all" accounts were already closed by default on my system and
> unknown accounts were rejected.
> Of course it helped, but still...
>
> On my personal domains, I have a few accounts, with public exposure,
> who are totally filled with backscatter, and I cannot filter out ALL
> failure notices, since it would be a big breach in the mail protocol.

You can reduce the amount of backscatter allowed in via header and body
checks.
I would recommend not using these accounts for a few months, and then
converting them to spamtraps (any host sending mail there is a spam
sending host, and you can safely stuff the ip of that host in a DNSBL).

<snip>

Devdas Bhagat

[ Post a follow-up to this message ]

Hello,

Thanks for your answer...

>
> You can reduce the amount of backscatter allowed in via header and body
> checks.

Yup, that's the kind of "tips" I was asking and actually looking for 
Do you have some ?

> I would recommend not using these accounts for a few months, and then
> converting them to spamtraps

Sometimes, it's possible, and this is what I made for most of the cases....
It helped to reduce....

Sometimes, it's not, with generic addresses like "webmaster@" used in
many computer address books....
And I can't remove those...

Plus, it wouldn't help the people I am hosting, most of them will
refuse to  change their addresses for the same reasons....

That's why I was more planning on the "filter" thingy.... 

>(any host sending mail there is a spam
> sending host, and you can safely stuff the ip of that host in a DNSBL).

Well, let's suppose that I receive just another failure notice, months
after, coming from BIG ISP who received a fake message from me, by
just another virus in the spread.

I don't think that blacklisting BIG ISP would be such a good idea....

Ugo PARSI

[ Post a follow-up to this message ]

Ugo PARSI wrote:

> Hello,
>
> Thanks for your answer.

Please do not reply to me directly, others lose the ability
to fix the same problem as you if it arises in the future.
 
>
> I don't think this "problem" is postfix related...

I asked for a 'postconf -n' output.  Whats the point of asking
for help if you do not provide the request information?

> It's just that on some of my tests, with hundreds of fake transactions
> per second, the load was very high....
> But the load problem only came from MySQL....and that's quite logical.

No, not really.

> I was actually planning to integrate memcached
> (http://www.danga.com/memcached/) with postfix in order to eliminate
> this "problem".
>
> Do you think this is a good idea ?

No. You will not benefit much from memcached, infact you
will probably make things worse. Read the section on MySQL
caching.

Cami

[ Post a follow-up to this message ]

Ugo PARSI wrote:
 
>
> Yup, that's the kind of "tips" I was asking and actually looking for 
> Do you have some ?

First you stated that the bottleneck is MySQL. Now you're
trying to attempt to let your machines do more work by
adding header/body checks?
 
>
> Sometimes, it's possible, and this is what I made for most of the cases...
.
> It helped to reduce....

This is not a good idea at all but if you are willing to
subject your users to this, its your own fault.

> Sometimes, it's not, with generic addresses like "webmaster@" used in
> many computer address books....
> And I can't remove those...
>
> Plus, it wouldn't help the people I am hosting, most of them will
> refuse to  change their addresses for the same reasons....
>
> That's why I was more planning on the "filter" thingy.... 

Untill you provide a 'postconf -n', no one can see what you
already have in place.

Cami

[ Post a follow-up to this message ]

> Please do not reply to me directly, others lose the ability
> to fix the same problem as you if it arises in the future.

I'm deeply confused....
I didn't check, and I am used to "automatic-reply to all" mailing-lists.....
Sorry.

> I asked for a 'postconf -n' output.  Whats the point of asking
> for help if you do not provide the request information?
>

Didn't mean to be rude with you....
I was just asking for help on backscatter, for "general" purposes
which could typically even work with all MTA on the market..

Here's my postconf -n :

broken_sasl_auth_clients = yes
command_directory = /usr/sbin
config_directory = /etc/postfix
daemon_directory = /usr/libexec/postfix
debug_peer_level = 2
default_destination_concurrency_limit = 20
html_directory = no
local_destination_concurrency_limit = 2
mail_owner = postfix
mailq_path = /usr/bin/mailq
manpage_directory = /usr/local/man
mydestination = $myhostname, localhost.$mydomain, localhost
mydomain = domain.domain.tld
myhostname = mail.domain.tld
mynetworks_style = class
newaliases_path = /usr/bin/newaliases
queue_directory = /var/spool/postfix
readme_directory = no
sample_directory = /etc/postfix
sendmail_path = /usr/sbin/sendmail
setgid_group = postdrop
smtp_tls_note_starttls_offer = yes
smtp_use_tls = yes
smtpd_banner = .:: $myhostname ::. ESMTP
smtpd_helo_required = yes
smtpd_recipient_restrictions = permit_sasl_authenticated
reject_invalid_hostname        reject_non_fqdn_sender
reject_non_fqdn_recipient        reject_unauth_destination
reject_unauth_pipelining
smtpd_reject_unlisted_sender = yes
smtpd_sasl_application_name = smtpd
smtpd_sasl_auth_enable = yes
smtpd_sasl_local_domain = $myhostname
smtpd_sasl_security_options = noanonymous
smtpd_tls_CAfile = /usr/local/ssl/certs/smtpd.pem
smtpd_tls_cert_file = /usr/local/ssl/certs/smtpd.pem
smtpd_tls_key_file = /usr/local/ssl/certs/smtpd.pem
smtpd_tls_loglevel = 0
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
smtpd_use_tls = yes
strict_rfc821_envelopes = yes
tls_random_source = dev:/dev/urandom
unknown_local_recipient_reject_code = 550
virtual_alias_maps = mysql:/etc/postfix/mysql_virtual_alias_maps.cf
virtual_create_maildirsize = yes
virtual_gid_maps = static:1002
virtual_mailbox_base = /usr/local/virtual
virtual_mailbox_domains = mysql:/etc/postfix/mysql_virtual_domains_maps.cf
virtual_mailbox_limit = 51200000
virtual_mailbox_limit_maps =
mysql:/etc/postfix/mysql_virtual_mailbox_limit_maps.cf
virtual_mailbox_limit_override = yes
virtual_mailbox_maps = mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf
virtual_maildir_limit_message = Sorry, the user's maildir has
overdrawn his diskspace quota, please try again later.
virtual_minimum_uid = 1005
virtual_overquota_bounce = yes
virtual_transport = virtual
virtual_uid_maps = static:1005


 
>
> No, not really.
>

Yes it was mysql only 

Postfix wasn't even runned during the last tests, and the load was high...
Only PERL test script <-> MySQL.

But that's not really a problem, I don't plan on doing that much
requests for the present time.
I only have one mail server, and I really doubt that only one regular
server could handle 800-1000 emails / sec....alone....
Most of all, actually, my concurrency limit is set to 20.

It's just that I was testing how well it could handle, on the cluster
I am currently building.

>
> No. You will not benefit much from memcached, infact you
> will probably make things worse.
>

Argghhhh 

Do you have some explanations/examples why ?

I was hoping it could solve the problem later....

> Read the section on mysql caching

On the mysql documentation ?
Because it seems that the cache is flushed everytime the table is changed...
.

If you mean on the Postfix documentation.....
I found nothing on the Postfix doc....nor on Googling the website.
Would be glad if there was some kind of documentation on that 
Any link ?

Thanks,

Ugo PARSI

[ Post a follow-up to this message ]

> First you stated that the bottleneck is MySQL.

On my tests, not on the present time...

I can handle with one server sharing mysql very well for the present time...
.

My mail wasn't dedicated for mysql issue.... but for general help/tips
on backscatter....

> Now you're
> trying to attempt to let your machines do more work by
> adding header/body checks?
>

Yes, since the problem is not located here....

A cluster will do fine the work, and the more CPU needed, the more
computers will be added....


>
> This is not a good idea at all but if you are willing to
> subject your users to this, its your own fault.
>

That's what I said !.....

For MY accounts, I could easily deal with most of it....
Not for other users....
And I'm not planning on using spamtraps like this....
Read my whole email....

>
> Untill you provide a 'postconf -n', no one can see what you
> already have in place.

Done...
But was only looking for tips/plan to filter...like on my first mail....
Not for debugging my actual configuration......

[ Post a follow-up to this message ]

> Again, please learn to at least CC the list.

Damn it !!!
I'm really sorry....

I'm so used to the kind of mailing-list with automatic reply-to set to
the mailing-list itself.....

Sorry again !

[ Post a follow-up to this message ]

>
> Seems i misread, please ignore my previous mail regarding memcache.
>
> It can/will do what you are wanting but if your mysql tables do not
> change that often, the mysql query cache will still perform well.
>

Fine.

Ok, but my mysql data may change at users' will, plus I will need a
way to spread the cache between separate computers (cluster) and not
only between processes.

I'll try both anyway and see who's most appropriate 

Thanks for your proxymap advice.....

>
> So what? Spammers dont care where you're doing the hosting,
> they simply try HELO with the domains they are delivering to.
>

No what I meant is that I can't do simples rules like
domain1.com|domain2.com|domain3.com explained on the Postfix
Backscatter's doc...

Or the regexp would be rather slow if I reach something like 5000
domains.... (I suppose)

I was looking for a more global/faster way to filter out....

Actually the IP thing explained in "Blocking backscatter mail with
other forged information", would be global enough....

The only problem is that I can't see examples/ideas to implement it....
 
>
> Hrmph.. What for? Why not simply use a btree database?

You mean a local dump generated from my mysql tables ?

That makes me loose a great part of my dynamic stuff, if I must
regenerate a file on recurrent basis...

And I could also generate all the mysql tables into local dumps...

> 
>
> Most of the time, yes. (Badly configured) Mailers on your network
> should be identified and fixed.
>

Actually all of the mailers on my network, or users' network, are SMTP
Authenticated...

So is it safe to say, that incoming e-mails and not authenticated,
should never "HELO my_domains.tld" ?

I think it is, but I would like it to be confirmed...

>
> You cant and i doubt you'd actually want to. Its much faster
> doing local pcre/database lookups.


So, for example, a 5000 domains regexp (domain1|domain2|domain3,
etc...) will be faster than a (cached) database lookup ?


Thanks,

Ugo PARSI

[ Post a follow-up to this message ]