Whitelisting by IP address

[Tom Johnson]

We have some customers who need to whitelist email for their domain based on the sender's IP address.

I don't want to set up custom policy banks for them (we have hundreds of domains, which have different needs and requirements), and I can't add it to a general spamassassin whitelist_from_rcvd setting, since not all customers are going to want that stuff whitelisted. We use sa_userpref's right now, but it's really slow to switch spamassassin contexts, and for this one use, just doesn't seem worth it.

The ideal would be to have it work with the standard wblist feature (we use sql).

Are there any plans to add whitelisting by IP address to the regular wblist feature? Or is this something we're going to have add on our own via a custom hook?

Thanks-

Tom

[Robert Schetterer]

Post by Tom Johnson
We have some customers who need to whitelist email for their domain based on the sender's IP address.
I don't want to set up custom policy banks for them (we have hundreds of domains, which have different needs and requirements), and I can't add it to a general spamassassin whitelist_from_rcvd setting, since not all customers are going to want that stuff whitelisted. We use sa_userpref's right now, but it's really slow to switch spamassassin contexts, and for this one use, just doesn't seem worth it.
The ideal would be to have it work with the standard wblist feature (we use sql).
Are there any plans to add whitelisting by IP address to the regular wblist feature? Or is this something we're going to have add on our own via a custom hook?
Thanks-
Tom

i am not up2date with amavis sql
but in spamassassin something like

https://wiki.apache.org/spamassassin/UsingSQL

username | preference | value |
+------------------+-------------------------+------------------------+
| $GLOBAL | required_hits | 4.00 |
| $GLOBAL | subject_tag | [SPAM-_HITS_]- |
| $GLOBAL | score USER_IN_WHITELIST | -10 |
| $GLOBAL | whitelist_from | *@sonicwall.com

should work

i think in amavis sql exists some equal

but it might be a better idea to use an own build rbl
with latest patches from Patrick

https://groups.google.com/forum/#!topic/mailing.unix.amavis-user/K_LufqxEBKk


Best Regards
MfG Robert Schetterer

[Benny Pedersen]

So foo-user trusted_networks 8.8.8.8 in sql user prefs, untested

Note whitelist_from is a joke

[J. Echter]

Post by Benny Pedersen
So foo-user trusted_networks 8.8.8.8 in sql user prefs, untested
Note whitelist_from is a joke

i'd also go for trusted_networks in local.cf of spamassassin.

Is greylisting also a problem?

[Benny Pedersen]

Post by J. Echter

Post by Benny Pedersen
So foo-user trusted_networks 8.8.8.8 in sql user prefs, untested
Note whitelist_from is a joke

i'd also go for trusted_networks in local.cf of spamassassin.
Is greylisting also a problem?

Yes trusted must be global in mta stage to be usefull for pr user in
spamassassin, so sync ips, dont use it in local.cf where its global, i hope
its possible to use trusted pr recipient, so do it in sql pr user

[Tom Johnson]

I'm sorry if I wasn't clear - adding to trusted_networks is not an option. Different users have different needs. One person might want x.x.x.x whitelisted, but another may not.

Each time that currently loaded configuration needs to be replaced by another or restored to a systemwide default, an initial SpamAssassin configuration is restored through SpamAssassin's copy_config() method. Note that saving an original SpamAssassin configuration, loading a user configuration, and restoring to the original configuration does not come cheap: it can take 200 ms for a load and restore, and 370 ms for the initial saving of the configuration (saving is only done once per child process, and only if needed). Saved configuration can occupy additional 2 MB of virtual memory, so use the feature sparingly. No penalty occurs until a child process does its first loading of a user configuration, so rarely activated or inactive policy banks or per-recipient setting using this feature do not cause any additional processing or occupy additional memory.

I'm trying to avoid this performance hit.

[Robert Schetterer]

Post by Tom Johnson
I'm sorry if I wasn't clear - adding to trusted_networks is not an
option. Different users have different needs. One person might want
x.x.x.x whitelisted, but another may not.

perhaps you like

http://wiki.policyd.org/policies
http://wiki.policyd.org/amavis

Post by Tom Johnson
I'm currently using the sa_userconf feature in amavisd to do this, but

Each time that currently loaded configuration needs to be replaced by
another or restored to a systemwide default, an initial SpamAssassin
configuration is restored through SpamAssassin's copy_config() method.
Note that saving an original SpamAssassin configuration, loading a
user configuration, and restoring to the original configuration does
not come cheap: it can take 200 ms for a load and restore, and 370 ms
for the initial saving of the configuration (saving is only done once
per child process, and only if needed). Saved configuration can occupy
additional 2 MB of virtual memory, so use the feature sparingly. No
penalty occurs until a child process does its first loading of a user
configuration, so rarely activated or inactive policy banks or
per-recipient setting using this feature do not cause any additional
processing or occupy additional memory.

I'm trying to avoid this performance hit.

Best Regards
MfG Robert Schetterer

[Indunil Jayasooriya]

Post by Tom Johnson
I'm sorry if I wasn't clear - adding to trusted_networks is not an
option. Different users have different needs. One person might want
x.x.x.x whitelisted, but another may not.

Why don't you need to whitelist or blacklist domains per user basis in
following way in amavisd.conf file

# ENVELOPE SENDER SOFT-WHITELISTING / SOFT-BLACKLISTING

@score_sender_maps = ({ # a by-recipient hash lookup table,
# results from all matching recipient tables are
summed

# ## per-recipient personal tables (NOTE: positive: black, negative: white)





*'***@example.com <***@example.com>' =>
[{'bla-***@example.com <bla-***@example.com>' =>
10.0}], '***@example.com <***@example.com>' => [{'.ebay.com
<http://ebay.com>' => -3.0}], '***@example.com
<***@example.com>' => [{'***@cleargreen.com
<***@cleargreen.com>' => -7.0,
'.cleargreen.com <http://cleargreen.com>' => -5.0}],*

[Tom Johnson]

Post by Tom Johnson
I'm sorry if I wasn't clear - adding to trusted_networks is not an option. Different users have different needs. One person might want x.x.x.x whitelisted, but another may not.

Why don't you need to whitelist or blacklist domains per user basis in following way in amavisd.conf file
# ENVELOPE SENDER SOFT-WHITELISTING / SOFT-BLACKLISTING
@score_sender_maps = ({ # a by-recipient hash lookup table,

We do whitelisting and blacklisting if senders (using sql).

But we also have some customers who need to whitelist everything coming from a given ip address.

[Michael H]

Post by Tom Johnson

Post by Tom Johnson
I'm sorry if I wasn't clear - adding to trusted_networks is not an
option. Different users have different needs. One person might
want x.x.x.x whitelisted, but another may not.
Why don't you need to whitelist or blacklist domains per user
basis in following way in amavisd.conf file
# ENVELOPE SENDER SOFT-WHITELISTING / SOFT-BLACKLISTING
@score_sender_maps = ({ # a by-recipient hash lookup table,

We do whitelisting and blacklisting if senders (using sql).
But we also have some customers who need to whitelist everything coming
from a given ip address.

Hi,

This is on topic but not quite as the previous thread required.

I have amended my spamassassin/local.cf and added trusted_networks and
internal_networks with all of my IP addresses listed.

I have an alarm system that is emailing without a date field in the
headers, this email originates from an IP address in my trusted_networks
but is still being blocked by amavisd.

Could someone please tell me the correct way to whitelist IP addresses
so that it is applied to amavisd as well as spamassassin?

thanks

Michael

[Michael H]

Post by Michael H

Post by Tom Johnson

Post by Tom Johnson
I'm sorry if I wasn't clear - adding to trusted_networks is not an
option. Different users have different needs. One person might
want x.x.x.x whitelisted, but another may not.
Why don't you need to whitelist or blacklist domains per user
basis in following way in amavisd.conf file
# ENVELOPE SENDER SOFT-WHITELISTING / SOFT-BLACKLISTING
@score_sender_maps = ({ # a by-recipient hash lookup table,

We do whitelisting and blacklisting if senders (using sql).
But we also have some customers who need to whitelist everything coming
from a given ip address.

Hi,
This is on topic but not quite as the previous thread required.
I have amended my spamassassin/local.cf and added trusted_networks and
internal_networks with all of my IP addresses listed.
I have an alarm system that is emailing without a date field in the
headers, this email originates from an IP address in my trusted_networks
but is still being blocked by amavisd.
Could someone please tell me the correct way to whitelist IP addresses
so that it is applied to amavisd as well as spamassassin?
thanks
Michael

Sorry, that was a little vague,

cat /etc/amavisd/amavis.conf

[...]
@mynetworks = qw( 127.0.0.0/8 [::1]
10.0.0.0/8
172.16.0.0/12
192.168.0.0/16
XXX.XXX.XXX.XXX/32
);


# allow all mail from local IPs:
$policy_bank{'MYNETS'} = { # clients in @mynetworks
bypass_spam_checks_maps => [1], # don't spam-check internal mail
bypass_banned_checks_maps => [1], # don't banned-check internal mail
bypass_header_checks_maps => [1], # don't header-check internal mail
};
[...]

cat /etc/mail/spamassassin/local.cf
[...]
required_hits 6
report_safe 0
rewrite_header Subject [SPAM]

internal_networks [IP's of my MX's]

trusted_networks [lots of ip addresses]
[...]

The IP address is in both of these files but the mail is still being
checked, what did I do wrong here?

thanks

[Michael H]

Post by Michael H

Post by Michael H

Post by Tom Johnson

Post by Tom Johnson
I'm sorry if I wasn't clear - adding to trusted_networks is not an
option. Different users have different needs. One person might
want x.x.x.x whitelisted, but another may not.
Why don't you need to whitelist or blacklist domains per user
basis in following way in amavisd.conf file
# ENVELOPE SENDER SOFT-WHITELISTING / SOFT-BLACKLISTING
@score_sender_maps = ({ # a by-recipient hash lookup table,

We do whitelisting and blacklisting if senders (using sql).
But we also have some customers who need to whitelist everything coming
from a given ip address.

Hi,
This is on topic but not quite as the previous thread required.
I have amended my spamassassin/local.cf and added trusted_networks and
internal_networks with all of my IP addresses listed.
I have an alarm system that is emailing without a date field in the
headers, this email originates from an IP address in my trusted_networks
but is still being blocked by amavisd.
Could someone please tell me the correct way to whitelist IP addresses
so that it is applied to amavisd as well as spamassassin?
thanks
Michael

Sorry, that was a little vague,
cat /etc/amavisd/amavis.conf
[...]
@mynetworks = qw( 127.0.0.0/8 [::1]
10.0.0.0/8
172.16.0.0/12
192.168.0.0/16
XXX.XXX.XXX.XXX/32
);
bypass_spam_checks_maps => [1], # don't spam-check internal mail
bypass_banned_checks_maps => [1], # don't banned-check internal mail
bypass_header_checks_maps => [1], # don't header-check internal mail
};
[...]
cat /etc/mail/spamassassin/local.cf
[...]
required_hits 6
report_safe 0
rewrite_header Subject [SPAM]
internal_networks [IP's of my MX's]
trusted_networks [lots of ip addresses]
[...]
The IP address is in both of these files but the mail is still being
checked, what did I do wrong here?
thanks

And here is the message being blocked;

Mar 10 11:57:54 mail1 amavis[22633]: (22633-07) Blocked BAD-HEADER-0
{BouncedInternal,Quarantined}, MYNETS LOCAL [XXX.XXX.XXX.XXX]:12001
[XXX.XXX.XXX.XXX] <***@domain.com> -> <***@domain.com>, quarantine:
badh-CgHOR2w6yANk, Queue-ID: 18EC6818E735, mail_id: CgHOR2w6yANk, Hits:
-, size: 461, 194 ms

Michael

[Michael H]

Post by Michael H

Post by Michael H

Post by Michael H

Post by Tom Johnson

Post by Tom Johnson
I'm sorry if I wasn't clear - adding to trusted_networks is not an
option. Different users have different needs. One person might
want x.x.x.x whitelisted, but another may not.
Why don't you need to whitelist or blacklist domains per user
basis in following way in amavisd.conf file
# ENVELOPE SENDER SOFT-WHITELISTING / SOFT-BLACKLISTING
@score_sender_maps = ({ # a by-recipient hash lookup table,

We do whitelisting and blacklisting if senders (using sql).
But we also have some customers who need to whitelist everything coming
from a given ip address.

Hi,
This is on topic but not quite as the previous thread required.
I have amended my spamassassin/local.cf and added trusted_networks and
internal_networks with all of my IP addresses listed.
I have an alarm system that is emailing without a date field in the
headers, this email originates from an IP address in my trusted_networks
but is still being blocked by amavisd.
Could someone please tell me the correct way to whitelist IP addresses
so that it is applied to amavisd as well as spamassassin?
thanks
Michael

Sorry, that was a little vague,
cat /etc/amavisd/amavis.conf
[...]
@mynetworks = qw( 127.0.0.0/8 [::1]
10.0.0.0/8
172.16.0.0/12
192.168.0.0/16
XXX.XXX.XXX.XXX/32
);
bypass_spam_checks_maps => [1], # don't spam-check internal mail
bypass_banned_checks_maps => [1], # don't banned-check internal mail
bypass_header_checks_maps => [1], # don't header-check internal mail
};
[...]
cat /etc/mail/spamassassin/local.cf
[...]
required_hits 6
report_safe 0
rewrite_header Subject [SPAM]
internal_networks [IP's of my MX's]
trusted_networks [lots of ip addresses]
[...]
The IP address is in both of these files but the mail is still being
checked, what did I do wrong here?
thanks

And here is the message being blocked;
Mar 10 11:57:54 mail1 amavis[22633]: (22633-07) Blocked BAD-HEADER-0
{BouncedInternal,Quarantined}, MYNETS LOCAL [XXX.XXX.XXX.XXX]:12001
-, size: 461, 194 ms
Michael

I'll answer myself then;

Configuring like this fails;
# allow all mail from local IPs:
$policy_bank{'MYNETS'} = { # clients in @mynetworks
bypass_spam_checks_maps => [1], # don't spam-check internal mail
bypass_banned_checks_maps => [1], # don't banned-check internal mail
bypass_header_checks_maps => [1], # don't header-check internal mail
};

adding these two lines solved it;
originating => 1, # is true in MYNETS by default
os_fingerprint_method => undef, # don't query p0f

Post by Michael H
bypass_spam_checks_maps => [1], # don't spam-check internal mail
bypass_banned_checks_maps => [1], # don't banned-check internal mail
bypass_header_checks_maps => [1], # don't header-check internal mail
};

Michael

[Benny Pedersen]

set mynetworks in amavisd, equal to internal_networks in spamassassin

[Michael H]

Post by Benny Pedersen
set mynetworks in amavisd, equal to internal_networks in spamassassin

Hi Benny,

I have done this and it appears to have solved the problem for my alarm
system.

I now have another issue; my mailserver generates a summary of posfix
logs using pflogsum.pl - this is sent from the same host I have amavisd
running on, mynetworks contains 127.0.0.0/8 which should whitelist the
email. it is still being spam checked and rejected, any suggestions?

thanks

Michael

[Benny Pedersen]

set policy bank in amavisd for that reject with a highter spam reject
score, or solve it in spamassassin with shourcircuit based on no_relays, or
all_trusted, so spammasssin is not so agrasive for things in local networks