Hi,

I have installed and am running milter-greylist 3.0 (will upgrade to
4.0 once it's 'released'), and I was wondering how triplet timeout works?

I configured and compiled using:

CFLAGS="-O2 -march=i686 -mcpu=i686" \
./configure --prefix=/usr/local \
              --sysconfdir=/etc \
              --with-delay=900 \
              --with-timeout=18000 \
              --with-autowhite=3110400

and looking at /var/milter-greylist/greylist.db, there are entries in
the greylist which are older that 18000 seconds (5 hours). Is
'timeout' for this purpose?

Collin

On Wed, May 02, 2007 at 02:14:14PM +0800, Collin Baillie wrote:
> and looking at /var/milter-greylist/greylist.db, there are entries in
> the greylist which are older that 18000 seconds (5 hours). Is
> 'timeout' for this purpose?

milter-greylist only clean up timed out tuples when it walks the database
and encounter them. So if the database is not searched up to an expired
tuple, it will remain here.

--
Emmanuel Dreyfus
manu@...

Hello,

after ten days of operation milter-greylist uses 1.8 GB virtual
memory with some estimated 50 to 100 MB more per day, but greylist.db
has only 42 MB.
If this is expected behaviour, we would plan an automatic re-start.
When to restart best?

Using 4.0a1, x86_64 SuSe Linux 10.1, dnsrbl and extended regexp configuration.

Michael

Michael Fromme <fromme@...> wrote:

> after ten days of operation milter-greylist uses 1.8 GB virtual
> memory with some estimated 50 to 100 MB more per day, but greylist.db
> has only 42 MB.

I'd say there is a leak.

--
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Hi all,

At home I run postfix as my MTA and I use 'gps' (the greylisting
policy service) for greylisting. A feature that I really enjoy there
which I think I could benefit with here at work is 'reverse
greylisting' Full details are available on the gps web page:

http://mimo.gn.apc.org/gps/

Basically I've had some trouble with milter-greylist and Gmail's many
email servers. I don't want to whitelist anything from gmail.com, but
I think the idea of gps's reverse greylisting would be very helpful,
especially considering the automatic fallback to weak greylisting
(also explained on the page).

Can this be done with milter-greylist? If not, is it a feature worth
requesting?

Collin

Collin Baillie <collin@...> wrote:

> http://mimo.gn.apc.org/gps/
(snip)

I sum it up for other readers:
greylist using (domain name of sender machine, sender e-mail, receiver
e-mail) instead of (IP, sender e-mail, receiver e-mail). eg: (yahoo.com,
sender@..., user@...)

> Can this be done with milter-greylist? If not, is it a feature worth
> requesting?

The idea is interesting, but doesn't that let much more spam from
botnets? Zombies connected from the same ISP will have similar tuples.

About implementing it: contributed code is welcome, but please discuss
here how you are going to do it, especially on the configuration file
format front.

--
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

At 13:17 4/05/2007, manu@... wrote:

>I sum it up for other readers:
>greylist using (domain name of sender machine, sender e-mail, receiver
>e-mail) instead of (IP, sender e-mail, receiver e-mail). eg: (yahoo.com,
><mailto:sender@...>sender@...,
><mailto:user@...>user@...)

Well, your example isn't technically correct. It would be more likely
eg of "dyn-ip.adsl.isp.com", "spoofed.address@...", "victim@..."

Since the reverse lookup is done on the IP of the sending 'MTA'.

> > Can this be done with milter-greylist? If not, is it a feature worth
> > requesting?
>
>The idea is interesting, but doesn't that let much more spam from
>botnets? Zombies connected from the same ISP will have similar tuples.

The zombies aren't really zombies though are they? Maybe if they're
virus-compromised machines, but the chances of more than one machine
in the same ISP being infected with the same virus which is sending
out the same 'from' address, is very very low. And as for spammers,
how often do they send using the same 'from' address (ie they tend to
generate random addresses)? How often to they send more than one span
from the same address to the same address?

It is call _weak_ greylisting for a reason ;), but I've never had any
problem with it in the year or so I have been running it on my domain.

>About implementing it: contributed code is welcome, but please discuss
>here how you are going to do it, especially on the configuration file
>format front.

I am not a coder, so I have no idea where to start. My coding skills
are 2nd year univeristy and very rusty :D

Collin

Hi

weak greylisting is possible with the -L option or with subnetmatch in
greylist.conf

The reverse greylisting is not possible with milter-greylist as far as i know.
I don't see the advantage form reverse greylisting over the
subnetmatch (weak greylisting). Could you give us an example where
reverse would be of use and a subnetmatch not?

regards

              Michael

Quoting Collin Baillie <collin@...>:

>
> Hi all,
>
> At home I run postfix as my MTA and I use 'gps' (the greylisting
> policy service) for greylisting. A feature that I really enjoy there
> which I think I could benefit with here at work is 'reverse
> greylisting' Full details are available on the gps web page:
>
> http://mimo.gn.apc.org/gps/
>
> Basically I've had some trouble with milter-greylist and Gmail's many
> email servers. I don't want to whitelist anything from gmail.com, but
> I think the idea of gps's reverse greylisting would be very helpful,
> especially considering the automatic fallback to weak greylisting
> (also explained on the page).
>
> Can this be done with milter-greylist? If not, is it a feature worth
> requesting?
>
> Collin
>
>



--------------------------------------------------------------------------------
M.Menge                                 Tel.: (49) 7071/29-70316
Universitaet Tuebingen                  Fax.: (49) 7071/29-5912
Zentrum fuer Datenverarbeitung          mail:
michael.menge@...
Waechterstrasse 76
72074 Tuebingen

On Fri, May 04, 2007 at 02:53:17PM +0800, Collin Baillie wrote:
> The zombies aren't really zombies though are they? Maybe if they're
> virus-compromised machines, but the chances of more than one machine
> in the same ISP being infected with the same virus which is sending
> out the same 'from' address, is very very low.

I would not bet on that. The zombies are tightly controlled, and the
spammer can command them to do it if it works around a pam filter.

--
Emmanuel Dreyfus
manu@...

At 15:13 4/05/2007, Michael Menge wrote:
>Hi
>
>weak greylisting is possible with the -L option or with subnetmatch in
>greylist.conf
>
>The reverse greylisting is not possible with milter-greylist as far as i know.
>I don't see the advantage form reverse greylisting over the
>subnetmatch (weak greylisting). Could you give us an example where
>reverse would be of use and a subnetmatch not?


I had considered using -L, and now that you raise it, I may look
closer at it. The difference of course is that if you use -L 24, you
accept a whole 'C class' block (ie 255 hosts). Most people won't have
255 email servers, and if they do, they're probably not all stuck on
the same 'subnet' or 'C class' network. Doing reverse lookups and all
the IPs used by a cluster of email gateways for a large provider (ie
google, yahoo etc) and they *should* all resolve back to the same domain.

Blindly accepting 255 hosts sounds less intelligent (to me) than
blindly accepting from all IPs which resolve to the same domain
(sending from the same address to the same address). Yes it opens up
_possibilities_ for abuse, but how would someone who may have an IP
address which resolves to the same domain as a real email server,
know the sender address and recipient address of those real email
servers? Once again, how many spammers or viruses send from the same address?

As it stands I am whitelisting everyone in our organisation who uses
gmail to commuicate with the internal network. However, this is going
to be a pain I think, as we experience fairly frequent changes in staffing.

Collin

Collin Baillie schreef:
>
>
> At 15:13 4/05/2007, Michael Menge wrote:
>  >Hi

What you are trying to do sounds a lot like the urlcheck I wrote and
emailed to the mailing list.

https://webmail.coltex.nl/spam/mxhostcheck.php?domain=zaobao.com.sg&ip=83.30.109\
.104&fuzz=22&delay=9

Pass it the sender domain, the sender ip, the subnet match (like -L),
and how long the greylist timeout must be if it does not match.

Experiment with the options a bit.

If you just use a browser you can see exactly what I'm doing.

Cheers,

Seth

Hi,

Quoting Collin Baillie <collin@...>:

>
> I had considered using -L, and now that you raise it, I may look
> closer at it. The difference of course is that if you use -L 24, you
> accept a whole 'C class' block (ie 255 hosts). Most people won't have
> 255 email servers, and if they do, they're probably not all stuck on
> the same 'subnet' or 'C class' network. Doing reverse lookups and all
> the IPs used by a cluster of email gateways for a large provider (ie
> google, yahoo etc) and they *should* all resolve back to the same domain.
>

If you whitelist by IP you can specify the netmask so you can
whitelist 1,2,4,8,...256,....all IPs if you want. The -L option is
only for greylistet tubles, so if a retrie is done by a other server
wrom that subnet it will pass.

ISP will normaly have there Mailserver in one or two nets and there
dynamic IPs in an other network, so if you resolve only by domainname
you will whitelist mailserver1.testdomain.de mailserver2.testdomain.de
and also dynamicip1.testdomain.de and dynamicip2.testdomain.de


> Blindly accepting 255 hosts sounds less intelligent (to me) than
> blindly accepting from all IPs which resolve to the same domain
> (sending from the same address to the same address). Yes it opens up
> _possibilities_ for abuse, but how would someone who may have an IP
> address which resolves to the same domain as a real email server,
> know the sender address and recipient address of those real email
> servers? Once again, how many spammers or viruses send from the same address?
>
> As it stands I am whitelisting everyone in our organisation who uses
> gmail to commuicate with the internal network. However, this is going
> to be a pain I think, as we experience fairly frequent changes in staffing.
>
> Collin
>
>



--------------------------------------------------------------------------------
M.Menge                                 Tel.: (49) 7071/29-70316
Universitaet Tuebingen                  Fax.: (49) 7071/29-5912
Zentrum fuer Datenverarbeitung          mail:
michael.menge@...
Waechterstrasse 76
72074 Tuebingen

>subnetmatch (weak greylisting). Could you give us an example where
>reverse would be of use and a subnetmatch not?

Yup. Right now, I'm experiencing problems with emails from Gmail. In
the logs, the from IPs come from:

wr-out-0506.google.com[64.233.184.228]
nz-out-0506.google.com[64.233.162.229]
mu-out-0910.google.com [209.85.134.188]
an-out-0708.google.com [209.85.132.251]

and so on. If I could use the reverse greylisting, which should
truncate the names to google.com, they would all work nicely together.

Collin

On Fri, May 04, 2007 at 04:45:42PM +0800, Collin Baillie wrote:
> wr-out-0506.google.com[64.233.184.228]
> nz-out-0506.google.com[64.233.162.229]
> mu-out-0910.google.com [209.85.134.188]
> an-out-0708.google.com [209.85.132.251]
>
> and so on. If I could use the reverse greylisting, which should
> truncate the names to google.com, they would all work nicely together.

But why do you want to greylist mail comming from Google mail farms?
Since that mail is from a legitimate machine, you don't want delays,
you'd be better whitelisting it.

racl whitelist domain /-out-[0-9]\{4\}\.google.com/

--
Emmanuel Dreyfus
manu@...

Hello,

searching for a reason I found the dnsrbl_list_cleanup function
of dnsrbl.c which is not called anywhere (?)

A place would be real_close, just before free(priv) like
#ifdef USE_DNSRBL
		 dnsrbl_list_cleanup(priv);
#endif

Is the dnsrbl list bound to a single email - or am I
missing something here? DNSBL lookups may take time, and
it's a good idea to keep knowledge for let's say an hour.

Michael

On Fri, May 04, 2007 at 01:57:46PM +0200, Michael Fromme wrote:
> searching for a reason I found the dnsrbl_list_cleanup function
> of dnsrbl.c which is not called anywhere (?)
>
> A place would be real_close, just before free(priv) like
> #ifdef USE_DNSRBL
> 	 dnsrbl_list_cleanup(priv);
> #endif

Indeed you found a memory leak.

> Is the dnsrbl list bound to a single email - or am I
> missing something here?

Yes, the goal is to avoid multiple DNS requests when the same DNSRBL
is used in several ACL lines.

> DNSBL lookups may take time, and
> it's a good idea to keep knowledge for let's say an hour.

I'm not sure it's interesting to build a cache system: you can
have a local DNS on your machine to make the lookups faster, and that
local DNS will respect the TTL set by the DNSRBL publisher.

--
Emmanuel Dreyfus
manu@...

Tere.

Well, first I trid in OSX, no luck, now with debian, same problems, can't get it
right:


./configure --enable-postfix --with-user=postfix --without-db --without-drac-db
\
--with-conffile=/etc/greylist/greylist.conf
--with-dumpfile=/etc/greylist/greylist.db \
--with-libmilter=/home/mart/sendmail-8.14.1/include/libmilter

Required libmilter not found. Use --with-libmilter
gcc -g -O2 -Wall -D_BSD_SOURCE   -c -o milter-greylist.o milter-greylist.c
milter-greylist.c:80:29: libmilter/mfapi.h: No such file or directory
In file included from dump.h:67,
                  from milter-greylist.c:82:
milter-greylist.h:40:29: libmilter/mfapi.h: No such file or directory


Tried:


CFLAGS=/home/mart/sendmail-8.14.1/include/libmilter

./configure --enable-postfix --with-user=postfix --without-db --without-drac-db
\
--with-conffile=/etc/greylist/greylist.conf
--with-dumpfile=/etc/greylist/greylist.db \
--with-libmilter=/home/mart/sendmail-8.14.1



Required libmilter not found. Use --with-libmilter
gcc -g -O2 -Wall -D_BSD_SOURCE   -c -o milter-greylist.o milter-greylist.c
milter-greylist.c:80:29: libmilter/mfapi.h: No such file or directory
In file included from dump.h:67,
                  from milter-greylist.c:82:
milter-greylist.h:40:29: libmilter/mfapi.h: No such file or directory


/home/mart/sendmail-8.14.1/include/libmilter# ls -al
total 40
drwxr-xr-x  2 1037 root  4096 Apr  4 02:32 .
drwxr-xr-x  6 1037 root  4096 Apr  4 02:32 ..
-rw-r--r--  1 1037 root 16588 Nov  2  2006 mfapi.h
-rw-r--r--  1 1037 root  4727 Mar 27 21:53 mfdef.h
-rw-r--r--  1 1037 root   641 May 23  2006 milter.h


Any hint?


--
Mart

On Fri, 04 May 2007 16:49:28 +0300 Mart Pirita <sysadmin@...> wrote:

> ./configure --enable-postfix --with-user=postfix --without-db
--without-drac-db \
> --with-conffile=/etc/greylist/greylist.conf
--with-dumpfile=/etc/greylist/greylist.db \
> --with-libmilter=/home/mart/sendmail-8.14.1/include/libmilter
>
> Required libmilter not found. Use --with-libmilter
> gcc -g -O2 -Wall -D_BSD_SOURCE   -c -o milter-greylist.o milter-greylist.c
> milter-greylist.c:80:29: libmilter/mfapi.h: No such file or directory

Is libmilter/mfapi.h in /home/mart/sendmail-8.14.1/include/libmilter?
I don't think so. You should probably use
--with-libmilter=/home/mart/sendmail-8.14.1/include.

Regards,
Nerijus

Here is 4.0 alpha3:
http://ftp.espci.fr/pub/milter-greylist/milter-greylist-4.0a3.tgz
MD5 (milter-greylist-4.0a3.tgz) = 76708dda6c3414321d4147e6274a7563

From ChangeLog:
         Add an urlcheck reply which is ignored: milterGreylistIgnore
	 Build fixes for Debian (Bernhard Schneider)
	 Add a configure flag to not use --rpath (Bernhard Schneider)
	 Cleanup stale PID file on startup (Bernhard Schneider)
	 Fix memory leak with DNSRBL (Michael Fromme)

--
Emmanuel Dreyfus
manu@...

Getting an error on start-up:

May  4 14:14:44 mostly milter-greylist: Cannot open pidfile
"/var/run/milter-greylist.pid" for writing: Permission denied

Configured with only one option:

./configure --with-user=smmsp

On 5/4/07, ca35763+mgl@...
<ca35763+mgl@...> wrote:
>  Getting an error on start-up:
>
>  May  4 14:14:44 mostly milter-greylist: Cannot open pidfile
>  "/var/run/milter-greylist.pid" for writing: Permission denied
>
>  Configured with only one option:
>
>  ./configure --with-user=smmsp

...and does the user smmsp have write permissions in /var/run?

--
/peter

Tere.
>
> Is libmilter/mfapi.h in /home/mart/sendmail-8.14.1/include/libmilter?
>
Yes it is - /home/mart/sendmail-8.14.1/include/libmilter/mfapi.h Didn't
You notice in the previous post:

/home/mart/sendmail-8.14.1/include/libmilter# ls -al
total 40
drwxr-xr-x  2 1037 root  4096 Apr  4 02:32 .
drwxr-xr-x  6 1037 root  4096 Apr  4 02:32 ..
-rw-r--r--  1 1037 root 16588 Nov  2  2006 mfapi.h
-rw-r--r--  1 1037 root  4727 Mar 27 21:53 mfdef.h
-rw-r--r--  1 1037 root   641 May 23  2006 milter.h



> I don't think so. You should probably use
--with-libmilter=/home/mart/sendmail-8.14.1/include.
>
>
I tried everything,  the milter-greylist.c line 80 is - #include
<libmilter/mfapi.h>, I even modified it to #include "mfapi.h" and copied
/home/mart/sendmail-8.14.1/include/libmilter/mfapi.h into
/home/mart/milter-greylist-3.1.8 with no luck.


In redhat I have:

/usr/include/libmilter
/usr/include/libmilter/mfapi.h
/usr/include/libmilter/mfdef.h

And milter-greylist-3.1.8 compiles fine without the --with-libmilter option.

Copied also stuff in debian into /usr/include/libmilter didn't help.

Can't figure out what is wrong?

--
Mart

On Fri, 4 May 2007, shuttlebox wrote:

> Date: Fri, 4 May 2007 21:40:27 +0200
> From: shuttlebox <shuttlebox@...>
> Reply-To: milter-greylist@yahoogroups.com
> To: milter-greylist@yahoogroups.com
> Subject: Re: [milter-greylist] Pidfile in 4.0 alpha3
>
> On 5/4/07, ca35763+mgl@...
> <ca35763+mgl@...> wrote:
>>  Getting an error on start-up:
>>
>>  May  4 14:14:44 mostly milter-greylist: Cannot open pidfile
>>  "/var/run/milter-greylist.pid" for writing: Permission denied
>>
>>  Configured with only one option:
>>
>>  ./configure --with-user=smmsp
>
> ...and does the user smmsp have write permissions in /var/run?
>
> --
> /peter
>

Obviously it can't write to /var/run!  My intent was to point out that
alpha 2 wrote the pidfile as root.  The alpha 3 "Cleanup stale PID file on
startup" apparently changed pidfile user too.  I didn't know if that was
the intended behavior.

Anyway, I moved the pidfile to the database directory which is writable
and works.

Thanks.

<ca35763+mgl@...> wrote:

> Obviously it can't write to /var/run!  My intent was to point out that
> alpha 2 wrote the pidfile as root.  The alpha 3 "Cleanup stale PID file on
> startup" apparently changed pidfile user too.  I didn't know if that was
> the intended behavior.

Yes, that change introduce a nasty upgrade trap. It will be fixed in
alpha 4.

--
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Just in case people did not notice, I repost it with a more visible
title. alpha4 will fix the issue.

Note that if the user milter-greylist runs as (smmsp for instance) can
write to the directory where the PID file is stored (/var/run for instance),
then you'll have no problem.

On Sat, May 05, 2007 at 08:11:21AM +0200, manu@... wrote:
> <ca35763+mgl@...> wrote:
>
> > Obviously it can't write to /var/run!  My intent was to point out that
> > alpha 2 wrote the pidfile as root.  The alpha 3 "Cleanup stale PID file on
> > startup" apparently changed pidfile user too.  I didn't know if that was
> > the intended behavior.
>
> Yes, that change introduce a nasty upgrade trap. It will be fixed in
> alpha 4.
>
> --
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> manu@...

--
Emmanuel Dreyfus
manu@...

On Sat, 05 May 2007 01:04:58 +0300 Mart Pirita <sysadmin@...> wrote:

> > Is libmilter/mfapi.h in /home/mart/sendmail-8.14.1/include/libmilter?
>
> Yes it is - /home/mart/sendmail-8.14.1/include/libmilter/mfapi.h Didn't
> You notice in the previous post:
>
> /home/mart/sendmail-8.14.1/include/libmilter# ls -al
> -rw-r--r--  1 1037 root 16588 Nov  2  2006 mfapi.h

I did notice. But didn't you notice that I deliberately asked about
"libmilter/mfapi.h",
not mfapi.h. If you do ls -l libmilter/mfapi.h in this directory, you'll get
ls: libmilter/mfapi.h: No such file or directory.
But if you do it in /home/mart/sendmail-8.14.1/include, you will get correct
result.
Now do you see the difference?

But I don't think it's worth doing what you are trying to do now - much easier
would be to install sendmail-devel (sendmail-dev?) package in Debian.

Regards,
Nerijus

> Here is 4.0 alpha3:
> http://ftp.espci.fr/pub/milter-greylist/milter-greylist-4.0a3.tgz
> MD5 (milter-greylist-4.0a3.tgz) = 76708dda6c3414321d4147e6274a7563
>
> From ChangeLog:
>         Add an urlcheck reply which is ignored: milterGreylistIgnore
>  Build fixes for Debian (Bernhard Schneider)
>  Add a configure flag to not use --rpath (Bernhard Schneider)
>  Cleanup stale PID file on startup (Bernhard Schneider)
>  Fix memory leak with DNSRBL (Michael Fromme)
>

Hello,

I get now this error message at configure stage on version 4.0a3:

flex: unknown flag '-'.  For usage, try
         flex --help
configure: error: flex missing yypop_buffer_state - upgrade to version 2.5.33 or
  later

Is this change expected?

Bests,
Attila

On Mon, May 07, 2007 at 04:25:19PM +0200, attila.bruncsak@... wrote:
> configure: error: flex missing yypop_buffer_state - upgrade to version 2.5.33
or
>  later
> Is this change expected?

No, probably an autoconf issue. i'll check it later and produce a new
snapshot.

--
Emmanuel Dreyfus
manu@...

Hello

Here is milter-greylist 4.0 alpha4:
http://ftp.espci.fr/pub/milter-greylist/milter-greylist-4.0a4.tgz
MD5 (milter-greylist-4.0a4.tgz) = 674a886164794c4b71137b961fc5af41

Changes from alpha3:
         Fix upgrade pitfal, where write access to the PID file is missing
	 Back out a Debian build fix that broke other systems

--
Emmanuel Dreyfus
manu@...

Tere.
>
> I did notice. But didn't you notice that I deliberately asked about
"libmilter/mfapi.h",
> not mfapi.h. If you do ls -l libmilter/mfapi.h in this directory, you'll get
> ls: libmilter/mfapi.h: No such file or directory.
> But if you do it in /home/mart/sendmail-8.14.1/include, you will get correct
result.
> Now do you see the difference?
>
Yes I see.
> But I don't think it's worth doing what you are trying to do now - much easier
> would be to install sendmail-devel (sendmail-dev?) package in Debian.
>
>
Ok, good idea, tried, no such package, installed libmilter-dev instead,
but now new error occured:

In file included from conf_yacc.y:1173:
conf_lex.l: In function `conf_dispose_input_file':
conf_lex.l:311: error: `YY_FLUSH_BUFFER' undeclared (first use in this
function)
conf_lex.l:311: error: (Each undeclared identifier is reported only once
conf_lex.l:311: error: for each function it appears in.)
conf_yacc.y: At top level:
conf_lex.c:2130: warning: `yyunput' defined but not used
make: *** [conf_yacc.o] Error 1

Now what?




--
Mart