Email questions

[Bob Tucker]

I am having problems with email handling. We have an Exchange server. The SMTP mail server in Wingate receives all Internet email. Wingate email is configured to forward all email to another server - which is our Exchange server. The Wingate email server checks against the SORBS, Spamhaus, and SPEWS databases and does anti-spoof detection. We are using Wingate 6.0.3 build 1005.

1.) Every day, something gets fouled up in handling one email. This apparently happens when Wingate email trying to move a message to the "dead" folder. The rcp file for the message ends up in the "postin" directory. The message itself will end up in any directory. Once the rcp file has ended in the postin directory, there will literally be thousands transactions and related entries in the SMTP log similar to the following:

09/30/04 23:50:29 Error: Message 0000001730 could not move .msg file error 2
09/30/04 23:50:29 Error: Message 0000001730 msg file not found, rcp file moved to dead

The cure for this is to stop Wingate and find and delete the mmsg file and rcp file for the errant message. Once this process starts, Wingate does not seem to have any way of recovering. Once I have fixed this, Wingate will run for 8 to 24 hours before the same thing happens again. Is there any way to deal with this?

2.) I am fighting a losing battle with SPAM control. There is some SPAM engine that sends huge amounts of SPAM addressed to non-existent users in our domain. Our Exchange server dutifully sends NDS messages to the senders of these messages - who are the real intended recipients. I have found that aggressive anti-spoof detection and using some of the more aggressive Open Relay databases has limited the amount of phony emails reaching us. I am not receiving some legitimate emails, so I need to employ a somehat different tactic. I have changed the email addresses for users here aso that they all contain an character sequence that, of course, is not in any of the fake addresses. I would like to be able to configure the Wingate email server to send and receive Internet mail for email addresses containing that character sequence and refuse to send or receive Internet mail for addresses in our domain without that sequence. What would be the best way I can do this?

I have scripted a little program that extracts the user mnames from the phony email being sent to us. There appear tro be about 2,000 names. Would it be possible to check fotr email addresses on that list and reject them?

[adrien]

Hi Bob

we have occasionally seen that problem with the inability to move the file, I see it makes for large log files if you have debug logging turned on. We are still looking into the cause.

As for the spam issue, there are several options.

1. Make WinGate know all valid email addresses, and deny the rest. This is potentially a 3 click process, if you use the NT user database in WinGate, point it to your Active Directory server, or wherever Exchange gets the list of users from, then you can create address handlers for all users in your domain (create all), and not accept mail for any other addresses in the domain. Then when the spammer tries to send to non-existant addresses, no DSN will be generated since the mail will not be accepted in the first place - the RCPT command will be rejected.

2. Use the code in the email address... this would require all users however to add the code to their address, and tell everyone to use it. Supposing you were able to overcome that hurdle, then it is simple to add a requirement to the SMTP policies whereby on the advanced tab, the recipient address must contain your code, or the recipient domain not be local (probably need to add domains for this). This will allow outbound, but require any inbound to be tagged.

3. Can you turn off DSNs to local addresses from local addresses?

If the Exchange server delivers the DSN back out through WinGate, you could set up a policy whereby if the connection was from the exchange server, and the sender address was empty (DSN), then don't allow if the recipient domain is local. This will stop the DSNs to your local users.

Adrien

[adrien]

Bob

Some of these policy set ups can be a bit mind-bending, especially since you need to lock down the scope of other policies to prevent them circumventing the new one you are trying to create. If you like, I could give you a hand by remote login, or remote desktop.

email me at adrien at qbik dot com if you are interested.

Adrien

[Bob Tucker]

Dear Adrien,

I hope it may be more simple than you suggest. All of the valid email addresses here are of the form first.last@company.com. The phony email comes from addresses of the form name@company.com. There is no period in the name in the fake email. I am hoping I can select on that period. I would like to set a policy or rule whereby if the connection is from the Internet, and the name in the recipient address does not contain an embedded period, then the email is not received.

I would have to duplicate all of the Exchange SMTP email addresses with handlers in the Wingate email server to achieve this end using the feature in Wingate email that excludes non-existent addresses. Each time someone joins or leaves this facility- which is a divion of an international company operating in Thailand, it would be necessary to create addresses both in Wingate and Exchange. There are language and complexity barriers that would be breached in this. I need to have a relatively simple rule that can be administrated without too much complexity.

[kgoodknecht]

Dear Adrien,

I hope it may be more simple than you suggest. All of the valid email addresses here are of the form first.last@company.com. The phony email comes from addresses of the form name@company.com. There is no period in the name in the fake email. I am hoping I can select on that period. I would like to set a policy or rule whereby if the connection is from the Internet, and the name in the recipient address does not contain an embedded period, then the email is not received.

I would have to duplicate all of the Exchange SMTP email addresses with handlers in the Wingate email server to achieve this end using the feature in Wingate email that excludes non-existent addresses. Each time someone joins or leaves this facility- which is a divion of an international company operating in Thailand, it would be necessary to create addresses both in Wingate and Exchange. There are language and complexity barriers that would be breached in this. I need to have a relatively simple rule that can be administrated without too much complexity.

I run Exchange behind Wingate and I have had a great deal of success in rejecting spam. Wingate makes it easy to import the AD user database, how well this works for you depends on how your usernames are set up in AD. If the username is name@company.com and your email addresses are first.lastname@company.com it makes it a lttle more difficult. I'm not sure why you haven't configured your accounts so that users logon with their e-mail address or why you get so much e-mail at name@company.com You should verify that first.lastname@company.com is the default E-mail address in the user properties. Remember it is this default E-mail address that appears in the from line for outgoing mail from your users.

[Bob Tucker]

Dear Kevin,

Thank you for the reply. As mentioned, this site is in Thailand and is a subsidiary of a European International.

The parent company has a secure single sign on. This does not permit using email addresses for login. This is true of all Internationals of which I am aware. It is certainly true of those with which we work.

Thai script is a significant issue. Win2K does not have a Thai script interface. Therefore, Windows NT is still in use at this site. Hence no AD.

I am happy to hear that Wingate provides effective SPAM control at your site. We did not enjoy a huge SPAM problem in Asia until a few months ago. The implementation of the CAN-SPAM act in the US has changed things . Spammers are relaying SPAM throughout Asia via what is being called "misdirected" email - NDS messages, Out-of-Office notificatons, etc. Email servers throughout Asia are targets. The problem here is from phony email which generates NDS messages via mail sent to non-existent users at organizations with SMTP servers. Mail servers at many - if not most - such organizations in Thailand have recently been overwhelmed generating NDS notices. Internet capacity is not great in Thailand, and overall Internet traffice has increased ablove the capacity of some ISPs here - and the increase is all SPAM. Intended recipients of all this SPAM, however, seem to be in the North America and Europe rather than Asia. The SPAM problem in Thailand is not similar to that in the US. The mail servers at every company we work with are hit with thousands of these phony email every day. Thereofere, effective action here, of course, must be somewhat different than in the US.

[kgoodknecht]

Dear Kevin,

Thank you for the reply. As mentioned, this site is in Thailand and is a subsidiary of a European International.

The parent company has a secure single sign on. This does not permit using email addresses for login. This is true of all Internationals of which I am aware. It is certainly true of those with which we work.

Thai script is a significant issue. Win2K does not have a Thai script interface. Therefore, Windows NT is still in use at this site. Hence no AD.

I am happy to hear that Wingate provides effective SPAM control at your site. We did not enjoy a huge SPAM problem in Asia until a few months ago. The implementation of the CAN-SPAM act in the US has changed things . Spammers are relaying SPAM throughout Asia via what is being called "misdirected" email - NDS messages, Out-of-Office notificatons, etc. Email servers throughout Asia are targets. The problem here is from phony email which generates NDS messages via mail sent to non-existent users at organizations with SMTP servers. Mail servers at many - if not most - such organizations in Thailand have recently been overwhelmed generating NDS notices. Internet capacity is not great in Thailand, and overall Internet traffice has increased ablove the capacity of some ISPs here - and the increase is all SPAM. Intended recipients of all this SPAM, however, seem to be in the North America and Europe rather than Asia. The SPAM problem in Thailand is not similar to that in the US. The mail servers at every company we work with are hit with thousands of these phony email every day. Thereofere, effective action here, of course, must be somewhat different than in the US.

Sorry, you didn't mention you were using Exchange 5.5, I was assuming Exchange 2k or 2k3 which requires Active Directory. The resolution is still the same in Wingate you'll have to configure Winagte to accept only e-mail for configured users and reject unknown users. It will take some time but It think it will be worth it in the long run.

[labull]

Please allow me to suggest an alternative.

Having WinGate accept email only for valid addresses does not work in our situation.

For many businesses if someone sends a mis-addressed email it gets put into an administrator mailbox. This allows the administrator to forward important messages - especially from customers - who mis-spell someones name.

As an alternative, you could set the Internet Mail Service to not send NDR's.


Larry

[Bob Tucker]

Dear Larry,

I also tested ysing Wingate to allow valid addresses only. As was the case for you, I found that this potential solution does not work for this site. Because we get so much email addressed to bogus users, it is not really practical for anyone here to try to sort through all the mis-addressed and bogus email. For the moment, we do DNS-based open relay and SPAM detection via SORBS, Spamhaus, and SPEWS - along with limited anti-spoof detection so as not to eliminate valid email. It would be good for this site to find a way to select for valid addresses (e.g. look for addresses of the format firstname.lastname@company.com - which is the address format used here or, at least, be able to exclude all the bogus email addresses used - as we capture these). However, all this is way beyond what I would expect the mail server in Wingate to do. I am very impressed at how well the Wingate mail server does what we need it to do. Adrien, if you read this, thank you! We are able to eliminate more than 95% of connections from open relays and SPAM servers before they can send email, so very little bandwidth is used. We will live with what we have for the moment.

One problem that I saw in dealing with this problem is the inability of the Wingate email server to move some messages. This caused problems in the log. It may be useful to know that this problem does not seem to exist if messages are sent directly rather than forwarded. In a related matter, when I try to use the feature of forwarding undeliverable messages only, I find that Wingate attempts to forward such emails 255 times rather than whatever one designatges in the email server dialog beofre moving them to dead.

[adrien]

Hi Bob

I'll take a look into the forwarding count things.

As for banning known spam addresses, this is possible, by adding them to the banlist in the SMTP server policies. we do this at Qbik, and it blocks a bunch of the common worm mails, which use a set dictionary of names to attempt in a domain.

Adrien

[labull]

Hi Adrien,

Where can we find that list of names?

Thanks!

Larry

[peter twartz]

Hi Adrien,
I too suffer from the "stuck email problem".
About once or twice a week an email gets stuck in "postin".
The solution for me is to check the .rcp number in "postin" directory and delete it and the corresponding .msg number in "holding" directory.

Basically I have to monitor "postin" every few days and do the above housekeeping else the SMPT log turns into ten's of MB instead of tens of KB.

I have two short cuts set up on the desk top to "postin" and "holding" however, it is an unnesscary task.

As with Bob Tucker's problem, the email recipient is not a valid email address on this domain, in fact it is usually one of two addresses, one once was a valid address on the domain, the other never was, and to make it more confusing not all mail to those two addresses gets stuck.

I am running Wingate 5.2.3 on a Win 2k box, that is not used for anything else. User names are all from the NT users.
There is no other email server. Wingate email is it.

Following is a sample from the log.

10/15/04 23:50:00 Error: Message 0000040903 could not move .msg file error 2
10/15/04 23:50:00 Error: Message 0000040903 msg file not found, rcp file moved to dead
10/15/04 23:50:01 Error: Message 0000040903 could not move .msg file error 2
10/15/04 23:50:01 Error: Message 0000040903 msg file not found, rcp file moved to dead

Cheers
pt

[Bob Tucker]

Dear Adrien,

My problem is nearly the same as that just dicussed by Peter Twartz. The only difference is that the msg file is sometimes found in Dead, and it sometimes found in Holding. The rcp file is always in Postin. As is the case discussed by Peter Twartz, the problem only occurs with email directed to invalid addresses on the domain. The email involved is small in size.

Kindest Regards,

Bob Tucker

[adrien]

Hi Bob, Peter

Do either of you run other AV software on that server that may be scanning files in the WinGate mail folders? WinGate relies on getting exclusive access to these files whilst they going through from incoming into the holding folders, and AV software which tries to scan these files can mess that up, and cause one or other file to be unable to be opened exclusively.

Adrien

[Bob Tucker]

Dear Adrien,

I run Symantec antivirus 8.1 corporate client on the system, but I exclude the mail directoruies from scan. However, a few viruses have recently slipped through the Wingate AV plugin, so I was going to include the mail directories in the scan. From what you say, I will need to find a plan B. SAV is intrusive, so it is possible it is accessing the msg files. COuld you comment as to what AV one might use in addition to the AV plugin on the Wingate system and how it might be configured?

[adrien]

Hi Bob

It should be fine to scan the "Holding" directory. I would just avoid scanning the incoming or postin directories.

Adrien

[peter twartz]

Hi Adrien,
The only AV software on that machine is the Wingate Kasperski AV for Wingate Plug In.
It scan all traffic in teh fly and I run a "QuickScan" accross the system manually once every week or two.
Cheers
pt

[labull]

In my Spool directory folders I have the following orphans:

Postin - 12 .rcp files - all for legit internal addresses

Incoming - 1 .rcp file

Holding - 6 .msg files

Dead - 18 rcp/msg pairs for NDRs.
7 msg files

Domains is empty so none of these seem to be current.

Some of the .rcp's in Postin match .msg's in Holding

Realtime scanning is turned off on the system.

Larry

[adrien]

OK... that's not good.

The MSG files - do they contain legitimate emails?

Adrien

[peter twartz]

Hi Adrien,
My stuck emails are "legimate emails" in the sense that they are received, and there is a sender and a message, but to the best of my reccolection they have always been spam, and always been to non extistant addresses on my domain.
Cheers
pt

[Bob Tucker]

Dear Adrien,

The msg files that are not moved have always been NDS messages that are generated our Exchange server.

The Exchange(v 5.5) server here receives all email properly addressed to the domain fort the power plant here. The Exchange server issues non-delivery messages for improperly addressed email and sends these messages them via the Wingate email server. I do not recall ever having had a problem with inbound msg files.

The files that are not moved are small. The original spam is typically just a few lines of advert with a URL in a text file. The stuck messages that I have looked at do have a return path. The messages that get stuck are refused on the receiving server. It appers that they get "stuck" at the point they are to be moved to Dead.

Kindest Regards,

Bob Tucker

[labull]

Yes, some in Postin & Holding are.

I copied the Spool folder set and can send it to you if you like.

Larry

[Bob Tucker]

Dear Adrien,

Thank you very much for your assistance on this non-movable message issue. And thank you for your assistance with the SPAM issue. Through the use of aggressive DNS-based SPAM and Open Realy lists, the SPAM issue is under control here. Thank you very much as Wingate handles use of multiple lsits very well. I would like to add request for a feature at some future date regarding SPAM. I had dinner with the regional sales rep for SUN two nights ago, and we discussed the SPAM problem that is currently hitting this part of Asia. The fellow from SUN thinks that this is a concerted effort on the part of a number of very large spammers to get names via Rumplestiltskin attacks. There is no question that these are organized attacks - at least the one at our plant must be - as it comes via thousands of open relays and a large number of servers under the control of a number of known SPAM organizations. It is difficult for me to think of this as a brute force dictionary attack as the names being used in the attack are European family names. How bright would a spammer have to be to realize that a brute foce attack with this particular dictionary will never find common Thai names such as Chaiyarat and Wattana - no matter how much time and energy the spammer might spend? But whatever the source and motivation of the attack, the attack itself depends on the use of open relays that are not yet listed on a number of blacklists. I would like an option to automatically report such open relays to any major blacklist. This might be something that may not be practical at all. If that is the case, forget that I asked. If it would be possible, it might be something for Net Patrol. If there is a method to report to a blacklist now, I would love to do so.

Kindest Regards,

Bob Tucker

[adrien]

Hi Bob

You're welcome on the support front - we are grateful you are all willing to help us fix any problems and improve our product.

As for the open relay issue, there are several options.

1. Add the IP to a banlist, or blackhole the IP in the ENS. If you are never going to accept a mail from that IP, you may as well minimise its use of your resources, the way to do that is add its IP to the ENS black hole.

2. Set up your own RBL. If you have a DNS server under your control, this is actually quite easy, but would require manual addition of the IP addresses. All you need is that the DNS server returns an IP address of 127.0.0.2 for an IP address concatenated to the end of some domain name that you specify. The way the RBLs work in WinGate, which you would be emulating in your DNS server, is that WinGate makes a special DNS lookup.

e.g. if you were checking the IP address 202.14.100.2, and the domain you had control over was say myrbl.com, then the lookup would be for a type A record matching 2.100.14.202.myrbl.com

If this returned 127.0.0.2, then WinGate treats it as an open relay. If you want to supply a reason, create a TXT record for the same name.

the advantage of having your own RBL, is that other mail servers can use it (i.e. secondary server)

Adrien

[Bob Tucker]

Dea Adrien,

I was blackholing all the open relay IPs. According to Spamhaus, a great many of the hosts involved in this are exploitable hosts that have effectively been hijacked. These open relays will be closed in time, so they should not be blackholed. I would like to find a mechanism to submit these open relays a blacklist for testing and lsiting without having to do each such listing manually. Every day, I receive email from a large number of open relays that are not yet listed. If I can submit these for testing promptly, I should be able to limit problems for a good number of people. As you did not speak to how I might list these, I suspect I can automate the process. There are many of these, and the listing process for each takes some time. I find that I do not have time to do it manually.

Kindest Regards,

Bob Tucker

[adrien]

Hi Bob

the problem with automatically submitting these, is that in order to submit automatically, you need to decide that the host is compromised automatically. If WinGate was able to do this then it would be blocking the mail already - it takes a human, or some more advanced heuristic methods.

Are you running AV scanning in WinGate? We find a lot of spams are actually virus emails from infected machines, which can be effectively blocked by antivirus scanning. Could be worth a 30 day trial of the plugin to see if it helps.

Adrien

[Bob Tucker]

Dear Adrien,

Thank you very much for your response. I appreciate your time very much.

I am using the KAV plugin. Anti-spoof detection cut down the number of viruses we see enormously.

The meessages that come to our server through open relays are of the form lastname@company.com. All such messages that reach the email server at the power plant here are bogus. Currently, such messages come by two paths: open relays and indirectly through out backup MX at our ISP. Very shortly, we are changing out backup MX to one that uses aggressive spam and open relay detection. After that, the messages that reach us of the form lastname@company.com will have come directly or indirectly via open relays that are not yet listed. We see a large number of these every day. I was looking for a method to progrmatically report these open relays to a blacklist for testing and listing as I do not have time to do it manually. I found a Perl script for Sendmail that I think may be made to work. I have contacted the author, and he is of the opion that it could work for this purpose. If there is time, I will look into this further.

Thank you again,

Kindest Regards,

Bob Tucker

[Bob Tucker]

Dear Adrien,

I eliminated Symantec Antivirus from the Wingate system. Unfortunately, the "stuck" message problem continues. The stuck message problem does not appear to have been related to SAV in this case.

Kindest Regards,

Bob Tucker

[Bob Tucker]

Dear Adrien,

I mentioned previously that there is a problem with the "Use Gateaway for Undeliverable Mail" switch. Wingate does not deliver to a few email servers. One of these is thaimail.com. They have two MX servers which I can connect to via telnet at port 25 and manually send test messages. For asome reason, this site, and one other, will cnot successfully communicate with our Wingate email server. Therefore, I selected to use our ISP email server as a gateway for undeliverable mail as this problem does not exist with respect to that email server.

Once the Wingate SMTP server has attempted to send the email directly and failed, it attempts to send that email via the gateway as requested. The email server at our ISP rejects some undeliverable email. Although I have checked "block blank return path" in Wingate, our ISP determines some return paths to be invalid; and the email server at out ISP refuses such email. Wingate attempts to send all such email via the gateway 255 times rather than the number of times I have requsted in the email delivery setup screen. Although we are able to send successfully to email servers which will not commmunicate correctly directly with the Wingate SMTP server by forwarding undeliverable email via the "gateway" email server at our ISP, we generate a substantial volume of mail unsuccessful transactions due to non-deliverable emails that cannot be forwarded. In a future version, would it be possible to provide a configuration switch the number of attempts that Wingate will makle to send email via the gateway in the event one has selected to use the gateway in the case of undeliverable mail? I have "attempt immediate delivery" checked. Is this a factor? In a related question. Wingate makes a series of attempts in rapid succession to send email if "attempt immeadiate delivery" is checked. If I uncheck this, how many times per day or at what intervals would Wingate attempt email?