smtp server problem in 6.1.4

[Bob Tucker]

We were using Wingate 6.1.4 at a customer site and have run into a smtp server problem. The Wingate smtp server is used to relay email to valid mailboxes on the Exchange 2003 server on the customer's domain. It appears that some inbound smtp connections to the wingate smtp server terminate prematurely. After a few days, the wingate server in 6.1.4 often would stop sending emails.

Sometimes, I could track this down to an incomplete email that the wingate smtp server was attempting to process. Often the MSG file was missing.

I have reverted to 6.1.3 as this problem occurs less frequently in that version.

Any suggestions?

Kindest Regards,

Bob Tucker

[adrien]

Hi Bob

I'll have a look into what changed.

Do you get domain jobs marked as "sending" in the email queue panel in GateKeeper, but not actually sending (i.e. nothing in the activity panel)?

Adrien

[Bob Tucker]

Hi Adrien,

Thank you for answering.

It does happpen that a domain is sending but nothing happens and the entire mail queue is actually not being processed. You are exactly correct.

When I checked this last time it happened, the msg file that was being sent did not exist, but the rcp existed. The domain job indicated "sending". Email just accumulated in the queue.

In some previous instances, the same thing happened in that the domain was sending but nothing happened. In some cases the msg file did exist but must have been corrupt. In those cases, I stopped Wingate, deleted the msg and rcp of the email that could not be sent, and restarted Wingate. That allowed email to work.

Kindest Regards,

Bob Tucker

[adrien]

Hi Bob

In the couple of times I've seen this recently on our server, there was an associated line in the log file called "Error!!! unhandled exception in ..."

Do you have log entries like this?

We think it's related to an issue creating a new thread, which then leaves the domain job in a "working" state, when in fact it isn't.

Normally the mail will be delivered fine just by restarting WinGate - there should be no need to delete files. However, if you are finding the same thing happening straight away on a restart which doesn't get cleared unless you delete the files, let me know.

Adrien

[Bob Tucker]

Hi Adrien,

I have found that it was necessary to delete the email that is being "sent". I am not ususally at the site invloved, but I now try to check the site remotely at least once each day. Mail usually accumulates on the server if an email is stuck as being "sent" in that fashion, so I need to monitor the server regularly.

I also see vaild email show up in the dead email folder. There are ususally no rcp files. I recreate the rcp files, move the msg files into the holding folder, and put the newly created rcp files into a properly named domain folder for the Exchange server. When an email is sent from the Internet that is properly addressed to a recipient on the Exchange server, the emails that I have moved from the dead folder are then passed properly to the Exchange server.

This is much more a problem in 6.1.4 than in 6.1.3. In fact, with 6.1.3 this happens very infrequently.

Kindest Regards,

Bob Tucker

[adrien]

HI Bob

I can't find any changes in mail between v 6.1.3 and 6.1.4.

We did fix an issue with MSG files being moved to dead. It was related to things holding the file open whilst the RCP file was being processed/moved, which prevented the MSG from being moved. On the next update the MSG was deemed an orphan and moved to dead.

So this could also happen if you have any other processes on your server opening the mail files for any reason (i.e. some other AV system). Do you have any other AV on that machine that could be scanning the mail files? If so, you'd need to configure it to not scan WinGate folders/subfolders.

Regards

Adrien

[Bob Tucker]

Hi Adrien,

I expect you are exactly right. I upgarded the anitvirus software on the network at that company some months ago. I see that mail directores were not excluded in the real-time protection.

Kindest Regards,

Bob Tucker

[Bob Tucker]

Hi Adrien,

I removed all of the Wingate directories from real-time AV protection (SAV 10.x) and returned to Wingate 6.1.4. I still have the problem of an email stopping while being "sent". I was able to restart Wiingate to process the Wingate queue.

Two things that are probably unrelated: (1.) I also noticed that there was an orphaned msg file in holding. I used to see a lot of those. And (2.) there are also a couple of emails that bounce between the Exchange server and Wingate. These eventuially end up dead. I think this has to do with the way some accounts are classified on Exchange and is, therefore, unrelated to Wingate.

Wingate is configured to use the Wingate smtp server to relay email to Exchange on an internal server. I create an address handler for each valid Exchange user in the domain email setup in Wingate, and only email sent to valid Exchange addresses is received in Wingate and passed to Exchange. This appears to be very secure. But there are the problems described. Is there a better way to integrate Wingate and Exchange?

Kindest Regards,

Bob Tucker

[adrien]

Hi Bob

OK, so for the first issue, if you were able to just restart WinGate and it went on and processed the domain job, sounds like a slightly different problem to the one before, and sounds like the issue I recently fixed here in the lab (so should be in the next release).

As for bouncing messages back and forth between Exchange and WinGate, that will happen if each server believes the other is responsible for it. It will just be a matter of finding out what address is involved and most likely requires either

a) adding this address to the exchange server so it will accept the mail and not try and relay it back out to WinGate; or
b) setting up an alias for this address in WinGate so it is relayed to an address that the Exchange server will accept.

WinGate sets a limit on the number of mail hops to stop endless loops, and that's when it will move a mail like that to Dead.

The orphaned MSG file in Holding isn't usually a problem as long as it has actually been delivered. Orphaned MSG files get cleaned up any time WinGate is started.

Regards

Adrien

[Bob Tucker]

Hi Adrien,

Thank you. I think you are right on all counts.

The mail problem does sound like the one you dealt with recently. I will look forward to the next release.

The bouncing problem comes from the way I set up some accounts on Exchange. I fixed that this morning.

And I am very glad to know that orphaned messages in holding do not mean that that the msg was not sent. If the rcp no longer exists, does that normally indicate that the email has been sent?

Kindest Regards,

Bob Tucker

[adrien]

Hi Bob

Yes, the RCP file should only be deleted if the message is sent.

We reference count the MSG file, since it can be referred to by many RCP files. It's possible there is something preventing us from cleaning up the file normally, but it should be sent if the RCP is gone.

Adrien

[Bob Tucker]

Hi Adrien,

I need to ask the question again about msg files that no longer have rcp files associted with them. I have run into a problem where a very important email appears not to have been forwarded into the Exchange server. The msg file for that email remained in the holding folder (and was moved into the dead folder when I restarted Wingate) as the rcp file was no longer in Wingate email.

After your last post on this subject, I understood that the absence of an rcp meant that the email had been forwarded. That appears not to be the case in this instance. My question, then, is does the absence of the rcp file indicate that the email has been sent? Might it also indicate that a problem occured and the email was not forwarded?

In the past, whenever a msg file appeared without a rcp file, I created an rcp file and manually forwarded the email. Is that necessary?

Regards,

Bob Tucker

[adrien]

Hi Bob

the true answer lies in the logs. If you have debug logging on you should be able to track down the message ID if it was moved to dead there should be an entry, and it should show clearly whether any particular mail was sent or not.

Adrien

[Bob Tucker]

Hi Adrien,

Within two weeks, I am going to be handing this site off to another company who will not monitor this process on a day to day basis. So I need to know if there are instances where emails are not forwarded. This used to be a problem with previous versions of Wingate. You will recall that we had a considerable discussion about this a year or two ago in this forum. If the problem still exsits, I will need to stop using the SMTP server in Wingate and used a mapping to forward all email through to Exchange. I would like to continue to use the Wingate mail server as this minimizes traffic on the network, but I can only do so if I can be very sure that there are no errors associated with this. There are a number of msg files that regularly accumulate in holding. So, if there are problems with emails not being forwarded, even rarely, I need to change the way I handle inbound email. Please comment. Even if there are rare problems, I need to know that so that I can set up the service appropriately.

The daily smtp log file is quite large even with debugging enabled. I have just enable debugging. I shudder to think how large this will be.

Kindest Regards,

Bob Tucker

[adrien]

Hi Bob

Understand the need for peace of mind when it comes to mail delivery.

depending on your search facilities in Exchange, you may be able to avoid the debug log option, since the name of the file is also its message ID, which gets stamped into the Received: header when WinGate first accepts the mail. If you can search through your exchange mailstore for messages containing the IDs of messages you have found orphaned in the holding folder, you can then tell that they were delivered if you find any.

There used to be a potential mechanism whereby mail could be moved to Holding, and the RCP end up in dead. This could happen if there was an error creating the new RCP files (under the domain folders) after the MSG file had already been moved to Holding. I thought I'd closed this loophole a while back. Other symptoms of this are that the RCP is moved into the dead folder, rather than disappearing from the face of the earth.

I've double-checked the code, and there should be no circumstance where the RCP file is deleted (i.e. isn't in dead or a domain folder) and the mail hasn't been delivered or delivery hasn't been terminated (i.e too many retries, destination unreachable, invalid recipient etc). In the cases where delivery is terminated, there should be a bounce message created unless the original return-path of the original message was empty. Any bounce messages that can't be delivered get put into Dead as well. If the return path was empty, there will be a log event logged under either Errors or Debug showing "Message %s moved to dead, undeliverable, but no return path" or "Message XXXXX RCP file could not be moved to Dead" however in this last case, we don't delete the RCP file either.


Do you get a lot of files in the Dead folder?

Adrien

[Bob Tucker]

Hi Adrien,

There a few msg files that show up in holding and are later moved to dead each day. Recently, I assumed thaey had been de;ivered. Then a message that had mutilple recipients did not show up in a single user's mailbox.

This user keeps some 4K emails in his inbox in Exchange. He uses an ost file to read emails in his Exchange mailbox when not in the office. The ost file, naturally enough, routinely becomes corrupted. And the guy flunked kindergarten (works and plays well with others). So it is not clear that Wingate is involved. Thus guy is the main reason I am abandoning this account. I am the Wingate vendor. If this guy calls after I leave, I want to be able to say with confidence that emails are not being dropped.

Kindest Regards,

Bob Tucker

[adrien]

Hi bob

Any MSG file fond in Holding that doesn't have an associated RCP file in a domain folder will be moved to dead when Wingate starts, but the real key we need to get to the bottom of is what is happening to the RCP files.

I still can't see anywhere where an RCP file can be deleted without:

1. the mail having been delivered
2. the mail having permanently failed delivery (bounce message sent)

So you don't get any RCP files for these messages in Dead?

PS, I have whittled my inbox down to a paltry 4800 messages... overall have over 700 folders and 200k messages. Using Thunderbird and WinGate IMAP. Not the best IMAP client, but a lot better than Outlook, Eudora and others I've tried.

[Bob Tucker]

Hi Adrien,

You are quite right. The orphaned msg messages are in holding and are moved to dead by Wingate when the service is restarted. There are no associated rcp files.

What you have indicated is that the msg files seem to be orphaned for some reason after having been sent. As that is the case, I will apparently not need to worry about it. That is fine by me. The drive array on the Wingate server is a SATA array with ~200GB of free drive space. A few orphaned msg files would not seem to pose much of a problem.

The Exchange server in this case has almost no available disk space, so I am not able to track emails effecfively. I have to perform a backup daily to get rid of the log files. That server will a problem for the new guy who will manage i, and I will be gald to get rid of it.

Kindest Regards,

Bob Tucker